Interaction between Humans and Puppets: Humanoid Robot Design using Empirical mode decomposition and Deep learning

2021 Fiscal Year Final Research Report

• Project/Area Number: 20K23352
• Research Category: Grant-in-Aid for Research Activity Start-up
• Allocation Type: Multi-year Fund
• Review Section: 1002:Human informatics, applied informatics and related fields
• Research Institution: Tokyo University of Technology
• Principal Investigator: DONG Ran 東京工科大学, コンピュータサイエンス学部, 助教 (80879891)
• Project Period (FY): 2020-09-11 – 2022-03-31
• Keywords: 経験的モード分解 / ロボット / インタラクションデザイン / 序破急 / 伝統芸能 / 人形浄瑠璃

Outline of Final Research Achievements

In Ningyo Joruri (puppet theater), the puppeteers use an unparalleled method to manipulate one puppet, and their movements have been praised as the most beautiful emotional expressions in the world. This is made possible by the use of a traditional performing technique in which the puppeteers perform the story to synchronize a Japanese musical form called Gidayu-bushi (Gidayu and Shamisen). In this study, we proposed a method for analyzing emotional expressions in the frequency domain from the viewpoint of Jo-Ha-Kyu that are considered important in this technique using the Hilbert-Huang transform. We also developed a robot motion generation framework that adopts the extracted features as training data for deep learning. Furthermore, we were able to apply the method developed in this study for interdisciplinary fusion and contribute to nonlinear problems in different fields.

Free Research Field

インタラクション

Academic Significance and Societal Importance of the Research Achievements

本研究では，初めて「音×動き×序破急メカニズム」の視点から，伝統芸能の動きを周波数領域空間で解析を行った．序破急という非決定論的日本伝統芸能メカニズムを用いることにより，人工情動知能アシスタントと連動できるコミュニケーションAIのインタラクション手法の確立に寄与できる．本研究が実施した伝統芸能からのロボット創造は，現在世界に注目されている日本文化や日本的感性のテクノロジーを世界へ発信できる学術研究として期待できると同時に，これまでのAIアシスタントの普及を妨げてきた，対人感覚の欠如を改善し，利用の広がりを飛躍的に進める可能性がある．