• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Interactive humanoid robot design using Bunraku puppet structure and Jo-ha-kyu Principle

Research Project

Project/Area Number 21K17833
Research Category

Grant-in-Aid for Early-Career Scientists

Allocation TypeMulti-year Fund
Review Section Basic Section 61050:Intelligent robotics-related
Research InstitutionChukyo University (2023)
Tokyo University of Technology (2021-2022)

Principal Investigator

Dong Ran  中京大学, 工学部, 講師 (80879891)

Project Period (FY) 2021-04-01 – 2024-03-31
Project Status Completed (Fiscal Year 2023)
Budget Amount *help
¥4,550,000 (Direct Cost: ¥3,500,000、Indirect Cost: ¥1,050,000)
Fiscal Year 2023: ¥780,000 (Direct Cost: ¥600,000、Indirect Cost: ¥180,000)
Fiscal Year 2022: ¥650,000 (Direct Cost: ¥500,000、Indirect Cost: ¥150,000)
Fiscal Year 2021: ¥3,120,000 (Direct Cost: ¥2,400,000、Indirect Cost: ¥720,000)
Keywordsロボティクス / 周波数解析 / 深層学習 / 人形浄瑠璃 / ヒューマンロイド / インタラクションデザイン / 伝統芸能 / 序破急 / 文楽人形 / モーション解析・生成 / ノンバーバルコミュニケーション / モーションデザイン / ヒルベルト-ファン変換 / 経験的モード分解 / ロボットインタラクション / 人工知能 / ヒューマンロボットインタラクション / 感情表現
Outline of Research at the Start

人形浄瑠璃文楽は,3人の人形遣いが独特な人形カラクリと人形感情表現プリンシプルを用いて感情表現を行う.人形の所作は義太夫節と呼ばれる音楽(義太夫と三味線使い)に合わせており,序破急メカニズムに基づいて三業一身で舞台を構成する.人形のカラクリ(頭の「円」と呼ばれる動きと伸縮表現など)は,序破急を表現できるように設計されている.本研究では,日本独特の三業から抽出された人形カラクリとその序破急メカニズムをベースに,やわらかい新しいロボット骨格をデザインし,その動作に合わせた人工感情表現の構築を行う.不気味の谷現象を超える人型AIアシスタントのインタラクションデザイン法を確立させる.

Outline of Final Research Achievements

This study aims to apply the movements and emotional expressions of the Japanese traditional puppet theater "Ningyo Joruri Bunraku" to robotics. Using motion capture, we analyzed the puppet's movements and identified techniques for emotional expression. Specifically, we confirmed that the puppet's stretching and chest joint movements synchronize with the voice of the Gidayu (narrative chanting) and the sound of the shamisen (three-stringed instrument), and are related to the mechanism of Jo-Ha-Kyu (a traditional Japanese concept of modulation). We implemented these Bunraku puppet mechanisms into a robot, developing a robot capable of expressing the stretching techniques. Furthermore, we proposed a method combining motion analysis in the frequency domain using the Hilbert-Huang Transform with deep learning, enabling the robot to perform the delicate movements of Bunraku puppets, optimized to the motor performance.

Academic Significance and Societal Importance of the Research Achievements

本研究は,文楽人形のカラクリをロボットの構造に取り入れることで,初めて序破急メカニズムに合わせた伸縮技法を表現できるロボットの開発を試みた.得られた知見は,ロボットインタラクションデザインに日本の伝統芸能の視点から新たなフレームワークを提供できる.また,モーション生成における周波数空間と深層学習の融合は,ヒューマノイドの自立的なインタラクション生成に寄与し,今後,人間社会に導入されるAIアシスタントの普及に貢献できると考えられる.

Report

(4 results)
  • 2023 Annual Research Report   Final Research Report ( PDF )
  • 2022 Research-status Report
  • 2021 Research-status Report
  • Research Products

    (20 results)

All 2023 2022 2021 Other

All Int'l Joint Research (3 results) Journal Article (12 results) (of which Int'l Joint Research: 4 results,  Peer Reviewed: 11 results,  Open Access: 7 results) Presentation (5 results) (of which Int'l Joint Research: 1 results,  Invited: 1 results)

  • [Int'l Joint Research] 吉林大学(中国)

    • Related Report
      2023 Annual Research Report
  • [Int'l Joint Research] chang'an university(中国)

    • Related Report
      2022 Research-status Report
  • [Int'l Joint Research] The Hong Kong Polytechnic University(中国)

    • Related Report
      2021 Research-status Report
  • [Journal Article] Biomechanical Analysis of Golf Swing Motion Using Hilbert-Huang Transform2023

    • Author(s)
      Dong Ran、Ikuno Soichiro
    • Journal Title

      Sensors

      Volume: 23 Issue: 15 Pages: 6698-6698

    • DOI

      10.3390/s23156698

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] The influence of online multiplayer games on social capital and interdependent well-being in Japan2023

    • Author(s)
      Ni Shaowen、Dong Ran、Ueichi Hideo
    • Journal Title

      Entertainment Computing

      Volume: 47 Pages: 100587-100587

    • DOI

      10.1016/j.entcom.2023.100587

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] A real-time recognition gait framework for personal authentication via image-based neural network: accelerated by feature reduction in time and frequency domains2023

    • Author(s)
      Xuan Huang,Ran Dong,Bo Wu,Kiminori Sato,Soichiro Ikuno,Zijun Wang and Shoji Nishimura
    • Journal Title

      Journal of Real-Time Image Processing

      Volume: 20 Issue: 5 Pages: 92-92

    • DOI

      10.1007/s11554-023-01349-w

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Learning multivariate empirical mode decomposition for spectral motion editing2023

    • Author(s)
      Dong Ran、Ikuno Soichiro、Yang Xi
    • Journal Title

      SIGGRAPH Asia 2023 Technical Communications

      Volume: 26 Pages: 1-4

    • DOI

      10.1145/3610543.3626174

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Behavioral Analysis of Mowing Workers Based on Hilbert-Huang Transform: Huang Transform: An Auxiliary Movement Analysis of Manual Mowing on the Slopes of Terraced Rice Fields2023

    • Author(s)
      Wu, B.; Wu, Y.; Dong, R.; Sato, K.; Ikuno, S.; Nishimura, S.; Jin, Q.
    • Journal Title

      Agriculture 2023

      Volume: 13 Issue: 2 Pages: 1-21

    • DOI

      10.3390/agriculture13020489

    • Related Report
      2022 Research-status Report
    • Peer Reviewed / Open Access
  • [Journal Article] Investigating the Effect of Jo-Ha-Kyu on Music Tempos and Kinematics across Cultures: Animation Design for 3D Characters Using Japanese Bunraku Theater2022

    • Author(s)
      Dong Ran、Cai Dongsheng、Hayano Shingo、Nakagawa Shinobu、Ikuno Soichiro
    • Journal Title

      Leonardo

      Volume: 55 Issue: 5 Pages: 468-474

    • DOI

      10.1162/leon_a_02250

    • Related Report
      2022 Research-status Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Analyzing Jo-Ha-Kyu Mechanism in Japanese Traditional Performing Art Ningyo Joruri2022

    • Author(s)
      Dong Ran、Ni Shaowen、Wu Bo、Hayano Shingo、Cai Dongsheng、Ikuno Soichiro
    • Journal Title

      In proceedings of IEEE DASC/PiCom/CBDCom/CyberSciTech 2022

      Volume: 22442370 Pages: 1-5

    • DOI

      10.1109/dasc/picom/cbdcom/cy55231.2022.9927889

    • Related Report
      2022 Research-status Report
    • Peer Reviewed
  • [Journal Article] Pre-braking behaviors analysis based on Hilbert Huang transform2022

    • Author(s)
      Wu Bo、Zhu Yishui、Dong Ran、Sato Kiminori、Ikuno Soichiro、Nishimura Shoji、Jin Qun
    • Journal Title

      CCF Transactions on Pervasive Computing and Interaction

      Volume: 0 Issue: 2 Pages: 0-0

    • DOI

      10.1007/s42486-022-00123-4

    • Related Report
      2022 Research-status Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] A Study of Emotional Expression Through the Movement of Circles in Ningyo Joruri : An Analysis Using Motion Capture2021

    • Author(s)
      Shingo HAYANO, Ran DONG
    • Journal Title

      都留文科大学研究紀要

      Volume: 94 Issue: 94 Pages: 1-10

    • DOI

      10.34356/00000742

    • NAID

      120007015756

    • URL

      https://tsuru.repo.nii.ac.jp/records/751

    • Year and Date
      2021-10-20
    • Related Report
      2021 Research-status Report
    • Peer Reviewed / Open Access
  • [Journal Article] Motion Feature Extraction and Stylization for Character Animation using Hilbert-Huang Transform2021

    • Author(s)
      Dong Ran, Lin Yangfei, Chang Qiong, Zhong Junpei, Cai Dongsheng, Ikuno Soichiro
    • Journal Title

      Proceedings of the 2021 ACM International Conference on Intelligent Computing and its Emerging Applications

      Volume: ACM ICEA '21 Pages: 16-21

    • DOI

      10.1145/3491396.3506524

    • Related Report
      2021 Research-status Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Interacting with Humanoid Robots: Affective Robot Motion Design with 3D Squash and Stretch Using Japanese Jo-ha-kyu Principles in Bunraku2021

    • Author(s)
      Ran Dong, Yuying He, Dongsheng Cai, Jinichi Yamaguchi, Hayato Kondo, Shinobu Nakagawa, Soichiro Ikuno, Shingo Hayano
    • Journal Title

      SIGGRAPH 2021 Talks

      Volume: 1巻 Pages: 45-45

    • DOI

      10.1145/3450623.3464669

    • Related Report
      2021 Research-status Report
    • Peer Reviewed
  • [Journal Article] Development of a humanoid robot with mechanisms to realize exaggerated motions abstracted from a Japanese Bunraku puppet2021

    • Author(s)
      KONDO Hayato, YAMAGUCHI Jin’ichi, DONG Ran, HE Yuying, CAI DongSheng, NAKAGAWA Shinobu
    • Journal Title

      The Proceedings of JSME annual Conference on Robotics and Mechatronics (Robomec)

      Volume: 2021 Issue: 0 Pages: 1P3-E07

    • DOI

      10.1299/jsmermd.2021.1P3-E07

    • NAID

      130008135275

    • ISSN
      2424-3124
    • Related Report
      2021 Research-status Report
  • [Presentation] 生体情報の周波数成分を利用したVRラバーハンド錯覚の検出2023

    • Author(s)
      奥村 颯次郎,董 然,宮崎 慎
    • Organizer
      NICOGRAPH 2023
    • Related Report
      2023 Annual Research Report
  • [Presentation] Nonlinear Motion Analysis, Learning and Synthesis in the Frequency-Domain using Hilbert-Huang Transform2023

    • Author(s)
      Ran Dong
    • Organizer
      The 3rd International Symposium on Intelligence Design (ISID 2023)
    • Related Report
      2022 Research-status Report
    • Int'l Joint Research / Invited
  • [Presentation] モーションデータを用いたEMDによる人型ロボットの動作学習最適化2022

    • Author(s)
      久保 翔太郎,董 然,生野 壮一郎
    • Organizer
      日本応用数理学会 2022年度年会(JSIAM2022)
    • Related Report
      2022 Research-status Report
  • [Presentation] ヒルベルト-ファン変換を適用したゴルフスイングの解析2022

    • Author(s)
      木村 勇大,董 然,生野 壮一郎
    • Organizer
      スポーツ工学・ヒューマンダイナミクス 2022
    • Related Report
      2022 Research-status Report
  • [Presentation] 文楽人形の誇張表現機構を持つ人間型ロボッ トに基づいた感情表現モーションデザイン2021

    • Author(s)
      董 然、蔡 嘉霖、蔡 東生、山口 仁一、近藤 逸人、中川 志信、生野 壮一郎
    • Organizer
      第39回 日本ロボット学会学術講演会
    • Related Report
      2021 Research-status Report

URL: 

Published: 2021-04-28   Modified: 2025-01-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi