Development of a Motion Learning Assist System Combining Visual and Vibration Stimulus -Application to Shamisen Learning Assist-

2022 Fiscal Year Final Research Report

• Project/Area Number: 18K18163
• Research Category: Grant-in-Aid for Early-Career Scientists
• Allocation Type: Multi-year Fund
• Review Section: Basic Section 62030:Learning support system-related
• Research Institution: Ibaraki University (2019-2022); Muroran Institute of Technology (2018)
• Principal Investigator: Shibata Takeshi 茨城大学, 理工学研究科(工学野), 助教 (90649550)
• Project Period (FY): 2018-04-01 – 2023-03-31
• Keywords: 身体動作 / 学習支援 / バーチャルリアリティ / 振動刺激 / 身体動作解析 / マルチモーダル

Outline of Final Research Achievements

Recognizing details of human motion is important for learning skills including musical instruments performance. A learning assist method by visual stimulus has been proposed in the literature; however, this approach cannot sufficiently recognize human motion in 3D space. Moreover, instead of visual stimulus, haptics stimulus is also used for learning assistance. However, the haptics feedback has a probability to impede the motion. Hence, in this study, we proposed a system that provides visual and haptics feedback without impeding the motion.
First, we discussed a fundamental method to represent human motion by haptic stimulus. After that, a motion representation system via visual and vibrational feedback for learning Japanese musical instrument, such as shamisen, was proposed. Our experimental results demonstrated that the combination stimulus of visual and vibrational feedback is better at learning the shamisen motion compared to a previously developed system.

Free Research Field

学習支援システム

Academic Significance and Societal Importance of the Research Achievements

本研究の成果によって，三味線学習の初期に必要となる右腕の三次元的な動きを，視覚と振動を組み合わせて提示可能となり，短い時間で基礎練習を完了してスムーズに曲の練習に移行可能となった．これにより，日本の代表的な民俗芸能である三味線の学習を支援することができる．また，本成果は，三次元的な動きの習得が必要な他の楽器や身体技能の習得にも活用できると考えられる．特に奥行方向の動きを含む動作で，従来の映像による学習が難しい動作においても効果的な学習支援が可能であり，幅広い応用が期待できる．