Internal model construction by observing others and policy acquisition through self learning

2021 Fiscal Year Final Research Report

• Project Area: Correspondence and Fusion of Artificial Intelligence and Brain Science
• Project/Area Number: 16H06565
• Research Category: Grant-in-Aid for Scientific Research on Innovative Areas (Research in a proposed research area)
• Allocation Type: Single-year Grants
• Review Section: Complex systems
• Research Institution: Kyoto University (2020-2021); Advanced Telecommunications Research Institute International (2016-2019)
• Principal Investigator: Morimoto Jun 京都大学, 情報学研究科, 教授 (10505986)
• Project Period (FY): 2016-06-30 – 2021-03-31
• Keywords: 強化学習 / モデル予測制御 / ヒューマノイドロボット / 非線形最適制御

Outline of Final Research Achievements

We introduced a brain-inspired motor learning framework for humanoid robot control.　Specifically, we developed a computationally efficient Hierarchical Model Predictive Control (HMPC) method for real-time control of humanoid robots. Although MPC is a highly useful approach to deriving a policy for the control of nonlinear dynamical systems, its application to a robot having many degrees of freedom is still a challenging problem because MPC is quite computationally intensive. To cope with this issue, we developed the HMPC method that implements a three-layer hierarchical optimization procedure where a middle layer modulates pre-acquired movement patterns from captured human motions to guide the top-layer exploration and a lower layer generates reactive movements to quickly cope with external disturbances. We evaluated the proposed method on a simulated model and a real humanoid robot.

Free Research Field

ロボティクス・機械学習

Academic Significance and Societal Importance of the Research Achievements

人工知能を用いた画像や音声の認識に関しては飛躍的にその性能が向上している一方で，すばやい身のこなしや，器用な物体の操りなど，ロボットが備える動作生成のための人工知能についてはいまだ人間の方がはるかに高い能力を持つ．特にロボットなど実世界で大量データを取得することが現実的でない応用においては、人間のように限られたサンプル（経験）から学習し目的とする動作を生成する方法が求められている．本研究では, 脳の階層的な学習制御システムを参考に，上記を実現する方法論の開発を目指した．ロボットが実環境で学習し多様な運動を実時間で生成可能となれば，日常生活においてもロボットが人を支える存在となり得る．