• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Internal model construction by observing others and policy acquisition through self learning

Planned Research

Project AreaCorrespondence and Fusion of Artificial Intelligence and Brain Science
Project/Area Number 16H06565
Research Category

Grant-in-Aid for Scientific Research on Innovative Areas (Research in a proposed research area)

Allocation TypeSingle-year Grants
Review Section Complex systems
Research InstitutionKyoto University (2020-2021)
Advanced Telecommunications Research Institute International (2016-2019)

Principal Investigator

Morimoto Jun  京都大学, 情報学研究科, 教授 (10505986)

Project Period (FY) 2016-06-30 – 2021-03-31
Project Status Completed (Fiscal Year 2021)
Budget Amount *help
¥69,550,000 (Direct Cost: ¥53,500,000、Indirect Cost: ¥16,050,000)
Fiscal Year 2020: ¥13,650,000 (Direct Cost: ¥10,500,000、Indirect Cost: ¥3,150,000)
Fiscal Year 2019: ¥13,650,000 (Direct Cost: ¥10,500,000、Indirect Cost: ¥3,150,000)
Fiscal Year 2018: ¥13,650,000 (Direct Cost: ¥10,500,000、Indirect Cost: ¥3,150,000)
Fiscal Year 2017: ¥13,650,000 (Direct Cost: ¥10,500,000、Indirect Cost: ¥3,150,000)
Fiscal Year 2016: ¥14,950,000 (Direct Cost: ¥11,500,000、Indirect Cost: ¥3,450,000)
Keywords強化学習 / モデル予測制御 / ヒューマノイドロボット / 非線形最適制御 / ロボティクス / 機械学習
Outline of Final Research Achievements

We introduced a brain-inspired motor learning framework for humanoid robot control. Specifically, we developed a computationally efficient Hierarchical Model Predictive Control (HMPC) method for real-time control of humanoid robots. Although MPC is a highly useful approach to deriving a policy for the control of nonlinear dynamical systems, its application to a robot having many degrees of freedom is still a challenging problem because MPC is quite computationally intensive. To cope with this issue, we developed the HMPC method that implements a three-layer hierarchical optimization procedure where a middle layer modulates pre-acquired movement patterns from captured human motions to guide the top-layer exploration and a lower layer generates reactive movements to quickly cope with external disturbances. We evaluated the proposed method on a simulated model and a real humanoid robot.

Academic Significance and Societal Importance of the Research Achievements

人工知能を用いた画像や音声の認識に関しては飛躍的にその性能が向上している一方で,すばやい身のこなしや,器用な物体の操りなど,ロボットが備える動作生成のための人工知能についてはいまだ人間の方がはるかに高い能力を持つ.特にロボットなど実世界で大量データを取得することが現実的でない応用においては、人間のように限られたサンプル(経験)から学習し目的とする動作を生成する方法が求められている.本研究では, 脳の階層的な学習制御システムを参考に,上記を実現する方法論の開発を目指した.ロボットが実環境で学習し多様な運動を実時間で生成可能となれば,日常生活においてもロボットが人を支える存在となり得る.

Report

(6 results)
  • 2021 Final Research Report ( PDF )
  • 2020 Annual Research Report
  • 2019 Annual Research Report
  • 2018 Annual Research Report
  • 2017 Annual Research Report
  • 2016 Annual Research Report
  • Research Products

    (48 results)

All 2022 2021 2020 2019 2018 2017 2016 Other

All Int'l Joint Research (4 results) Journal Article (19 results) (of which Int'l Joint Research: 7 results,  Peer Reviewed: 17 results,  Open Access: 13 results) Presentation (22 results) (of which Int'l Joint Research: 16 results,  Invited: 5 results) Book (1 results) Patent(Industrial Property Rights) (2 results)

  • [Int'l Joint Research] ヨージェフ・ステファン研究所(スロベニア)

    • Related Report
      2020 Annual Research Report
  • [Int'l Joint Research] ヨージェフ・ステファン研究所(スロベニア)

    • Related Report
      2019 Annual Research Report
  • [Int'l Joint Research] ヨージェフ・ステファン研究所(スロベニア)

    • Related Report
      2018 Annual Research Report
  • [Int'l Joint Research] Jozef Stefan Institute(スロベニア)

    • Related Report
      2016 Annual Research Report
  • [Journal Article] Implicit Contact Dynamics Modeling With Explicit Inertia Matrix Representation for Real-Time, Model-Based Control in Physical Environment2022

    • Author(s)
      Takeshi D. Itoh, Koji Ishihara, Jun Morimoto
    • Journal Title

      Neural Computation

      Volume: 34 Issue: 2 Pages: 360-377

    • DOI

      10.1162/neco_a_01465

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Deep learning, reinforcement learning, and world models2022

    • Author(s)
      Yutaka Matsuo, Yann LeCun, Maneesh Sahani, Doina Precup, David Silver, Masashi Sugiyama, Eiji Uchibe, Jun Morimoto
    • Journal Title

      Neural Networks

      Volume: 152 Pages: 267-275

    • DOI

      10.1016/j.neunet.2022.03.037

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Composing an Assistive Control Strategy Based on Linear Bellman Combination From Estimated User's Motor Goal2021

    • Author(s)
      Furukawa Jun-ichiro、Morimoto Jun
    • Journal Title

      IEEE Robotics and Automation Letters

      Volume: 6 Issue: 2 Pages: 1051-1058

    • DOI

      10.1109/lra.2021.3051562

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] A Collaborative Filtering Approach Toward Plug-and-Play Myoelectric Robot Control2021

    • Author(s)
      Jun-ichiro Furukawa, Shinya Chiyohara, Tatsuya Teramae, Asuka Takai, Jun Morimoto
    • Journal Title

      IEEE Transactions on Human-Machine Systems

      Volume: 51 Issue: 5 Pages: 514-523

    • DOI

      10.1109/thms.2021.3098115

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Parallel and hierarchical neural mechanisms for adaptive and predictive behavioral control2021

    • Author(s)
      Tom Macpherson, Masayuki Matsumoto, Hiroaki Gomi, Jun Morimoto, Eiji Uchibe, Takatoshi Hikida
    • Journal Title

      Neural Networks

      Volume: 144 Pages: 507-521

    • DOI

      10.1016/j.neunet.2021.09.009

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Whole-body Optimal Control using a Full-body Dynamics Model2021

    • Author(s)
      石原 弘二, 森本 淳
    • Journal Title

      Journal of the Robotics Society of Japan

      Volume: 39 Issue: 7 Pages: 597-600

    • DOI

      10.7210/jrsj.39.597

    • NAID

      130008083625

    • ISSN
      0289-1824, 1884-7145
    • Related Report
      2020 Annual Research Report
  • [Journal Article] Phase portraits as movement primitives for fast humanoid robot controls2020

    • Author(s)
      Guilherme Maeda, Okan Koc, Jun Morimoto
    • Journal Title

      Neural Networks

      Volume: 129 Pages: 109-122

    • DOI

      10.1016/j.neunet.2020.04.007

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Training of deep neural networks for the generation of dynamic movement primitives2020

    • Author(s)
      Rok Pathic, Barry Ridge, Andrej Gams, Jun Morimoto, Ales Ude
    • Journal Title

      Neural Networks

      Volume: 127 Pages: 121-131

    • DOI

      10.1016/j.neunet.2020.04.010

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Quaternion-Based Trajectory Optimization of Human Postures for Inducing Target Muscle Activation Patterns2020

    • Author(s)
      Tatsuya Teramae, Takamitsu Matsubara, Tomoyuki Noda, Jun Morimoto
    • Journal Title

      IEEE Robotics and Automation Letters

      Volume: 5 Issue: 4 Pages: 6607-6614

    • DOI

      10.1109/lra.2020.3015460

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Stable control of force, position, and stiffness for robot joints powered via pneumatic muscles2019

    • Author(s)
      Barkan UGURLU,Paolo FORNI,Corinne DOPPMANN,Emre SARIYILIZ,Jun MORIMOTO
    • Journal Title

      IEEE Transactions on Industrial Informatics

      Volume: 15 Issue: 12 Pages: 6270-6279

    • DOI

      10.1109/tii.2019.2916228

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Assistive arm-exoskeleton control based on human muscular manipulability2019

    • Author(s)
      Tadej PETRIC,Luka PETERNEL,Jun MORIMOTO,Jan BABIC
    • Journal Title

      Frontiers in Neurorobotics

      Volume: 13

    • DOI

      10.3389/fnbot.2019.00030

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Design of physical user-robot interactions for model identification of soft actuators on exoskeleton robots2019

    • Author(s)
      Masashi HAMAYA,Takamitsu MATSUBARA,Tatsuya TERAMAE,Tomoyuki NODA,Jun MORIMOTO
    • Journal Title

      International Journal of Robotics Research

      Volume: - Issue: 1 Pages: 1-14

    • DOI

      10.1177/0278364919853618

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Full-body optimal control toward versatile and agile behaviors in a humanoid robot2019

    • Author(s)
      Koji ISHIHARA,Takeshi D. ITOH,Jun MORIMOTO
    • Journal Title

      IEEE Robotics and Automation Letters

      Volume: 15 Issue: 1 Pages: 119-126

    • DOI

      10.1109/lra.2019.2947001

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] A Gaussian Process-Based Active Learning Approach for Exoskeleton Robots2018

    • Author(s)
      濱屋 政志、松原 崇充、森本 淳
    • Journal Title

      SYSTEMS, CONTROL AND INFORMATION

      Volume: 62 Issue: 10 Pages: 423-428

    • DOI

      10.11509/isciesci.62.10_423

    • NAID

      130007632587

    • ISSN
      0916-1600, 2424-1806
    • Year and Date
      2018-10-15
    • Related Report
      2018 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] 特集 社会性と脳 ヒト動作の模倣によるヒト型ロボット動作学習2018

    • Author(s)
      森本淳
    • Journal Title

      生体の科学

      Volume: 69 Issue: 1 Pages: 51-53

    • DOI

      10.11477/mf.2425200754

    • ISSN
      0370-9531, 1883-5503
    • Year and Date
      2018-02-15
    • Related Report
      2017 Annual Research Report
  • [Journal Article] Human-in-the-loop control and task learning for pneumatically actuated muscle based robots2018

    • Author(s)
      Tatsuya TERAMAE,Koji ISHIHARA,Jan BABIC,Jun MORIMOTO,Erhan OZTOP
    • Journal Title

      Frontiers in Human-in-the-Loop Robot Control and Learning

      Volume: 12 Pages: 71-71

    • DOI

      10.3389/fnbot.2018.00071

    • Related Report
      2018 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] An optimal control strategy for hybrid actuator systems: Application to an artificial muscle with electric motor assist2018

    • Author(s)
      Koji Ishihara, Jun Morimoto
    • Journal Title

      Neural Networks

      Volume: 99 Pages: 92-100

    • DOI

      10.1016/j.neunet.2017.12.010

    • Related Report
      2017 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Skill learning and action recognition by arc-length dynamic movement primitives2017

    • Author(s)
      T.Gaspar, B.Nemec, J.Morimoto, A.Ude
    • Journal Title

      Robotics and Autonomous Systems

      Volume: 100 Pages: 225-235

    • DOI

      10.1016/j.robot.2017.11.012

    • Related Report
      2017 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Soft humanoid motor learning2017

    • Author(s)
      Jun Morimoto
    • Journal Title

      Science Robotics

      Volume: 2 Issue: 13

    • DOI

      10.1126/scirobotics.aaq0989

    • Related Report
      2017 Annual Research Report
    • Peer Reviewed
  • [Presentation] Computationally Affordable Hierarchical Framework for Humanoid Robot Control2021

    • Author(s)
      Koji Ishihara, Jun Morimoto
    • Organizer
      2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
    • Related Report
      2020 Annual Research Report
    • Int'l Joint Research
  • [Presentation] MPC for humanoid control2020

    • Author(s)
      Koji Ishihara, Jun Morimoto
    • Organizer
      Robotics Science and Systems (RSS) Workshop, Robotics Retrospectives
    • Related Report
      2020 Annual Research Report
  • [Presentation] 階層モデルを用いた人型ロボットの運動生成2020

    • Author(s)
      石原弘二
    • Organizer
      異分野融合ワークショップ「脳型情報処理によるロボットラーニングの技術革新」
    • Related Report
      2019 Annual Research Report
    • Invited
  • [Presentation] Motor learning algorithms2019

    • Author(s)
      Jun MORIMOTO
    • Organizer
      Skolkovo Robotics 2019
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] Learning to write anywhere with spatial transformer image-to-motion encoder-decoder networks2019

    • Author(s)
      Barry RIDGE,Rok PAHIC,Ales UDE,Jun MORIMOTO
    • Organizer
      IEEE International Conference on Robotics and Automation(ICRA2019)
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Exploiting human and robot muscle synergies for human-in-the-loop optimization of EMG-based assistive strategies2019

    • Author(s)
      Masashi HAMAYA,Takamitsu MATSUBARA,Jun-ichiro FURUKAWA,Yuting SUN,Satoshi YAGI,Tatsuya TERAMAE,Tomoyuki NODA,Jun MORIMOTO
    • Organizer
      IEEE International Conference on Robotics and Automation(ICRA2019)
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Convolutional encoder-decoder networks for robust image-to-motion prediction2019

    • Author(s)
      Barry RIDGE,Rok PAHIC,Ales UDE,Jun MORIMOTO
    • Organizer
      28th International Conference on Robotics in Alpe-Adria-Danube Region(RAAD2019)
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 状態予測に基づくロボットの階層的な運動制御2019

    • Author(s)
      森本淳
    • Organizer
      第13回Motor Control研究会
    • Related Report
      2019 Annual Research Report
    • Invited
  • [Presentation] Model-based approaches to humanoid motor learning2019

    • Author(s)
      Jun MORIMOTO
    • Organizer
      5th CiNet Conference, Computation and Representation in Brains and Machines
    • Related Report
      2018 Annual Research Report
    • Invited
  • [Presentation] Estimating humerus head elevation in human shoulder joints via gaussian process regression2018

    • Author(s)
      Sara HAMDAN,Erhan OZTOP,Jun-ichiro FURUKAWA,Jun MORIMOTO,Barkan UGURLU
    • Organizer
      Turkey Robotics Conference(TORK2018)
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Robotizing double-bar ankle-foot orthosis2018

    • Author(s)
      Tomoyuki NODA,Asuka TAKAI,Tatsuya TERAMAE,Eiko HIROOKA,Kimitaka HASE,Jun MORIMOTO
    • Organizer
      IEEE International Conference on Robotics and Automation(ICRA2018)
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Deep encoder-decoder networks for mapping raw images to dynamic movement primitives2018

    • Author(s)
      Rok PAHIC,Andrej GAMS,Ales UDE,Jun MORIMOTO
    • Organizer
      IEEE International Conference on Robotics and Automation(ICRA2018)
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Shoulder glenohumeral elevation estimation based on upper arm orientation2018

    • Author(s)
      Sara HAMDAN,,Erhan OZTOP,Jun-ichiro FURUKAWA,,Jun MORIMOTO,Barkan UGURLU
    • Organizer
      40th International Conference of the IEEE Engineering in Medicine and Biology Society
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Bayesian optimisation of exoskeleton design parameters2018

    • Author(s)
      Daniel F.N. GORDON,Takamitsu MATSUBARA,Tomoyuki NODA,Tatsuya TERAMAE,Jun MORIMOTO,Sethu VIJAYAKUMAR
    • Organizer
      7th IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics(BioRob 2018)
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] A data-driven approach for estimating human behavior with collaborative filtering2018

    • Author(s)
      Jun-ichiro FURUKAWA,Jun MORIMOTO
    • Organizer
      Joint workshop of UCL-ICN, NTT, UCL-Gatsby, and AIBS
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Humanoid motor learning from observed human behaviors2018

    • Author(s)
      Jun MORIMOTO
    • Organizer
      Joint Workshop of UCL-ICN,NTT,UCL-Gatsby,and AIBS
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Reinforcement learning of phase oscillators for fast adaptation to moving targets2018

    • Author(s)
      Guilherme MAEDA,Okan KOC,Jun MORIMOTO
    • Organizer
      Conference on Robot Learning(CoRL2018)
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] ユーザ嗜好に基づくEMGを用いた運動支援制御器のベイズ最適化2018

    • Author(s)
      濱屋政志, 松原崇充, 古川淳一朗, 孫雨庭, 八木聡明, 寺前達也, 野田智之, 森本淳
    • Organizer
      第19回計測自動制御学会システムインテグレーション部門講演会(SI2018)
    • Related Report
      2018 Annual Research Report
  • [Presentation] A forward and inverse optimal control framework to generate humanoid robot movements with hierarchical MPC2017

    • Author(s)
      K.Ishihara,J.Furukawa, J.Morimoto
    • Organizer
      3rd Multidisciplinary Conference on Reinforcement Learning and Decision Making(RLDM2017)
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Trajectory representation by nonlinear scaling of dynamic movement primitives2016

    • Author(s)
      A.Ude, R.Vuga, B.Nemec, J.Morimoto
    • Organizer
      IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS2016)
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Presentation] ヒューマノイドロボットの運動学習2016

    • Author(s)
      森本淳
    • Organizer
      第16回Kフォーラムざっくばらんフォーラム「情報学からの価値創造」
    • Related Report
      2016 Annual Research Report
    • Invited
  • [Presentation] A hierarchical model predictive control approach to generate biped robot movements in real-time2016

    • Author(s)
      K.Ishihara, J.Morimoto
    • Organizer
      IEEE International Conference on Simulation, Modeling, and Programming for Autonomous Robots(SIMPAR2016) Workshop
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Book] これからの強化学習 「高次元・実環境における強化学習」2016

    • Author(s)
      森本淳、杉本徳和
    • Total Pages
      9
    • Publisher
      森北出版
    • ISBN
      4627880316
    • Related Report
      2016 Annual Research Report
  • [Patent(Industrial Property Rights)] ロボット制御装置、ロボット制御方法、および、プログラム2021

    • Inventor(s)
      石原 弘二、森本 淳
    • Industrial Property Rights Holder
      石原 弘二、森本 淳
    • Industrial Property Rights Type
      特許
    • Industrial Property Number
      2021-096353
    • Filing Date
      2021
    • Related Report
      2020 Annual Research Report
  • [Patent(Industrial Property Rights)] 制御システム2019

    • Inventor(s)
      伊藤健史、石原弘二、森本淳
    • Industrial Property Rights Holder
      伊藤健史、石原弘二、森本淳
    • Industrial Property Rights Type
      特許
    • Filing Date
      2019
    • Related Report
      2019 Annual Research Report

URL: 

Published: 2016-07-04   Modified: 2023-01-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi