Spatial recognition based on deep learning and its application to sensory integrated myoelectric hand

2021 Fiscal Year Final Research Report

• Project/Area Number: 19K04296
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Review Section: Basic Section 20020:Robotics and intelligent system-related
• Research Institution: Saga University
• Principal Investigator: Fukuda Osamu 佐賀大学, 理工学部, 教授 (20357891)
• Co-Investigator(Kenkyū-buntansha): 卜 楠 熊本高等専門学校, 電子情報システム工学系AEグループ, 准教授 (80425743) ; 村木 里志 九州大学, 芸術工学研究院, 教授 (70300473)
• Project Period (FY): 2019-04-01 – 2022-03-31
• Keywords: マスタースレーブ / 電動義手 / 深層学習 / 画像認識 / 感覚統合

Outline of Final Research Achievements

In this study, we discussed a novel master-slave control method. The proposed hand was equipped with a vision sensor, an IMU unit, and artificial intelligence based on deep learning technique. The hand can recognize general objects in the environment and estimate their posture, as well as measure the posture of the hand itself. These functions make it possible to perform complicated hand grasping operations that were difficult for conventional master-slave control method. We also tried to design a distinctive neural network that can be trained with multi-modal inputs in end-to-end manner. The validity and effectiveness of the proposed method were verified with experiments using various general objects.

Free Research Field

情報学

Academic Significance and Societal Importance of the Research Achievements

本研究が提案する新しいマスタースレーブの制御方法では，エンドエフェクタ側にセンサやAIを搭載することで，ユーザー側で複雑な操作をすることなく，半自動的に複雑なエンドエフェクタの動作を制御することが可能となる．例えば，この技術を電動義手に導入すれば，これまでは多くても10動作程度に限られていた義手の動作の自由度を，飛躍的に高めることができる．また，複数のセンサ情報を組み合わせて制御を決定する方法は，ユーザの操作意図をより正確に推定することを可能とし，一部のセンサに加わった外乱に対しても，システムを頑強に保つことができる．