• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Spatial recognition based on deep learning and its application to sensory integrated myoelectric hand

Research Project

Project/Area Number 19K04296
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeMulti-year Fund
Section一般
Review Section Basic Section 20020:Robotics and intelligent system-related
Research InstitutionSaga University

Principal Investigator

Fukuda Osamu  佐賀大学, 理工学部, 教授 (20357891)

Co-Investigator(Kenkyū-buntansha) 卜 楠  熊本高等専門学校, 電子情報システム工学系AEグループ, 准教授 (80425743)
村木 里志  九州大学, 芸術工学研究院, 教授 (70300473)
Project Period (FY) 2019-04-01 – 2022-03-31
Project Status Completed (Fiscal Year 2021)
Budget Amount *help
¥4,290,000 (Direct Cost: ¥3,300,000、Indirect Cost: ¥990,000)
Fiscal Year 2021: ¥1,430,000 (Direct Cost: ¥1,100,000、Indirect Cost: ¥330,000)
Fiscal Year 2020: ¥1,300,000 (Direct Cost: ¥1,000,000、Indirect Cost: ¥300,000)
Fiscal Year 2019: ¥1,560,000 (Direct Cost: ¥1,200,000、Indirect Cost: ¥360,000)
Keywordsマスタースレーブ / 電動義手 / 深層学習 / 画像認識 / 感覚統合 / 義手 / インタフェース / 空間認知
Outline of Research at the Start

筋電義手は,実空間において人間と機械とが複雑な操作を要求されるマスタースレーブの好例である.本研究では,スレーブ側に複数のセンサとAI(深層学習)を導入し,マスター操作を高度に支援する新たな制御方法を提案する.スレーブ側の義手は,深層学習を利用して,どこに,何が,どのように存在するかを空間認識するとともに,カメラ,加速度・ジャイロセンサなどからの感覚を統合し,自身の状態(姿勢や位置など)を把握する.マスター側では,深層学習を利用して操作者の運動意図を筋電位信号から推定し,これをスレーブ側の情報と統合して制御に利用する.構築したシステムで実践的な評価実験を実施する.

Outline of Final Research Achievements

In this study, we discussed a novel master-slave control method. The proposed hand was equipped with a vision sensor, an IMU unit, and artificial intelligence based on deep learning technique. The hand can recognize general objects in the environment and estimate their posture, as well as measure the posture of the hand itself. These functions make it possible to perform complicated hand grasping operations that were difficult for conventional master-slave control method. We also tried to design a distinctive neural network that can be trained with multi-modal inputs in end-to-end manner. The validity and effectiveness of the proposed method were verified with experiments using various general objects.

Academic Significance and Societal Importance of the Research Achievements

本研究が提案する新しいマスタースレーブの制御方法では,エンドエフェクタ側にセンサやAIを搭載することで,ユーザー側で複雑な操作をすることなく,半自動的に複雑なエンドエフェクタの動作を制御することが可能となる.例えば,この技術を電動義手に導入すれば,これまでは多くても10動作程度に限られていた義手の動作の自由度を,飛躍的に高めることができる.また,複数のセンサ情報を組み合わせて制御を決定する方法は,ユーザの操作意図をより正確に推定することを可能とし,一部のセンサに加わった外乱に対しても,システムを頑強に保つことができる.

Report

(4 results)
  • 2021 Annual Research Report   Final Research Report ( PDF )
  • 2020 Research-status Report
  • 2019 Research-status Report
  • Research Products

    (32 results)

All 2022 2021 2020 2019

All Journal Article (11 results) (of which Int'l Joint Research: 5 results,  Peer Reviewed: 11 results,  Open Access: 6 results) Presentation (18 results) (of which Int'l Joint Research: 1 results,  Invited: 5 results) Patent(Industrial Property Rights) (3 results)

  • [Journal Article] Bimodal control of a walk-assist robot for a person tracking2022

    • Author(s)
      Ryosuke MIYATA, Yeoh Wen Liang, Osamu FUKUDA, Nobuhiko YAMAGUCHI, Hiroshi OKUMURA
    • Journal Title

      Proceedings of the 26th International Symposium on Artificial Life and Robotics

      Volume: 1

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Object Search Using Edge-AI Based Mobile Robot2021

    • Author(s)
      Miyata Ryosuke、Fukuda Osamu、Yamaguchi Nobuhiko、Okumura Hiroshi
    • Journal Title

      2021 6th International Conference on Intelligent Informatics and Biomedical Sciences (ICIIBMS)

      Volume: 1 Pages: 198-203

    • DOI

      10.1109/iciibms52876.2021.9651591

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Object Searching Robot Controlled by Edge-AI2021

    • Author(s)
      Miyata Ryosuke、Fukuda Osamu、Yamaguchi Nobuhiko、Okumura Hiroshi
    • Journal Title

      Journal of Information and Communication Engineering (JICE)

      Volume: 7 Pages: 455-461

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Bimodal Control of a Vision-Based Myoelectric Hand2021

    • Author(s)
      Fukuda Osamu、Sakaguchi Daisuke、He Yunan、Yamaguchi Nobuhiko、Okumura Hiroshi
    • Journal Title

      IEEE Access

      Volume: 9 Pages: 98369-98380

    • DOI

      10.1109/access.2021.3096040

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Task assistance with human-augmented hand and its performance analysis2021

    • Author(s)
      Shunji MIYAGAWA, Osamu FUKUDA Nobuhiko YAMAGUCHI and Hiroshi OKUMURA, and Anik Nur Handayani
    • Journal Title

      Proceedings of the 7th International Conference on Electrical, Electronics and Information Engineering(ICEEIE)

      Volume: 1 Pages: 390-394

    • DOI

      10.1109/iceeie52663.2021.9616867

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Vision-based Scene Recognition for Product Search2021

    • Author(s)
      Takayanagi Miho、Fukuda Osamu、Yamaguchi Nobuhiko、Okumura Hiroshi、Handayani Anik Nur
    • Journal Title

      Proceedings of the 7th International Conference on Electrical, Electronics and Information Engineering(ICEEIE)

      Volume: 1 Pages: 1-5

    • DOI

      10.1109/iceeie52663.2021.9616880

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Noble Control Scheme for Prosthetic Hands through Spatial Understanding2020

    • Author(s)
      He Yunan、Fukuda Osamu、Yamaguchi Nobuhiko、Okumura Hiroshi、Arai Kohei
    • Journal Title

      International Journal of Advanced Computer Science and Applications

      Volume: 11 Issue: 10 Pages: 719-725

    • DOI

      10.14569/ijacsa.2020.0111088

    • Related Report
      2020 Research-status Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Vision-Based Assistance for Myoelectric Hand Control2020

    • Author(s)
      He Yunan、Kubozono Ryusuke、Fukuda Osamu、Yamaguchi Nobuhiko、Okumura Hiroshi
    • Journal Title

      IEEE Access

      Volume: 8 Pages: 201956-201965

    • DOI

      10.1109/access.2020.3036115

    • Related Report
      2020 Research-status Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Vision-Based Robot Hand Using Open Source Software2020

    • Author(s)
      Kubozono Ryusuke、He Yunan、Fukuda Osamu、Yamaguchi Nobuhiko、Okumura Hiroshi、Handayani Anik Nur
    • Journal Title

      Proceedings of the 4th International Conference on Vocational Education and Training (ICOVET2020)

      Volume: 1 Pages: 180-184

    • DOI

      10.1109/icovet50258.2020.9230275

    • Related Report
      2020 Research-status Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Development of a Practical Tool in Pick-and-Place Tasks for Human Workers2020

    • Author(s)
      Yunan HE, Osamu FUKUDA, Daisuke SAKAGUCHI, Nobuhiko YAMAGUCHI and Hiroshi OKUMURA
    • Journal Title

      International Journal of Advanced Computer Science and Applications

      Volume: 11 Issue: 4

    • DOI

      10.14569/ijacsa.2020.01104101

    • Related Report
      2019 Research-status Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Development of distributed control system for vision-based myoelectric prothetic hand2019

    • Author(s)
      Yunan HE, Ryusei SHIMA, Osamu FUKUDA, Nan BU, Nobuhiko YAMAGUCHI and Hiroshi OKUMURA
    • Journal Title

      IEEE ACCESS

      Volume: 7 Pages: 54542-54549

    • DOI

      10.1109/access.2019.2911968

    • Related Report
      2019 Research-status Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Presentation] 商品探索を目的としたシーン認識2021

    • Author(s)
      高柳美保,福田修,奥村浩,山口暢彦
    • Organizer
      第74回電気・情報関係学会九州支部連合大会
    • Related Report
      2021 Annual Research Report
  • [Presentation] エッジAIに基づく移動ロボットの開発2021

    • Author(s)
      宮田竜輔,福田修,奥村浩,山口暢彦
    • Organizer
      第74回電気・情報関係学会九州支部連合大会
    • Related Report
      2021 Annual Research Report
  • [Presentation] 人間拡張ハンドによる作業支援とパフォーマンスの解析2021

    • Author(s)
      宮川俊二,福田修,奥村浩,山口暢彦
    • Organizer
      第74回電気・情報関係学会九州支部連合大会
    • Related Report
      2021 Annual Research Report
  • [Presentation] 人工の手とAI ーロボットハンドを使って人工知能を育てるー2021

    • Author(s)
      福田修
    • Organizer
      第52回サイエンスカフェ in SAGA
    • Related Report
      2021 Annual Research Report
    • Invited
  • [Presentation] スマートエアリアルハンドの実現に向けた深度カメラに基づく把持動作制御2020

    • Author(s)
      窪園隆介,福田修,中山 功一,大島千佳,林喜章
    • Organizer
      2020年度(第73回)電気・情報関係学会九州支部連合大会
    • Related Report
      2020 Research-status Report
  • [Presentation] Implementation of artificial intelligence on image processing, Keynote speach2020

    • Author(s)
      Osamu FUKUDA
    • Organizer
      Guest lecture organized by Departmant of Electorical Engineering in Malang, Indonesia
    • Related Report
      2020 Research-status Report
    • Int'l Joint Research / Invited
  • [Presentation] AIロボットを知ろう2020

    • Author(s)
      福田修,村上玲奈
    • Organizer
      佐賀県立鳥栖高等学校 ダイバーシティ入門講座付き実験体験会
    • Related Report
      2020 Research-status Report
    • Invited
  • [Presentation] ロボットハンドと人工知能2020

    • Author(s)
      福田修
    • Organizer
      鳥栖工業高等学校 講演
    • Related Report
      2020 Research-status Report
    • Invited
  • [Presentation] Prosthetic hand control system based on object matching and tracking2019

    • Author(s)
      Yunan HE, Osamu FUKUDA, Nan BU, Nobuhiko YAMAGUCHI, Hiroshi OKUMURA
    • Organizer
      ロボティクス・メカトロニクス講演会
    • Related Report
      2019 Research-status Report
  • [Presentation] 一般対象物認識を利用した筋電位信号からの 5指動作認識2019

    • Author(s)
      坂口大介,何宇楠,福田修,山口暢彦,奥村浩
    • Organizer
      ロボティクス・メカトロニクス講演会
    • Related Report
      2019 Research-status Report
  • [Presentation] 搭載カメラを利用した筋電義手の操作意図推定2019

    • Author(s)
      窪園隆介,何宇楠,福田修,山口暢彦,奥村浩
    • Organizer
      ロボティクス・メカトロニクス講演会
    • Related Report
      2019 Research-status Report
  • [Presentation] Smart hand: grasp objects based on spatial awareness2019

    • Author(s)
      Yunan HE, Osamu FUKUDA, Nan BU, Nobuhiko YAMAGUCHI, Hiroshi OKUMURA
    • Organizer
      第37回日本ロボット学会学術講演会
    • Related Report
      2019 Research-status Report
  • [Presentation] Vision-based Sentence Extraction from Hand-object Interaction2019

    • Author(s)
      Clement ROUX, Yunan HE, Osamu FUKUDA, Nobuhiko YAMAGUCHI, Hiroshi OKUMURA
    • Organizer
      The 72th Joint Conference of Electrical, Electronics and Information Engineering
    • Related Report
      2019 Research-status Report
  • [Presentation] 異常検知における画像生成技術の有効性2019

    • Author(s)
      村上 玲奈,福田 修,山口 暢彦,奥村 浩
    • Organizer
      電気・情報関係学会九州支部第72回連合大会
    • Related Report
      2019 Research-status Report
  • [Presentation] 一般対象物認識と筋電位信号に基づく5指動作認識2019

    • Author(s)
      坂口 大介,何 宇楠,福田 修,山口 暢彦,奥村 浩
    • Organizer
      電気・情報関係学会九州支部第72回連合大会
    • Related Report
      2019 Research-status Report
  • [Presentation] 義手搭載カメラによる物体認識と把持対象の選択2019

    • Author(s)
      窪園 隆介,何 宇楠,福田 修,山口 暢彦,奥村 浩
    • Organizer
      電気・情報関係学会九州支部第72回連合大会
    • Related Report
      2019 Research-status Report
  • [Presentation] 見て・考えて・動作するスマートハンド2019

    • Author(s)
      福田 修
    • Organizer
      大学・高専・スタートアップ発! Meet ユニークテクノロジー,一般財団法人高度技術社会推進協会(TEPIA)
    • Related Report
      2019 Research-status Report
    • Invited
  • [Presentation] 見て・考えて・動作するスマートハンド2019

    • Author(s)
      福田 修
    • Organizer
      イノベーション・ジャパン2019 ~大学見本市&ビジネスマッチング~,科学技術振興機構
    • Related Report
      2019 Research-status Report
  • [Patent(Industrial Property Rights)] 義手装置2020

    • Inventor(s)
      福田修,坂口大介
    • Industrial Property Rights Holder
      国立大学法人佐賀大学
    • Industrial Property Rights Type
      意匠
    • Filing Date
      2020
    • Acquisition Date
      2021
    • Related Report
      2020 Research-status Report
  • [Patent(Industrial Property Rights)] 制御システム及び制御プログラム2019

    • Inventor(s)
      福田修,何宇楠,窪園隆介
    • Industrial Property Rights Holder
      国立大学法人佐賀大学
    • Industrial Property Rights Type
      特許
    • Industrial Property Number
      2019-225747
    • Filing Date
      2019
    • Related Report
      2019 Research-status Report
  • [Patent(Industrial Property Rights)] 義手装置2019

    • Inventor(s)
      福田修,坂口大介
    • Industrial Property Rights Holder
      国立大学法人佐賀大学
    • Industrial Property Rights Type
      意匠
    • Filing Date
      2019
    • Related Report
      2019 Research-status Report

URL: 

Published: 2019-04-18   Modified: 2023-03-16  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi