Automatic Process-Modeling of Precise Fingering and Its Application to Robotic Hand Operation

2017 Fiscal Year Final Research Report

• Project/Area Number: 15H02764
• Research Category: Grant-in-Aid for Scientific Research (B)
• Allocation Type: Single-year Grants
• Section: 一般
• Research Field: Intelligent robotics
• Research Institution: Ritsumeikan University
• Principal Investigator: Nobutaka Shimada 立命館大学, 情報理工学部, 教授 (10294034)
• Co-Investigator(Kenkyū-buntansha): 田中 弘美 立命館大学, 情報理工学部, 教授 (10268154) ; 李 周浩 立命館大学, 情報理工学部, 教授 (80366434)
• Project Period (FY): 2015-04-01 – 2018-03-31
• Keywords: 深層学習 / ＣＮＮ / 持ち方パラメータ / 物体機能 / 想起 / プロセスモデリング

Outline of Final Research Achievements

We studied modeling method of fingering process of object manipulation. We broke down whole problem into 5 sub-problems:(1) precise measurement of fingers postures and fingertip positions from image,(2) recall of human’s grasping patterns based on 3-D shape of objects by DNN,(3) describing fingering process in object manipulation as state transition model,(4) recall of next action to be taken which causes required scene/object state change,(5) controlling the robot hand based on recalled grasping pattern.We applied DNN frameworks including CNN and RNN commonly to the above sub-problems and then acquired relationships between human action and object's state as static frame and dynamic sequence from human behavior observation.
We proposed the first step methods to build process model of human’s operation of object, under several limitations.

Free Research Field

コンピュータビジョン、知能ロボティクス