Dense 3-axis tactile sensing and AI to implement human-like manual skills in robots

2021 Fiscal Year Annual Research Report

• Project/Area Number: 19H02116
• Research Institution: Waseda University
• Principal Investigator: シュミッツ アレクサンダー 早稲田大学, 理工学術院, 准教授(任期付) (30729455)
• Co-Investigator(Kenkyū-buntansha): 尾形 哲也 早稲田大学, 理工学術院, 教授 (00318768) ; 玉城 絵美 琉球大学, 工学部, 教授 (30515086) ; Somlor Sophon 早稲田大学, 理工学術院総合研究所(理工学研究所), 次席研究員(研究院講師) (40791231)
• Project Period (FY): 2019-04-01 – 2022-03-31
• Keywords: Tactile Sensing

Outline of Annual Research Achievements

We have further developed our tactile sensors. 368 3-axis tactile sensors cover the robot hand. Furthermore, we have further developed a robot hand that allows to cover the palmar side of the robot hand nearly completely with thick skin. We have also further developed the tactile sensors for human hands, which now allow measuring the contact position in addition to the contact force on the fingertip without covering the human skin that is in contact with the object. Moreover, we have used the tactile sensors embedded in robot hands and grippers for tactile AI. The robot was able to perform various in-hand manipulations, detect object properties, and predict grasp stability as well as detect slip. We use, implement and improve state of the art machine learning methods such as attention mechanisms, ensemble learning and graph neural networks.

Research Progress Status

令和3年度が最終年度であるため、記入しない。

Strategy for Future Research Activity

令和3年度が最終年度であるため、記入しない。

Research Products

• [Journal Article] A Robotic Grasping State Perception Framework with Multi-Phase Tactile Information and Ensemble Learning (2022)
  • Author(s): Yan Gang、Schmitz Alexander、Funabashi Satoshi、Somlor Sophon、Tomo Tito Pradhono、Sugano Shigeki
  • Journal Title: IEEE Robotics and Automation Letters Volume: 1 Pages: 1～1
  • DOI: 10.1109/LRA.2022.3151260
  • Peer Reviewed / Int'l Joint Research
• [Journal Article] Multi-Fingered In-Hand Manipulation With Various Object Properties Using Graph Convolutional Networks and Distributed Tactile Sensors (2022)
  • Author(s): Funabashi Satoshi、Isobe Tomoki、Hongyi Fei、Hiramoto Atsumu、Schmitz Alexander、Sugano Shigeki、Ogata Tetsuya
  • Journal Title: IEEE Robotics and Automation Letters Volume: 7 Pages: 2102～2109
  • DOI: 10.1109/LRA.2022.3142417
  • Peer Reviewed / Int'l Joint Research
• [Presentation] Detection of Slip from Vision and Touch (2022)
  • Author(s): Yan, G., Schmitz, A., Tomo, T. P., Somlor, S., Funabashi, S., Sugano, S.
  • Organizer: IEEE International Conference on Robotics and Automation
  • Int'l Joint Research
• [Presentation] A Multi-Fingered Robot Hand with Remote Center of Motion Mechanisms for Covering Joints with Soft Skin (2021)
  • Author(s): Khullar, G., Schmitz, A., Hsu, C., Sathe, P., Funabashi, S., Sugano, S.
  • Organizer: IEEE International Conference on Robotics and Biomimetics
  • Int'l Joint Research
• [Presentation] “Safe Skin” - A Low-Cost Capacitive Proximity-Force-Fusion Sensor For Safety in Robots (2021)
  • Author(s): Wang, Z., Gao, H., Schmitz, A., Somlor, S., Tomo, T.P., Sugano, S.
  • Organizer: IEEE/RSJ International Conference on Intelligent Robots and Systems
  • Int'l Joint Research
• [Presentation] SCT-CNN: A Spatio-Channel-Temporal Attention CNN for Grasp Stability Prediction (2021)
  • Author(s): Yan, G., Schmitz, A., Funabashi, S., Somlor, S., Tomo, T.P., Sugano, S.
  • Organizer: IEEE International Conference on Robotics and Automation
  • Int'l Joint Research