Dense 3-axis tactile sensing and AI to implement human-like manual skills in robots

2020 Fiscal Year Annual Research Report

• Project/Area Number: 19H02116
• Research Institution: Waseda University
• Principal Investigator: シュミッツ アレクサンダー 早稲田大学, 理工学術院, 准教授(任期付) (30729455)
• Co-Investigator(Kenkyū-buntansha): 尾形 哲也 早稲田大学, 理工学術院, 教授 (00318768) ; 玉城 絵美 早稲田大学, 理工学術院, 准教授(任期付) (30515086) ; Somlor Sophon 早稲田大学, 理工学術院総合研究所(理工学研究所), 次席研究員(研究院講師) (40791231)
• Project Period (FY): 2019-04-01 – 2022-03-31
• Keywords: Tactile Sensors

Outline of Annual Research Achievements

We have further developed the tactile sensors. In particular, the sensors are stretchable now, so that they can be used for robot joints for example in the future. Previously, the design required silver paste and multiple resins for elements such as wiring, wiring fixation, and substrate fixation. In terms of the work process, delicate temperature control and technology were required when bonding the sheet, resin, and substrate due to the effects of thermal deformation. In the new design, copper wire and silicon are the main materials used to create the elastic wiring, and the process no longer requires special work, making it easier and shorter than before. Therefore, it is expected to lead to increased efficiency when automating the process in the future.

Moreover, we have continued to use the tactile sensors embedded in robot hands and grippers for tactile AI. The robot is able to perform various in-hand manipulations, detect object properties, and predict grasp stability as well as detect slip. We use, implement and improve state of the art machine learning methods such as attention mechanisms, ensemble learning and graph neural networks.

Current Status of Research Progress

2: Research has progressed on the whole more than it was originally planned.

Reason

We finished the project as scheduled. There were no unexpected occurrences.

Strategy for Future Research Activity

In the future we want to cover the robot completely with tactile sensors.

Research Products

• [Journal Article] Multi-Fingered In-Hand Manipulation With Various Object Properties Using Graph Convolutional Networks and Distributed Tactile Sensors (2022)
  • Author(s): Funabashi Satoshi、Isobe Tomoki、Hongyi Fei、Hiramoto Atsumu、Schmitz Alexander、Sugano Shigeki、Ogata Tetsuya
  • Journal Title: IEEE Robotics and Automation Letters Volume: 7 Pages: 2102～2109
  • DOI: 10.1109/LRA.2022.3142417
  • Peer Reviewed / Int'l Joint Research
• [Journal Article] Tactile Transfer Learning and Object Recognition With a Multifingered Hand Using Morphology Specific Convolutional Neural Networks (2022)
  • Author(s): Funabashi Satoshi、Yan Gang、Hongyi Fei、Schmitz Alexander、Jamone Lorenzo、Ogata Tetsuya、Sugano Shigeki
  • Journal Title: IEEE Transactions on Neural Networks and Learning Systems Volume: 1 Pages: 1～15
  • DOI: 10.1109/TNNLS.2022.3215723
  • Peer Reviewed / Int'l Joint Research
• [Journal Article] A Robotic Grasping State Perception Framework With Multi-Phase Tactile Information and Ensemble Learning (2022)
  • Author(s): Yan Gang、Schmitz Alexander、Funabashi Satoshi、Somlor Sophon、Tomo Tito Pradhono、Sugano Shigeki
  • Journal Title: IEEE Robotics and Automation Letters Volume: 7 Pages: 6822～6829
  • DOI: 10.1109/LRA.2022.3151260
  • Peer Reviewed / Int'l Joint Research
• [Presentation] Detection of Slip from Vision and Touch (2022)
  • Author(s): Yan, G., Schmitz, A., Tomo, T. P., Somlor, S., Funabashi, S., Sugano, S.
  • Organizer: IEEE International Conference on Robotics and Automation (ICRA 2022)
  • Int'l Joint Research