Human-robot collaboration mechanisms in remote collaborative object search tasks

• Project/Area Number: 17H01799
• Research Category: Grant-in-Aid for Scientific Research (B)
• Allocation Type: Single-year Grants
• Section: 一般
• Research Field: Intelligent robotics
• Research Institution: Toyohashi University of Technology
• Principal Investigator: Miura Jun 豊橋技術科学大学, 工学(系)研究科(研究院), 教授 (90219585)
• Project Period (FY): 2017-04-01 – 2021-03-31
• Project Status: Completed (Fiscal Year 2020)
• Budget Amount: ¥17,940,000 (Direct Cost: ¥13,800,000、Indirect Cost: ¥4,140,000); Fiscal Year 2020: ¥3,640,000 (Direct Cost: ¥2,800,000、Indirect Cost: ¥840,000); Fiscal Year 2019: ¥2,470,000 (Direct Cost: ¥1,900,000、Indirect Cost: ¥570,000); Fiscal Year 2018: ¥3,120,000 (Direct Cost: ¥2,400,000、Indirect Cost: ¥720,000); Fiscal Year 2017: ¥8,710,000 (Direct Cost: ¥6,700,000、Indirect Cost: ¥2,010,000)
• Keywords: サービスロボット / 人間ロボット協調 / 遠隔協調物体探索 / 移動ロボット / ロボット操作インタフェース / 遠隔協調探索 / 知能ロボット / 生活支援ロボット

Outline of Final Research Achievements

The objective of this research is to develop a robot that can find a user-specified object with occasional interaction with the user. We have developed novel technologies for such robots, including ontology-based knowledge management, query generation for resolving ambiguities in the user's command, and various interfaces for natural human-robot interaction. We also developed fundamental technologies for increasing the autonomy of the robots, such as scene recognition, recognition planning, and motion planning.

Academic Significance and Societal Importance of the Research Achievements

生活支援ロボットにおいて，ユーザの指示する物体を探して持ってくることは基本的な機能の一つである．本研究では，ロボットが環境に関する自身の知識を適切に管理することを中心機能として，必要に応じてユーザとやりとりをしながら物体発見を確実に行うためのさまざまな手法を開発した．ユーザの負担を軽減し，かつ自然なやりとりを実現するこれらの手法は学術的な価値があり，また近い将来の生活支援ロボットの実用化に有用なものである．

Research Products

• [Journal Article] Ontology-based knowledge management with verbal interaction for command interpretation and execution by home service robots (2021)
  • Author(s): Villamar Gomez L.、Miura J.
  • Journal Title: Robotics and Autonomous Systems Volume: 140 Pages: 103763-103763
  • DOI: 10.1016/j.robot.2021.103763
  • Peer Reviewed
• [Journal Article] RDS-SLAM: Real-Time Dynamic SLAM Using Semantic Segmentation Methods (2021)
  • Author(s): Liu Yubao、Miura Jun
  • Journal Title: IEEE Access Volume: 9 Pages: 23772-23785
  • DOI: 10.1109/access.2021.3050617
  • Peer Reviewed / Open Access
• [Journal Article] A Framework for DRL Navigation With State Transition Checking and Velocity Increment Scheduling (2020)
  • Author(s): Dewa Chandra Kusuma、Miura Jun
  • Journal Title: IEEE Access Volume: 8 Pages: 191826-191838
  • DOI: 10.1109/access.2020.3033016
  • Peer Reviewed / Open Access / Int'l Joint Research
• [Presentation] Ontology Learning of New Concepts combining Textural Knowledge and Visual Analysis for Service Robot Application (2021)
  • Author(s): L. Villamar Gomez
  • Organizer: 2021 IEEE Int. Conf. on Intelligence and Safety for Robotics
  • Int'l Joint Research
• [Presentation] KMOP-vSLAM: Dynamic Visual SLAM for RGB-D Cameras using K-means and OpenPose (2021)
  • Author(s): Y. Liu
  • Organizer: 2021 IEEE/SICE Int. Conf. on Systems Integration
  • Int'l Joint Research
• [Presentation] An Operator and Observer Style Teleoperation User Interface for Remote Object Search (2021)
  • Author(s): K. Hayashi
  • Organizer: 2021 IEEE/SICE Int. Conf. on Systems Integration
  • Int'l Joint Research
• [Presentation] 遠隔物体探索のための操作者－観察者型インタフェースの提案と評価 (2020)
  • Author(s): 三浦純
  • Organizer: 計測自動制御学会システムインテグレーション部門講演会2020
• [Presentation] 屋内物体探索における3次元未観測領域推定に基づく視点計画 (2020)
  • Author(s): 三明優介
  • Organizer: 日本ロボット学会学術講演会2020
• [Presentation] Deep Learning Applications for Robotics (2020)
  • Author(s): J. Miura
  • Organizer: Int. Electronics Symposium 2020
  • Int'l Joint Research / Invited
• [Presentation] 遠隔操作ロボットの操作意図の早期推定による操作性の向上 (2020)
  • Author(s): 上野康平
  • Organizer: 2020年ロボティクス・メカトロニクス講演会
• [Presentation] 遠隔物体探索におけるユーザ指示のあいまいさ解消のための質問生成 (2020)
  • Author(s): 伊藤魁一
  • Organizer: 2020年ロボティクス・メカトロニクス講演会
• [Presentation] 移動ロボットによる机上物体探索のための視点計画 (2020)
  • Author(s): 三明優介
  • Organizer: 2020年ロボティクス・メカトロニクス講演会
• [Presentation] Query Generation for Resolving Ambiguity in User's Command for a Mobile Service Robot (2019)
  • Author(s): Jun Miura
  • Organizer: 2019 European Conf. on Mobile Robots
  • Int'l Joint Research
• [Presentation] EarlyEstimation of User's Intention of Tele-operation using Object Affordance and Hand Motion in a Dual First-Person Vision (2019)
  • Author(s): Jun Miura
  • Organizer: ICCV-2019 Workshop on Egocentric Perception, Interaction, and Computing
  • Int'l Joint Research
• [Presentation] RoboCup Japan Open 2019@Home DSPLにおけるチームAISL-TUTの取り組み (2019)
  • Author(s): 松崎成道
  • Organizer: 第37回日本ロボット学会学術講演会
• [Presentation] Sweeping Motions for Finding and Picking Up a Specific Object in a Pile of Objects (2018)
  • Author(s): Jun Miura
  • Organizer: IROS-2018 Workshop on Towards Robots that Exhibit Manipulation Intelligence
  • Int'l Joint Research
• [Presentation] An Integrated Planning of Exploration, Coverage, and Object Localization for an Efficient Indoor Semantic Mapping (2018)
  • Author(s): Jun Miura
  • Organizer: 15th Int. Conf. on Intelligent Autonomous Systems
  • Int'l Joint Research
• [Presentation] Machine Learning Applications for Personal Service Robots (2018)
  • Author(s): Jun Miura
  • Organizer: IAS-15 Workshop on Learning Applications for Intelligent Autonomous Robots
  • Int'l Joint Research / Invited
• [Presentation] Machine Learning Applications for Personal Service Robots (2018)
  • Author(s): Jun Miura
  • Organizer: 10th Int. Conf. on Information Technology and Electrical Engineering
  • Int'l Joint Research / Invited
• [Presentation] 移動型生活支援ロボットの開発 (2018)
  • Author(s): 松下正忠
  • Organizer: 日本機械学会ロボティクスメカトロニクス講演会2018
• [Presentation] An Integrated Exploration and Observation Planning for an Efficient Indoor 3D Mapping (2017)
  • Author(s): Diar Fahruddin Sasongko
  • Organizer: 2017 Int. Conf. on Mechatronics and Automation
  • Int'l Joint Research
• [Presentation] Object Placement Estimation with Occlusions and Planning of Robotic Handling Strategies (2017)
  • Author(s): Jun Miura
  • Organizer: 2017 IEEE Int. Conf. on Advanced Intelligent Mechatronics
  • Int'l Joint Research
• [Presentation] 隠蔽を考慮した物体配置推定と目的物体探索のためのハンド動作計画 (2017)
  • Author(s): 三浦 純
  • Organizer: 計測自動制御学会システムインテグレーション部門講演会2017
• [Remarks] Active Intelligent Systems Laboratory, TUT
  • URL: http://www.aisl.cs.tut.ac.jp/
• [Remarks] 豊橋技術科学大学行動知能システム学研究室（三浦研究室）
  • URL: http://www.aisl.cstut.ac.jp/
• [Remarks] 豊橋技術科学大学行動知能システム学研究室（三浦研究室）
  • URL: http://www.aisl.cs.tut.ac.jp/
• [Remarks] 豊橋技術科学大学行動知能システム学研究室
  • URL: http://www.aisl.cs.tut.ac.jp/