Generating Robot's Attendance Actions by Estimating Person's Awareness to the Environment

• Project/Area Number: 25280093
• Research Category: Grant-in-Aid for Scientific Research (B)
• Allocation Type: Partial Multi-year Fund
• Section: 一般
• Research Field: Intelligent robotics
• Research Institution: Toyohashi University of Technology
• Principal Investigator: Miura Jun 豊橋技術科学大学, 工学(系)研究科(研究院), 教授 (90219585)
• Co-Investigator(Renkei-kenkyūsha): SATAKE JUNJI 豊橋技術科学大学, 大学院工学研究科, 助教 (60392726) ; OISHI SHUJI 豊橋技術科学大学, 大学院工学研究科, 助教 (30759618)
• Project Period (FY): 2013-04-01 – 2017-03-31
• Project Status: Completed (Fiscal Year 2016)
• Budget Amount: ¥16,380,000 (Direct Cost: ¥12,600,000、Indirect Cost: ¥3,780,000); Fiscal Year 2016: ¥3,250,000 (Direct Cost: ¥2,500,000、Indirect Cost: ¥750,000); Fiscal Year 2015: ¥3,380,000 (Direct Cost: ¥2,600,000、Indirect Cost: ¥780,000); Fiscal Year 2014: ¥3,640,000 (Direct Cost: ¥2,800,000、Indirect Cost: ¥840,000); Fiscal Year 2013: ¥6,110,000 (Direct Cost: ¥4,700,000、Indirect Cost: ¥1,410,000)
• Keywords: 付き添いロボット / アウェアネス / 移動ロボット / 人物検出・識別 / 人物行動推定 / ロボットの行動生成 / 見守りロボット / 生活支援ロボティクス / 視点計画 / 人物追従 / 人物検出・認識

Outline of Final Research Achievements

There is an increasing demand for personal service robots such as attendant robots that support people in their daily life. Aiming at realizing such an attendant robot, we developed the following methods: (1) detecting and tracking people in a variety of poses and discriminating a specific person to whom services are provided; (2) estimating a person's awareness to the environment from his/her actions with estimation of face poses; (3) generating appropriate attendance actions and switching them depending on a person's intention and/or state. We implemented and tested these methods using a real robot to show their effectiveness.

Research Products

• [Journal Article] Identification of a Specific Person using Color, Height, and Gait Features for a Person Following Robot (2016)
  • Author(s): Kenji Koide and Jun Miura
  • Journal Title: Robotics and Autonomous Systems Volume: 84 Pages: 877-891
  • DOI: 10.1016/j.robot.2016.07.004
  • Peer Reviewed / Acknowledgement Compliant
• [Journal Article] OptiFuzz: A Robust Illumination Invariant Face Recognition System and Its Implementation (2016)
  • Author(s): Bima Sena Bayu Dewantara and Jun Miura
  • Journal Title: Machine Vision and Applications Volume: 27 Issue: 6 Pages: 877-891
  • DOI: 10.1007/s00138-016-0790-6
  • Peer Reviewed / Acknowledgement Compliant
• [Journal Article] Estimating Head Orientation Using a Combination of Multiple Cues (2016)
  • Author(s): Bima Sena Bayu Dewantara and Jun Miura
  • Journal Title: IEICE Transactions on Information and Systems Volume: E99.D Issue: 6 Pages: 1603-1614
  • DOI: 10.1587/transinf.2015EDP7375
  • NAID: 130005154733
  • ISSN: 0916-8532, 1745-1361
  • Peer Reviewed / Acknowledgement Compliant
• [Journal Article] Estimating Head Orientation using a Combination of Multiple Cues (2016)
  • Author(s): Bima Sena Bayu Dewantara and Jun Miura
  • Journal Title: IEICE Transactions on Information and Systems Volume: E99-D
  • NAID: 130005154733
  • Peer Reviewed / Acknowledgement Compliant
• [Journal Article] Specific Person Tracking using 3D LIDAR and ESPAR Antenna for Mobile Service Robots (2015)
  • Author(s): Kazuki Misu and Jun Miura
  • Journal Title: Advanced Robotics Volume: 29 Issue: 22 Pages: 1483-1495
  • DOI: 10.1080/01691864.2015.1093429
  • Peer Reviewed / Acknowledgement Compliant
• [Journal Article] Partial Least Squares-based Human Upper Body Estimation with Combined Detection and Tracking (2014)
  • Author(s): Igi Ardiyanto and Jun Miura
  • Journal Title: Image and Vision Computing Volume: 32 Issue: 11 Pages: 904-915
  • DOI: 10.1016/j.imavis.2014.08.002
  • Peer Reviewed
• [Journal Article] Visual Person Identification using a Distance-Dependent Appearance Model for a Person Following Robot (2013)
  • Author(s): Junji Satake, Masaya Chiba, Jun Miura
  • Journal Title: International Journal of Automation and Computing Volume: 10 Issue: 5 Pages: 438-446
  • DOI: 10.1007/s11633-013-0740-y
  • Peer Reviewed
• [Presentation] 3D LIDARを用いた携行型人物行動計測システム (2017)
  • Author(s): 小出健司
  • Organizer: 2017年ロボティクス・メカトロニクス講演会
  • Place of Presentation: 郡山
  • Year and Date: 2017-05-10
• [Presentation] 温度情報付き点群の生成と温熱環境評価への適用 (2016)
  • Author(s): 出村充宏
  • Organizer: SI2016
  • Place of Presentation: 札幌
  • Year and Date: 2016-12-15
• [Presentation] LIDAR-based Body Orientation Estimation by Integrating Shape and Motion Information (2016)
  • Author(s): Shuji Oishi
  • Organizer: 2016 IEEE Int. Conf. on Robotics and Biemimetics (ROBIO 2016)
  • Place of Presentation: Qingdao, China
  • Year and Date: 2016-12-03
  • Int'l Joint Research
• [Presentation] Estimating Person's Awareness of an Obstacle using HCRF for an Attendant Robot (2016)
  • Author(s): Kenji Koide
  • Organizer: 4th Int. Conf. on Human-Agent Interaction (iHAI 2016)
  • Place of Presentation: Singapore
  • Year and Date: 2016-10-04
  • Int'l Joint Research
• [Presentation] Generation of a Socially Aware Behavior of a Guide Robot Using Reinforcement Learning (2016)
  • Author(s): Bima Sena Bayu Dewantara
  • Organizer: 2016 Int. Electronics Symp. (IES 2016)
  • Place of Presentation: Bali, Indonesia
  • Year and Date: 2016-09-29
  • Int'l Joint Research
• [Presentation] Toward a Robotic Attendant Adaptively Behaving according to Human State (2016)
  • Author(s): Shuji Oishi
  • Organizer: 2016 IEEE Int. Symp. on Robot and Human Interactive Communication (RO-MAN 2016)
  • Place of Presentation: New York, USA
  • Year and Date: 2016-08-26
  • Int'l Joint Research
• [Presentation] 人の状況に応じた付き添いロボットの行動計画 (2016)
  • Author(s): 小針義貴
  • Organizer: 2016年ロボティクス・メカトロニクス講演会
  • Place of Presentation: 横浜
  • Year and Date: 2016-06-08
• [Presentation] 人の状況に応じた付き添いロボットの行動計画 (2016)
  • Author(s): 小針義貴
  • Organizer: 2016年日本機械学会ロボティクス・メカトロニクス講演会
  • Place of Presentation: 横浜
  • Year and Date: 2016-06-08
• [Presentation] 移動ロボットのためのレーザ距離センサを用いた人物追跡と状態向き推定の融合 (2015)
  • Author(s): 清水政伸
  • Organizer: 計測自動制御学会SI部門講演会2015
  • Place of Presentation: 名古屋
  • Year and Date: 2015-12-14
• [Presentation] 付き添い行動計測ウェアラブルデバイスの開発 (2015)
  • Author(s): 高羽雄太
  • Organizer: 計測自動制御学会SI部門講演会2015
  • Place of Presentation: 名古屋
  • Year and Date: 2015-12-14
• [Presentation] The AISL Head Orientation Database and Preliminary Evaluations (2015)
  • Author(s): Bima Sena Bayu Dewantara
  • Organizer: 2015 International Electronics Symposium
  • Place of Presentation: スラバヤ，インドネシア
  • Year and Date: 2015-09-29
  • Int'l Joint Research
• [Presentation] Person Identification Based on the Matching of Foot Strike Timings Obtained by IMU and LRF (2015)
  • Author(s): Kenji Koide
  • Organizer: IROS-2015 Workshop on Assistance and Service Robotics in a Human Environment
  • Place of Presentation: ハンブルグ，ドイツ
  • Year and Date: 2015-09-28
  • Int'l Joint Research
• [Presentation] Human Motion Prediction Considering Enviromental Context (2015)
  • Author(s): Igi Ardiyanto
  • Organizer: 2015 IAPR International Conference on Machine Vision Applications
  • Place of Presentation: Tokyo, Japan
  • Year and Date: 2015-05-18 – 2015-05-22
• [Presentation] Human Motion Prediction Considering Environmental Context (2015)
  • Author(s): Igi Ardiyanto
  • Organizer: 2015 IAPR International Conference on Machine Vision Applications
  • Place of Presentation: 東京
  • Year and Date: 2015-05-18
  • Int'l Joint Research
• [Presentation] IMUとLRFを用いた接地タイミングの同時性に基づく移動ロボットのための人物認証 (2015)
  • Author(s): 小出健司
  • Organizer: 2015年ロボティクス・メカトロニクス講演会
  • Place of Presentation: 京都
  • Year and Date: 2015-05-17 – 2015-05-19
• [Presentation] 移動ロボットによる住環境センシングと生活支援への応用 (2015)
  • Author(s): 三浦 純
  • Organizer: 2015年ロボティクス・メカトロニクス講演会
  • Place of Presentation: 京都
  • Year and Date: 2015-05-17 – 2015-05-19
• [Presentation] 見守りロボットのための多様な臥位に対応した頭部位置推定 (2015)
  • Author(s): 西佳一郎
  • Organizer: 2015年ロボティクス・メカトロニクス講演会
  • Place of Presentation: 京都
  • Year and Date: 2015-05-17 – 2015-05-19
• [Presentation] IMUとLRFを用いた接地タイミングの同時性に基づく移動ロボットのための人物認証 (2015)
  • Author(s): 小出健司
  • Organizer: 2015年日本機械学会ロボティクス・メカトロニクス講演会
  • Place of Presentation: 京都
  • Year and Date: 2015-05-17
• [Presentation] 移動ロボットのためのビジョン (2014)
  • Author(s): 三浦 純
  • Organizer: ビジョン技術の実利用化ワークショップ
  • Place of Presentation: 東京
  • Year and Date: 2014-12-04 – 2014-12-05
  • Invited
• [Presentation] Cameraman Robot: Dynamic Trajectory Tracking with Final Time Constratint using State-time Space Stochastic Approach (2014)
  • Author(s): Igi Ardiyanto
  • Organizer: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems
  • Place of Presentation: Chicago, USA
  • Year and Date: 2014-09-14 – 2014-09-18
• [Presentation] 3次元レーザ距離センサとエスパアンテナを用いた移動ロボットによる特定人物発見・追跡 (2014)
  • Author(s): 三浦 純
  • Organizer: 第32回日本ロボット学会学術講演会
  • Place of Presentation: 福岡
  • Year and Date: 2014-09-04 – 2014-09-06
• [Presentation] 服の色・身長・歩容特徴を用いた付き添いロボットのための人物識別 (2014)
  • Author(s): 小出健司
  • Organizer: 第32回日本ロボット学会学術講演会
  • Place of Presentation: 福岡
  • Year and Date: 2014-09-04 – 2014-09-06
• [Presentation] Specific Person Detection and Tracking by a Mobile Robot using 3D LIDAR and ESPAR Antenna (2014)
  • Author(s): Jun Miura
  • Organizer: 13th International Conference on Intelligent Autonomous Systems
  • Place of Presentation: Padova, Italy
• [Presentation] RTミドルウェアの産業利用を想定した遠隔運用可能な自律移動型ロボットシステムの開発 (2014)
  • Author(s): 三浦純
  • Organizer: 2014年日本機械学会ロボティクスメカトロニクス講演会
  • Place of Presentation: 富山
• [Presentation] 移動型生活支援ロボットによる屋内照度分布計測と照明機器の操作 (2014)
  • Author(s): 可児周平
  • Organizer: 2014年日本機械学会ロボティクスメカトロニクス講演会
  • Place of Presentation: 富山
• [Presentation] カメラマンロボットの研究：対象物体の移動に適応した視点のオンライン計画 (2014)
  • Author(s): 清水政伸
  • Organizer: 2014年日本ロボティクスメカトロニクス講演会
  • Place of Presentation: 富山
• [Presentation] Visibility-Based Viewpoint Planning for Guard Robot using Skeletonization and Geodesic Motion Model (2013)
  • Author(s): Igi Ardiyanto
  • Organizer: 2013 IEEE International Conference on Robotics and Automation
  • Place of Presentation: Karlsruhe, Germany
• [Presentation] Fuzzy-based Illumination Normalization for Face Recognition (2013)
  • Author(s): Bima Sena Bayu Dewantara
  • Organizer: 2013 IEEE Workshop on Advanced Robotics and Its Social Impacts
  • Place of Presentation: Tokyo, Japan
• [Presentation] Robots That Can Attend People (2013)
  • Author(s): Jun Miura
  • Organizer: Regional Conference on Computer and Information Engineering
  • Place of Presentation: Bangkok, Thailand
  • Invited
• [Presentation] 付き添いロボットのためのカメラとレーザ距離センサを用いた人物発見・追跡 (2013)
  • Author(s): 小出健司
  • Organizer: 計測自動制御学会システムインテグレーション部門講演会2013
  • Place of Presentation: 神戸
• [Remarks] 豊橋技術科学大学 行動知能システム学研究室（三浦研究室）
  • URL: http://www.aisl.cs.tut.ac.jp/
• [Remarks] 豊橋技術科学大学 行動知能システム学研究室
  • URL: http://www.aisl.cs.tut.ac.jp/
• [Remarks] 豊橋技術科学大学行動知能システム学研究室（三浦研究室）
  • URL: http://www.aisl.cs.tut.ac.jp/
• [Remarks] 豊橋技術科学大学行動知能システム学研究室（三浦研究室）
  • URL: http://www.aisl.cs.tut.ac.jp/
• [Patent(Industrial Property Rights)] 特定人物識別装置およびその方法 (2015)
  • Inventor(s): 三浦純，小出健司
  • Industrial Property Rights Holder: 豊橋技術科学大学
  • Industrial Property Rights Type: 特許
  • Industrial Property Number: 2015-099988
  • Filing Date: 2015-05-15