• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Experimental Study on Human-Machine Interfaces Based on Real-time Close Interaction

Research Project

Project/Area Number 09555080
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeSingle-year Grants
Section展開研究
Research Field Intelligent mechanics/Mechanical systems
Research InstitutionOsaka University

Principal Investigator

KUNO Yoshinori  Graduate School of Engneering, Osaka University, Associate Professor, 大学院・工学研究科, 助教授 (10252595)

Co-Investigator(Kenkyū-buntansha) SHIMADA Nobutaka  Graduate School of Engineering, Osaka University, Research Associate, 大学院・工学研究科, 助手 (10294034)
MIURA Jun  Graduate School of Engineering, Osaka University, Associate Professor, 大学院・工学研究科, 助教授 (90219585)
SHIRAI Yoshiaki  Graduate School of Engineering, Osaka University, Professor, 大学院・工学研究科, 教授 (50206273)
SHIOHARA Morito  Fujitsu Laboratories Ltd. Senior Researcher, マルチメディアシステム研究所, 主任研究員
Project Period (FY) 1997 – 1999
Project Status Completed (Fiscal Year 1999)
Budget Amount *help
¥12,800,000 (Direct Cost: ¥12,800,000)
Fiscal Year 1999: ¥4,100,000 (Direct Cost: ¥4,100,000)
Fiscal Year 1998: ¥4,000,000 (Direct Cost: ¥4,000,000)
Fiscal Year 1997: ¥4,700,000 (Direct Cost: ¥4,700,000)
Keywordscomputer vision / human interface / virtual reality / gesture recognition / real-time system / face direction / intention understanding / wheelchair / 状態遷移図
Research Abstract

Vision-based human interfaces recognizing nonverbal behaviors such as gestures and gaze from video images have attracted great interest. However, conventional such interfaces impose several unfriendly restrictions on the users. In addtion, they sometimes fail in their recognition. And once they fail, most of them will never recover from the failure. In this project, we have proposed to solve these issues by exploiting real-time close interaction between the user and the machine, and developed experimental systems to verify the effectiveness.
First, we have relaxed restrictions on the user's position using active cameras to track the user. Then, we have eased restrictions on the user's unintentional hand motions. Conventional systems may erroneously recognize such motions as gestures intended to give commands. We use another nonverbal behavior, face direction, to choose only intentional motions. The assumption here is that the user is watching the target object when he/she wants to operate on it. Finally, we have proposed to use interaction by speech and action to recognize unknown gestures and to recover recognition failures. When the system is not certain of its recognition results, it shows the currentsituation by speech and/or action, which is the nonverbal behavior for the machine. Then it observes the user's reaction. Experimental results show that we can eventually pass our intention to the system by iterating such interaction.
We have demonstrated the effectiveness of our approach by developing a system in which we can move objects in a virtual world by hand gestures. In addition, we have developed an intelligent wheelchair to examine whether or not such nonverbal interfaces are applicable to systems working in the real world. We use face direction to control the wheelchair. We can turn it by looking in the direction where we want to go. Experimental results confirm this as a good means of interface.

Report

(4 results)
  • 1999 Annual Research Report   Final Research Report Summary
  • 1998 Annual Research Report
  • 1997 Annual Research Report
  • Research Products

    (56 results)

All Other

All Publications (56 results)

  • [Publications] K. Hayashi, Y. Kuno, and Y. Shirai: "Pointing gesture recognition system permitting user's freedom of movement"Proc. Workshop on Perceptual User Interfaces. 16-19 (1997)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] K. H. Jo, Y. Kuno, and Y. Shirai: "Context-based recognition of manipulative hand gestures for human computer interaction"Proc. 3rd Asian Conference on Computer Vision. 2. 368-375 (1998)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] K. H. Jo, Y. Kuno, and Y. Shirai: "Manipulative hand gestures recognition using task knowledge for human computer interaction"Proc. 3rd IEEE International Conference on Face and Gesture Recognition. 468-473 (1998)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Y. Kuno, T. Ishiyama, K. H. Jo, N. Shimada, and Y. Shirai: "Vision-based human interface system selectively recognizing intentional hand gestures"Proc. ISATED International Conference on Computer Graphics and Imaging. 219-222 (1998)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] T. Takahashi, S. Nakanishi, Y. Kuno and Y. Shirai: "Helping computer vision by verbal and nonverbal communication"Proc. 14th International Conference on Pattern Recognition. 1216-1218 (1998)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Y. Adachi, Y. Kuno, N. Shimada, and Y. Shirai: "Intelligent wheelchair using visual information on human faces"Proc. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. 354-359 (1998)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] T. Takahashi, S. Nakanishi, Y. Kuno and Y. Shirai: "Human-robot interface by verbal and nonverbal communication"Proc. 1998 IEEE/RSJInternational Conference on Intelligent Robots and Systems. 924-929 (1998)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Y. Kuno, T. Ishiyama, Y. Adachi and Y. Shirai: "Human interface systems using intentional and unintentional behaviors"Proc. 1998 Workshop on Perceptual User Interfaces. 55-58 (1998)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] 林健太郎,久野義徳,白井良明: "ユーザの位置の拘束のないジェスチャによるヒューマンインタフェース"情報処理学会論文誌. 40・2. 556-566 (1999)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Y. Kuno, Y. Adachi, T. Murashima, and Y. Shirai: "Intelligent wheelchair looking at its user"Proc. 2nd International Conference on Audio- and Video-based Biometric Person Authentication. 84-89 (1999)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] 足立佳久,中西 知,久野義徳,島田伸敬,白井良明: "顔の視覚情報処理を用いた知的車椅子"日本ロボット学会誌. 17・3. 423-431 (1999)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Y. Kuno, S. Nakanishi, T. Murashima, N. Shimada and Y. Shirai: "Robotic wheelchair with three control modes"Proc. 1998 IEEE International Conference on Robots and Automation. 2590-2595 (1999)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Y. Kuno, T. Ishiyama, S. Nakanishi, and Y. Shirai: "Combining observations of intentional and unintentional behaviors for human-computer interaction"Proc. ACM SIGCHI Conference on Human Factors in Computing Systems(CHI99). 238-245 (1999)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Y. Kuno, S. Nakanishi, T. Murashima, N. Shimada and Y. Shirai: "Robotic wheelchair observing its inside and outside"Proc. IARA International Conference on Image Analysis and Proceeding. 502-507 (1999)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] S. Nakanishi, Y. Kuno, N. Shimada, and Y. Shirai: "Robotic wheelchair based on observations of both user and environment"Proc. 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. 912-917 (1999)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Y. Kuno, S. Nakanishi, T. Murashima, N. Shimada and Y. Shirai: "Intelligent wheelchair based on the integration of human and environment observations"Proc. 1999 IEEE International Conference on Information Intelligence and Systems. 342-349 (1999)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] K. Hayashi, Y. Kuno, N. Shimada, and Y. Shirai: "Recovery of human postures using robust dynamic calibration"Proc. Fourth Asian Conference on Computer Vision. 49-56 (2000)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] 林健太郎,久野義徳,島田伸敬,白井良明: "動的ロバストキャリブレーションによる人体の姿勢復元"電子情報通信学会論文誌D-II. J83-D-II・掲載決定. (2000)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] 村嶋照久,久野義徳,島田伸敬,白井良明: "人間と機械のインタラクションを通じたジェスチャの理解と学習"日本ロボット学会誌. 18・掲載決定. (2000)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] K. Hayashi, Y. Kuno, and Y. Shirai: "Pointing gesture recognition system permitting user's freedom of movement"Proc. Workshop on Perceptual User Interfaces. 16-19 (1997)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] K. H. Jo, Y. Kuno, and Y. Shirai: "Context-based recognition of manipulative hand gestures for human computer interaction"Proc. 3rd Asian Conference on Computer Vision. vol. 2. 368-375 (1998)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] K. H. Jo, Y. Kuno, and Y. Shirai: "Manipulative hand gesture recognition using task knowledge for human computer interaction"Proc. 3rd IEEE International Conference on Face and Gesture Recognition. 468-473 (1998)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Y. Kuno, T. Ishiyama, K. H. Jo, N. Shimada, and Y. Shirai: "Vision-based human interface system selectively recognizing intentional hand gestures"Proc. IASTED International Conference on Computer Graphics and Imaging. 219-222 (1998)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] T. Takahashi, S. Nakanishi, Y. Kuno, and Y. Shirai: "Helping computer vision by verbal and nonverbal communication"Proc. 14th International Conference on Pattern Recognition. 1216-1218 (1998)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Y. Adachi, Y. Kuno, N. Shimada, and Y. Shirai: "Intelligent wheelchair using visual information on human faces"Proc. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. 354-359 (1998)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] T. Takahashi, S. Nakanishi, Y. Kuno, and Y. Shirai.: "Human-robot interface by verbal and nonverbal communication"Proc. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems.. 924-929 (1998)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Y. Kuno, T. Ishiyama, Y. Adachi, and Y. Shirai: "Human interface systems using intentional and unintentional behaviors"Proc. 1998 Workshop on Perceptual User Interfaces. 55-58 (1998)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] K. Hayashi, Y. Kuno, and Y. Shirai: "Position-free human interface by pointing gestures"Trans. Information Society of Japan (in Japanese). vol. 40, no. 2. 556-566 (1999)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Y. Kuno, Y. Adachi, T. Murashima, and Y. Shirai: "Intelligent wheelchair looking at its user"Proc. Second International Conference on Audio- and Videobased Biometric Person Authentication. 84-89 (1999)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Y. Adachi, S. Nakanishi, Y. Kuno, N. Shimada, and Y. Shirai: "Intelligent wheelchair using visual information from the human faces"Journal of the Robotics Society of Japan (in Japanese). vol.17, no.3. 423-431 (1999)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Y. Kuno, S. Nakanishi, T. Murashima, N. Shimada, and Y. Shirai: "Robotic wheelchair with three control modes"Proc. 1999 IEEE International Conference on Robotics and Automation. 2590-2595 (1999)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Y. Kuno, T. Ishiyama, S. Nakanishi, and Y. Shirai: "Combining observations of intentional and unintentional behaviors for human-computer interaction"Proc. ACM SIGCHI Conference on Human Factors in Computing Systems (CHI 99). 238-245 (1999)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Y. Kuno, S. Nakanishi, T. Murashima, N. Shimada, and Y. Shirai: "Robotic wheelchair observing its inside and outside"Proc. IAPR International Conference on Image Analysis and Processing. 502-507 (1999)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] S. Nakanishi, Y. Kuno, N. Shimada, and Y. Shirai: "Robotic wheelchair based on observations of both user and environment"Proc. 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. 912-917 (1999)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Y. Kuno, S. Nakanishi, T. Murashima, N. Shimada, and Y. Shirai: "Intelligent wheelchair based on the integration of human and environment observations"Proc. 1999 IEEE International Conference on Information Intelligence and Systems. 342-349 (1999)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] K. Hayashi, Y. Kuno, N. Shimada, and Y. Shirai: "Recovery of human postures using robust dynamic calibration"Proc. Fourth Asian Conference on Computer Vision. 49-56 (2000)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] K. Hayashi, Y. Kuno, N. Shimada, and Y. Shirai: "Recovery of human postures using robust dynamic calibration"Trans. IEICE D-II. vol.J83-D-II (to appear). (2000)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] T. Murashima, Y. Kuno, N. Shimada, and Y. Shirai: "Understanding and learning of gestures through human-machine interaction"Journal of the Robotics Society of Japan (in Japanese). vol.18 (to appear). (2000)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] 足立佳久,中西知,久野義徳,島田伸敬,白井良明: "顔の視覚情報処理を用いた知的車椅子"日本ロボット学会誌. 17・3. 423-431 (1999)

    • Related Report
      1999 Annual Research Report
  • [Publications] Y. Kuno , S. Nakanishi, T. Murashima, N.Shimada, and Y.Shirai: "Robotic wheelchair with three control modes"Proc. 1999 IEEE Int. Conf. on Robotics and Automation. 2590-2595 (1999)

    • Related Report
      1999 Annual Research Report
  • [Publications] Y. Kuno, T. Ishiyama, S. Nakanishi and Y. Shirai: "Combining observations of intentional and unintentional behaviors for human computer interaction"Proc. ACM SIGCHI Conf. on Human Factors in Computing Systems (CHI 99). 238-245 (1999)

    • Related Report
      1999 Annual Research Report
  • [Publications] Y. Kuno ,S. Nakanishi, T. Murashima, N.Shimada, and Y.Shirai: "Robotic wheelchair observing its inside and outside"Proc. IAPR International Conference on Image Analysis and Processing. 502-507 (1999)

    • Related Report
      1999 Annual Research Report
  • [Publications] S. Nakanishi,Y.Kuno, N. Shimada, and Y. Shirai: "Robotic wheelchair based on observations of both user and environment"Proc. 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. 912-917 (1999)

    • Related Report
      1999 Annual Research Report
  • [Publications] Y. Kuno, S. Nakanishi, T. Murashima, N.Shimada, and Y.Shirai: "Intelligent wheelchair based on the integration of human and environment observations"Proc. 1999 IEEE International Conference on Information Intelligence and Systems. 342-349 (1999)

    • Related Report
      1999 Annual Research Report
  • [Publications] 林健太郎,久野義徳,島田伸敬,白井良明: "動的ロバストキャリブレーションによる人体の姿勢復元"電子情報通信学会論文詩D-II. J83-D-II(掲載決定). (2000)

    • Related Report
      1999 Annual Research Report
  • [Publications] 村嶋照久,久野義徳,島田伸敬,白井良明: "人間と機械のインタラクションを通じたジェスチャの理解と学習"日本ロボット学会誌. 18(掲載決定). (2000)

    • Related Report
      1999 Annual Research Report
  • [Publications] 三浦 純,白井良明: "プラニングコストと視覚の不確かさを考慮した移動ロボットの視覚と行動のプラニング" 人工知能学会誌. 13巻4号. 588-596 (1998)

    • Related Report
      1998 Annual Research Report
  • [Publications] I.H.Moon,J.Miura,and Y.Shirai: "On-line Viewpoint and Motion Planning for Efficient Visual Navigation under Uncer-tainty" Robotics and Autonomous Systems. (to appear). (1999)

    • Related Report
      1998 Annual Research Report
  • [Publications] J.Miura and Y.Shirai: "Scheduling Parellel Execution of Planning and Action for a Mobile Robot Considering Planning Cost and Vision Uncertainty" Proc.Int.Conf.on Intelligent Robots and Systems. 545-550 (1998)

    • Related Report
      1998 Annual Research Report
  • [Publications] T.Yoneda,Y.Shirai,and J.Miura: "Navigation of Vision-Guided Mobile Robot in an Unkonwn Envirionment" Proc.1998 Japan-U.S.A.Symposium on Flexible Automation. 569-572 (1998)

    • Related Report
      1998 Annual Research Report
  • [Publications] 三浦 純,白井良明: "プラニングコストと視覚の不確かさを考慮した移動ロボットのプラニングと行動の並列スケジューリング" 日本機械学会1998年ロボティクス・メカトロニクス講演会. 1CI3-1 (1998)

    • Related Report
      1998 Annual Research Report
  • [Publications] 文 仁赫,白井良明,三浦 純: "不確かさを考慮した視覚ランドマークのオンラインモデリングと選択" 日本機械学会1999年ロボティクス・メカトロニクス講演会. (発表予定). (1999)

    • Related Report
      1998 Annual Research Report
  • [Publications] K.Hayashi, Y.Kuno, and Y.Shirai: "Pointing gesture recognition system permitting user's freedom of movement" Proc.Workshop on Perceptual User Interfaces. 16-19 (1997)

    • Related Report
      1997 Annual Research Report
  • [Publications] K.H.Jo, Y.Kuno, and Y.Shirai: "Context-based recognition of manipulative hand gestures for human computer interaction" Proc.3rd Asian Conference on Computer Vision. 2. 368-375 (1998)

    • Related Report
      1997 Annual Research Report
  • [Publications] K.H.Jo, Y.Kuno, and Y.Shirai: "Manipulative hand gesture recognition using task knowledge for human computer interaction" Proc.3rd IEEE International Conference on Face and Gesture Recognition. (発表予定). (1998)

    • Related Report
      1997 Annual Research Report
  • [Publications] Y.Kuno, T.Ishiyama, K.H.Jo, N.Shimada, and Y.Shirai: "Vision-besed human interface sysyem selectively recognizing intentional hand gestures" Proc.IASTED Int.Conf.on Computer Graphics and Imaging. (発表予定). (1998)

    • Related Report
      1997 Annual Research Report

URL: 

Published: 1997-04-01   Modified: 2016-04-21  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi