• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to project page

1999 Fiscal Year Final Research Report Summary

Experimental Study on Human-Machine Interfaces Based on Real-time Close Interaction

Research Project

Project/Area Number 09555080
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeSingle-year Grants
Section展開研究
Research Field Intelligent mechanics/Mechanical systems
Research InstitutionOsaka University

Principal Investigator

KUNO Yoshinori  Graduate School of Engneering, Osaka University, Associate Professor, 大学院・工学研究科, 助教授 (10252595)

Co-Investigator(Kenkyū-buntansha) SHIMADA Nobutaka  Graduate School of Engineering, Osaka University, Research Associate, 大学院・工学研究科, 助手 (10294034)
MIURA Jun  Graduate School of Engineering, Osaka University, Associate Professor, 大学院・工学研究科, 助教授 (90219585)
SHIRAI Yoshiaki  Graduate School of Engineering, Osaka University, Professor, 大学院・工学研究科, 教授 (50206273)
SHIOHARA Morito  Fujitsu Laboratories Ltd. Senior Researcher, マルチメディアシステム研究所, 主任研究員
Project Period (FY) 1997 – 1999
Keywordscomputer vision / human interface / virtual reality / gesture recognition / real-time system / face direction / intention understanding / wheelchair
Research Abstract

Vision-based human interfaces recognizing nonverbal behaviors such as gestures and gaze from video images have attracted great interest. However, conventional such interfaces impose several unfriendly restrictions on the users. In addtion, they sometimes fail in their recognition. And once they fail, most of them will never recover from the failure. In this project, we have proposed to solve these issues by exploiting real-time close interaction between the user and the machine, and developed experimental systems to verify the effectiveness.
First, we have relaxed restrictions on the user's position using active cameras to track the user. Then, we have eased restrictions on the user's unintentional hand motions. Conventional systems may erroneously recognize such motions as gestures intended to give commands. We use another nonverbal behavior, face direction, to choose only intentional motions. The assumption here is that the user is watching the target object when he/she wants to operate on it. Finally, we have proposed to use interaction by speech and action to recognize unknown gestures and to recover recognition failures. When the system is not certain of its recognition results, it shows the currentsituation by speech and/or action, which is the nonverbal behavior for the machine. Then it observes the user's reaction. Experimental results show that we can eventually pass our intention to the system by iterating such interaction.
We have demonstrated the effectiveness of our approach by developing a system in which we can move objects in a virtual world by hand gestures. In addition, we have developed an intelligent wheelchair to examine whether or not such nonverbal interfaces are applicable to systems working in the real world. We use face direction to control the wheelchair. We can turn it by looking in the direction where we want to go. Experimental results confirm this as a good means of interface.

  • Research Products

    (38 results)

All Other

All Publications (38 results)

  • [Publications] K. Hayashi, Y. Kuno, and Y. Shirai: "Pointing gesture recognition system permitting user's freedom of movement"Proc. Workshop on Perceptual User Interfaces. 16-19 (1997)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] K. H. Jo, Y. Kuno, and Y. Shirai: "Context-based recognition of manipulative hand gestures for human computer interaction"Proc. 3rd Asian Conference on Computer Vision. 2. 368-375 (1998)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] K. H. Jo, Y. Kuno, and Y. Shirai: "Manipulative hand gestures recognition using task knowledge for human computer interaction"Proc. 3rd IEEE International Conference on Face and Gesture Recognition. 468-473 (1998)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] Y. Kuno, T. Ishiyama, K. H. Jo, N. Shimada, and Y. Shirai: "Vision-based human interface system selectively recognizing intentional hand gestures"Proc. ISATED International Conference on Computer Graphics and Imaging. 219-222 (1998)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] T. Takahashi, S. Nakanishi, Y. Kuno and Y. Shirai: "Helping computer vision by verbal and nonverbal communication"Proc. 14th International Conference on Pattern Recognition. 1216-1218 (1998)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] Y. Adachi, Y. Kuno, N. Shimada, and Y. Shirai: "Intelligent wheelchair using visual information on human faces"Proc. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. 354-359 (1998)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] T. Takahashi, S. Nakanishi, Y. Kuno and Y. Shirai: "Human-robot interface by verbal and nonverbal communication"Proc. 1998 IEEE/RSJInternational Conference on Intelligent Robots and Systems. 924-929 (1998)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] Y. Kuno, T. Ishiyama, Y. Adachi and Y. Shirai: "Human interface systems using intentional and unintentional behaviors"Proc. 1998 Workshop on Perceptual User Interfaces. 55-58 (1998)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] 林健太郎,久野義徳,白井良明: "ユーザの位置の拘束のないジェスチャによるヒューマンインタフェース"情報処理学会論文誌. 40・2. 556-566 (1999)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] Y. Kuno, Y. Adachi, T. Murashima, and Y. Shirai: "Intelligent wheelchair looking at its user"Proc. 2nd International Conference on Audio- and Video-based Biometric Person Authentication. 84-89 (1999)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] 足立佳久,中西 知,久野義徳,島田伸敬,白井良明: "顔の視覚情報処理を用いた知的車椅子"日本ロボット学会誌. 17・3. 423-431 (1999)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] Y. Kuno, S. Nakanishi, T. Murashima, N. Shimada and Y. Shirai: "Robotic wheelchair with three control modes"Proc. 1998 IEEE International Conference on Robots and Automation. 2590-2595 (1999)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] Y. Kuno, T. Ishiyama, S. Nakanishi, and Y. Shirai: "Combining observations of intentional and unintentional behaviors for human-computer interaction"Proc. ACM SIGCHI Conference on Human Factors in Computing Systems(CHI99). 238-245 (1999)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] Y. Kuno, S. Nakanishi, T. Murashima, N. Shimada and Y. Shirai: "Robotic wheelchair observing its inside and outside"Proc. IARA International Conference on Image Analysis and Proceeding. 502-507 (1999)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] S. Nakanishi, Y. Kuno, N. Shimada, and Y. Shirai: "Robotic wheelchair based on observations of both user and environment"Proc. 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. 912-917 (1999)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] Y. Kuno, S. Nakanishi, T. Murashima, N. Shimada and Y. Shirai: "Intelligent wheelchair based on the integration of human and environment observations"Proc. 1999 IEEE International Conference on Information Intelligence and Systems. 342-349 (1999)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] K. Hayashi, Y. Kuno, N. Shimada, and Y. Shirai: "Recovery of human postures using robust dynamic calibration"Proc. Fourth Asian Conference on Computer Vision. 49-56 (2000)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] 林健太郎,久野義徳,島田伸敬,白井良明: "動的ロバストキャリブレーションによる人体の姿勢復元"電子情報通信学会論文誌D-II. J83-D-II・掲載決定. (2000)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] 村嶋照久,久野義徳,島田伸敬,白井良明: "人間と機械のインタラクションを通じたジェスチャの理解と学習"日本ロボット学会誌. 18・掲載決定. (2000)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] K. Hayashi, Y. Kuno, and Y. Shirai: "Pointing gesture recognition system permitting user's freedom of movement"Proc. Workshop on Perceptual User Interfaces. 16-19 (1997)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] K. H. Jo, Y. Kuno, and Y. Shirai: "Context-based recognition of manipulative hand gestures for human computer interaction"Proc. 3rd Asian Conference on Computer Vision. vol. 2. 368-375 (1998)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] K. H. Jo, Y. Kuno, and Y. Shirai: "Manipulative hand gesture recognition using task knowledge for human computer interaction"Proc. 3rd IEEE International Conference on Face and Gesture Recognition. 468-473 (1998)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] Y. Kuno, T. Ishiyama, K. H. Jo, N. Shimada, and Y. Shirai: "Vision-based human interface system selectively recognizing intentional hand gestures"Proc. IASTED International Conference on Computer Graphics and Imaging. 219-222 (1998)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] T. Takahashi, S. Nakanishi, Y. Kuno, and Y. Shirai: "Helping computer vision by verbal and nonverbal communication"Proc. 14th International Conference on Pattern Recognition. 1216-1218 (1998)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] Y. Adachi, Y. Kuno, N. Shimada, and Y. Shirai: "Intelligent wheelchair using visual information on human faces"Proc. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. 354-359 (1998)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] T. Takahashi, S. Nakanishi, Y. Kuno, and Y. Shirai.: "Human-robot interface by verbal and nonverbal communication"Proc. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems.. 924-929 (1998)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] Y. Kuno, T. Ishiyama, Y. Adachi, and Y. Shirai: "Human interface systems using intentional and unintentional behaviors"Proc. 1998 Workshop on Perceptual User Interfaces. 55-58 (1998)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] K. Hayashi, Y. Kuno, and Y. Shirai: "Position-free human interface by pointing gestures"Trans. Information Society of Japan (in Japanese). vol. 40, no. 2. 556-566 (1999)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] Y. Kuno, Y. Adachi, T. Murashima, and Y. Shirai: "Intelligent wheelchair looking at its user"Proc. Second International Conference on Audio- and Videobased Biometric Person Authentication. 84-89 (1999)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] Y. Adachi, S. Nakanishi, Y. Kuno, N. Shimada, and Y. Shirai: "Intelligent wheelchair using visual information from the human faces"Journal of the Robotics Society of Japan (in Japanese). vol.17, no.3. 423-431 (1999)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] Y. Kuno, S. Nakanishi, T. Murashima, N. Shimada, and Y. Shirai: "Robotic wheelchair with three control modes"Proc. 1999 IEEE International Conference on Robotics and Automation. 2590-2595 (1999)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] Y. Kuno, T. Ishiyama, S. Nakanishi, and Y. Shirai: "Combining observations of intentional and unintentional behaviors for human-computer interaction"Proc. ACM SIGCHI Conference on Human Factors in Computing Systems (CHI 99). 238-245 (1999)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] Y. Kuno, S. Nakanishi, T. Murashima, N. Shimada, and Y. Shirai: "Robotic wheelchair observing its inside and outside"Proc. IAPR International Conference on Image Analysis and Processing. 502-507 (1999)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] S. Nakanishi, Y. Kuno, N. Shimada, and Y. Shirai: "Robotic wheelchair based on observations of both user and environment"Proc. 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. 912-917 (1999)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] Y. Kuno, S. Nakanishi, T. Murashima, N. Shimada, and Y. Shirai: "Intelligent wheelchair based on the integration of human and environment observations"Proc. 1999 IEEE International Conference on Information Intelligence and Systems. 342-349 (1999)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] K. Hayashi, Y. Kuno, N. Shimada, and Y. Shirai: "Recovery of human postures using robust dynamic calibration"Proc. Fourth Asian Conference on Computer Vision. 49-56 (2000)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] K. Hayashi, Y. Kuno, N. Shimada, and Y. Shirai: "Recovery of human postures using robust dynamic calibration"Trans. IEICE D-II. vol.J83-D-II (to appear). (2000)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] T. Murashima, Y. Kuno, N. Shimada, and Y. Shirai: "Understanding and learning of gestures through human-machine interaction"Journal of the Robotics Society of Japan (in Japanese). vol.18 (to appear). (2000)

    • Description
      「研究成果報告書概要(欧文)」より

URL: 

Published: 2001-10-23  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi