• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to project page

1997 Fiscal Year Final Research Report Summary

Construction of On-line Multimodal Dictionary for Automatic Human Behavior Understanding

Research Project

Project/Area Number 08458073
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeSingle-year Grants
Section一般
Research Field Intelligent informatics
Research InstitutionUniversity of Tsukuba

Principal Investigator

OHTA Yuichi  University of Tsukuba, Institute of Information Sciences and Electronics, Professor, 電子情報工学系, 教授 (50115804)

Co-Investigator(Kenkyū-buntansha) SUZUKI Emiko  Tokyo Kasei Gakuin Tsukuba Junior College, Associate Propessor, 筑波女子大学・情報処理科, 助教授 (70226497)
NAKAMURA Yuichi  University of Tsukuba, Institute of Information Sciences and Electronics, Assist, 電子情報工学系, 講師 (40227947)
Project Period (FY) 1996 – 1997
Keywordshuman benavior understanding / multimodal dictionary / human behavior recording / human motion analysis / presentation analysis / human motion tracking / video generation / video editing
Research Abstract

We investigated description for various aspects on human behaviors, and accumulated actual data in a machine readable format. This research contains, 1)a study for the human behavior description and accumulation in various situations, and development of a prototype database, 2)human motion observation by multiple moving cameras, 3)view sclection and editing from the views by multiple cameras.
On the first topic, we gathered various records : body movements measured by a magnetic motion capture system, videos from multiple cameras aiming at important portions for understanding human motions, audio, speech transcripts, and human movements interprctation by human. They are related to each other by their occurrence time. We can examine, by referring the data, how the different modalities are cooperatively and complementarily used in real situations. This cxamination over different situations is essential for understanding human behaviors, since they are heavily dependent on their contexts and personal charactcristics.
On the second topic, we developed a human behavior tracking system with multple cameras. Multiple moving cameras at different. positions aim at important portions on a human body and something around the body, such as face, hands, or objects on a desk. This record help us to investigate not only how humans moved but also how those important portions are seen in actual communications and what information they carry.
On the third topic, we investigated a new method to choose the timing, focused point for observing and recording the human behaviors by using the data taken by the above system. This recarch aims at generation of comprehensible report or summary of presentations or lectures by selecting the cameras and frames by using human motion understanding.

  • Research Products

    (22 results)

All Other

All Publications (22 results)

  • [Publications] Y.Nakamura その他: "Semantic Analysis for Video Cont-ents Extraction" ACM Multimedia'97. 394-401 (1997)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] 松浦 友彦 その他: "隠れ検出の可能な多眼ステレオ法" 電子情報通信学会論文誌. Vol.J80-DII. 1432-1440 (1997)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] 向川 康博 その他: "複数の顔画像の組合せによる任意方向・任意表情の顔画像の生成" 電子情報通信学会論文誌. Vol.J80-DII. 1555-1562 (1997)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] Y.Mukaigawa その他: "Face Synthesis with Arbitrary Po-se and Expression from Several Images" Asian Conference on Computer Vis-ion. Vol.1. 680-687 (1998)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] 木村 善文 その他: "発話情報と動作情報を用いたプレゼンテーション映像の要約" 信学技報. PRMU97-197. 7-14 (1998)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] Y.Nakamura その他: "Multimodal Multi-View Integrated Database for Human Behavior Understandi-ng" IEEE International Conference on Automatic Face and Gesture Recognition. (1998)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] Y.Nakamura: "Spotting by Association in News Video" Proc.AAAI 1997 Spring Symposium : Intelligent Integration and Use of Text, Image, Video and Audio Corpora. 113-119 (1997)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] Y.Nakamura: "Semantic Analysis for Video Contents Extraction-Spotting by Association in News video" Proc.ACM Multimedia. 393-401 (1997)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] Y.Mukaigawa: "Face Synthesis with Arbitrary Pose and Expression from Several Images" Trans.IEICE. Vol.J80-DII. 1555-1562 (1997)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] Y.Mukaigawa: "Face Synthesis with Arbitrary Pose and Expression from Several Images-An integration of Image-based and Model-based Approach" Proc.Asian Conference on Computer Vision. Vol.1. 680-687 (1998)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] Y.Nakamura: "MMID : Multimodal Multiview Integrated Database for Human Behavior Understanding" Proc.IEEE International Conference on Automatic Face and Gesture Recognition. (to appear). (1998)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] L.Gao: "Synthesis of Facial Images with Lip Motion from Several Real Views" Proc.IEEE International Conference on Automatic Face and Gesture Recognition. (to appear). (1998)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] Y.Nakamura: "Speaker's Motion Understanding in Presentation (in Japanese)" lst SME Robomech Symposia. No.96-8. 40-45 (1996)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] Y.Uchori: "Temporal Structure Analysis of Television Commercial by Image and Sound Processing (in Japanese)" IEICE Convention of the System Society. 553-554 (1996)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] Y.Kimura: "Focus Selection for Human Behavior Understanding by using Speech Analysis (in Japanese)" 54th Annual Convention of IPSJ. 17-18 (1996)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] Y.Kimura: "Multimodal Database on Presentation and its Utilization (in Japanese)" Intelligent Information Media Symposium. 165-172 (1997)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] Y.Kimura: "Video Generation and Summarization by Utilizing Speech and Motion Analysis (in Japanese)" IEICE SIG-PRMU. 97-197. 7-14 (1998)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] L.Gao: "Synthesis of Facial Images with Lip Motion based on Real Views (in Japanese)" IEICE,SIG-PRMU. 97-157. 41-46 (1997)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] Y.Nakamura: "Knowledge Extraction from Diagram and Text for Media Integration" Proc.Int.Conference on Multimedia Computation and Systems. 448-492 (1996)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] Y.Nakamura: "Occlusion Detectable Stereo-Occlusion Patterns in Camera Matrix-" Proc.IEEE Conference on Computer Vision and Pattern Recognition '96. 371-378 (1996)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] T.Kurata: "Feature Tracking on Occluding Boundaries for Modeling-by-videotaping" Proc.Int.Conference on Virtual Systems and Multimedia. 123-128 (1996)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] K.Yang: "Hand Region Extraction with Adaptive Color Modeling" Proc.CONTROL,AUTOMATION.ROBOTICS AND VISION. Vol.1. 401-405 (1996)

    • Description
      「研究成果報告書概要(欧文)」より

URL: 

Published: 1999-03-16  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi