• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Construction of On-line Multimodal Dictionary for Automatic Human Behavior Understanding

Research Project

Project/Area Number 08458073
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeSingle-year Grants
Section一般
Research Field Intelligent informatics
Research InstitutionUniversity of Tsukuba

Principal Investigator

OHTA Yuichi  University of Tsukuba, Institute of Information Sciences and Electronics, Professor, 電子情報工学系, 教授 (50115804)

Co-Investigator(Kenkyū-buntansha) SUZUKI Emiko  Tokyo Kasei Gakuin Tsukuba Junior College, Associate Propessor, 筑波女子大学・情報処理科, 助教授 (70226497)
NAKAMURA Yuichi  University of Tsukuba, Institute of Information Sciences and Electronics, Assist, 電子情報工学系, 講師 (40227947)
Project Period (FY) 1996 – 1997
Project Status Completed (Fiscal Year 1997)
Budget Amount *help
¥6,400,000 (Direct Cost: ¥6,400,000)
Fiscal Year 1997: ¥2,400,000 (Direct Cost: ¥2,400,000)
Fiscal Year 1996: ¥4,000,000 (Direct Cost: ¥4,000,000)
Keywordshuman benavior understanding / multimodal dictionary / human behavior recording / human motion analysis / presentation analysis / human motion tracking / video generation / video editing / 人間行動理解 / マルチモーダル辞書 / 動作認識 / 発活の理解 / ヒューマンインタフェース / 映像の構造化
Research Abstract

We investigated description for various aspects on human behaviors, and accumulated actual data in a machine readable format. This research contains, 1)a study for the human behavior description and accumulation in various situations, and development of a prototype database, 2)human motion observation by multiple moving cameras, 3)view sclection and editing from the views by multiple cameras.
On the first topic, we gathered various records : body movements measured by a magnetic motion capture system, videos from multiple cameras aiming at important portions for understanding human motions, audio, speech transcripts, and human movements interprctation by human. They are related to each other by their occurrence time. We can examine, by referring the data, how the different modalities are cooperatively and complementarily used in real situations. This cxamination over different situations is essential for understanding human behaviors, since they are heavily dependent on their contexts and personal charactcristics.
On the second topic, we developed a human behavior tracking system with multple cameras. Multiple moving cameras at different. positions aim at important portions on a human body and something around the body, such as face, hands, or objects on a desk. This record help us to investigate not only how humans moved but also how those important portions are seen in actual communications and what information they carry.
On the third topic, we investigated a new method to choose the timing, focused point for observing and recording the human behaviors by using the data taken by the above system. This recarch aims at generation of comprehensible report or summary of presentations or lectures by selecting the cameras and frames by using human motion understanding.

Report

(3 results)
  • 1997 Annual Research Report   Final Research Report Summary
  • 1996 Annual Research Report
  • Research Products

    (34 results)

All Other

All Publications (34 results)

  • [Publications] Y.Nakamura その他: "Semantic Analysis for Video Cont-ents Extraction" ACM Multimedia'97. 394-401 (1997)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1997 Final Research Report Summary
  • [Publications] 松浦 友彦 その他: "隠れ検出の可能な多眼ステレオ法" 電子情報通信学会論文誌. Vol.J80-DII. 1432-1440 (1997)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1997 Final Research Report Summary
  • [Publications] 向川 康博 その他: "複数の顔画像の組合せによる任意方向・任意表情の顔画像の生成" 電子情報通信学会論文誌. Vol.J80-DII. 1555-1562 (1997)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1997 Final Research Report Summary
  • [Publications] Y.Mukaigawa その他: "Face Synthesis with Arbitrary Po-se and Expression from Several Images" Asian Conference on Computer Vis-ion. Vol.1. 680-687 (1998)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1997 Final Research Report Summary
  • [Publications] 木村 善文 その他: "発話情報と動作情報を用いたプレゼンテーション映像の要約" 信学技報. PRMU97-197. 7-14 (1998)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1997 Final Research Report Summary
  • [Publications] Y.Nakamura その他: "Multimodal Multi-View Integrated Database for Human Behavior Understandi-ng" IEEE International Conference on Automatic Face and Gesture Recognition. (1998)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1997 Final Research Report Summary
  • [Publications] Y.Nakamura: "Spotting by Association in News Video" Proc.AAAI 1997 Spring Symposium : Intelligent Integration and Use of Text, Image, Video and Audio Corpora. 113-119 (1997)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1997 Final Research Report Summary
  • [Publications] Y.Nakamura: "Semantic Analysis for Video Contents Extraction-Spotting by Association in News video" Proc.ACM Multimedia. 393-401 (1997)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1997 Final Research Report Summary
  • [Publications] Y.Mukaigawa: "Face Synthesis with Arbitrary Pose and Expression from Several Images" Trans.IEICE. Vol.J80-DII. 1555-1562 (1997)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1997 Final Research Report Summary
  • [Publications] Y.Mukaigawa: "Face Synthesis with Arbitrary Pose and Expression from Several Images-An integration of Image-based and Model-based Approach" Proc.Asian Conference on Computer Vision. Vol.1. 680-687 (1998)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1997 Final Research Report Summary
  • [Publications] Y.Nakamura: "MMID : Multimodal Multiview Integrated Database for Human Behavior Understanding" Proc.IEEE International Conference on Automatic Face and Gesture Recognition. (to appear). (1998)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1997 Final Research Report Summary
  • [Publications] L.Gao: "Synthesis of Facial Images with Lip Motion from Several Real Views" Proc.IEEE International Conference on Automatic Face and Gesture Recognition. (to appear). (1998)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1997 Final Research Report Summary
  • [Publications] Y.Nakamura: "Speaker's Motion Understanding in Presentation (in Japanese)" lst SME Robomech Symposia. No.96-8. 40-45 (1996)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1997 Final Research Report Summary
  • [Publications] Y.Uchori: "Temporal Structure Analysis of Television Commercial by Image and Sound Processing (in Japanese)" IEICE Convention of the System Society. 553-554 (1996)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1997 Final Research Report Summary
  • [Publications] Y.Kimura: "Focus Selection for Human Behavior Understanding by using Speech Analysis (in Japanese)" 54th Annual Convention of IPSJ. 17-18 (1996)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1997 Final Research Report Summary
  • [Publications] Y.Kimura: "Multimodal Database on Presentation and its Utilization (in Japanese)" Intelligent Information Media Symposium. 165-172 (1997)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1997 Final Research Report Summary
  • [Publications] Y.Kimura: "Video Generation and Summarization by Utilizing Speech and Motion Analysis (in Japanese)" IEICE SIG-PRMU. 97-197. 7-14 (1998)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1997 Final Research Report Summary
  • [Publications] L.Gao: "Synthesis of Facial Images with Lip Motion based on Real Views (in Japanese)" IEICE,SIG-PRMU. 97-157. 41-46 (1997)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1997 Final Research Report Summary
  • [Publications] Y.Nakamura: "Knowledge Extraction from Diagram and Text for Media Integration" Proc.Int.Conference on Multimedia Computation and Systems. 448-492 (1996)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1997 Final Research Report Summary
  • [Publications] Y.Nakamura: "Occlusion Detectable Stereo-Occlusion Patterns in Camera Matrix-" Proc.IEEE Conference on Computer Vision and Pattern Recognition '96. 371-378 (1996)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1997 Final Research Report Summary
  • [Publications] T.Kurata: "Feature Tracking on Occluding Boundaries for Modeling-by-videotaping" Proc.Int.Conference on Virtual Systems and Multimedia. 123-128 (1996)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1997 Final Research Report Summary
  • [Publications] K.Yang: "Hand Region Extraction with Adaptive Color Modeling" Proc.CONTROL,AUTOMATION.ROBOTICS AND VISION. Vol.1. 401-405 (1996)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1997 Final Research Report Summary
  • [Publications] Y.Nakamura その他: "Semantic Analysis for Video Contents Extraction" ACM Multimedia'97. 394-401 (1997)

    • Related Report
      1997 Annual Research Report
  • [Publications] 松浦 友彦, その他: "隠れ検出の可能な多眼ステレオ法" 電子情報通信学会論文誌. Vol.J80-DII. 1432-1440 (1997)

    • Related Report
      1997 Annual Research Report
  • [Publications] 向川 康博, その他: "複数の顔画像の組合せによる任意方向・任意表情の顔画像の生成" 電子情報通信学会論文誌. Vol.J80-DII. 1555-1562 (1997)

    • Related Report
      1997 Annual Research Report
  • [Publications] Y.Mukaigawa その他: "Face Synthesis with Arbitrary Pose and Expression from Several Images" Asian Conference on Compoter Vision. Vol.1. 680-687 (1998)

    • Related Report
      1997 Annual Research Report
  • [Publications] 木村 善文・その他: "発話情報と動作情報を用いたプレゼンテーション映像の要約" 信学技報. PRMU97-197. 7-14 (1998)

    • Related Report
      1997 Annual Research Report
  • [Publications] Y.Nakamura その他: "Multimodel Multi-view Integrated Database for Human Behavior Understanding" IEEE International Conference on Automatic Face and Gesture Recognition. (1998)

    • Related Report
      1997 Annual Research Report
  • [Publications] 中村裕一 他: "Knowledge Extraction from Diagram and Text for Media Integration" Int.Conference on Multimedia and Communication Systems. 437-441 (1996)

    • Related Report
      1996 Annual Research Report
  • [Publications] 中村裕一 他: "Occlusion Detectable Stereo-Occlusion Patterus in Camera Matrix" IEEE Conference on Computer Vision and Pattern Recognition. 371-378 (1996)

    • Related Report
      1996 Annual Research Report
  • [Publications] 木村善文他: "人物行動理解のための発活情報を用いた注目要素選択" 情報処理学会平成9年前期全国大会. A280-286 (1997)

    • Related Report
      1996 Annual Research Report
  • [Publications] E.Hussein 他: "Analysis of Detailed Patterns of Contour Shapes Using Wavelet Extrema" Int.Conference on Patlern Recognition. B335-339 (1996)

    • Related Report
      1996 Annual Research Report
  • [Publications] 蔵田武士 他: "Feature Tracking on Occluding Boundaries for Modeling-by-videotaping" Int.Conference on Virtual Systems and Maltimedia. 123-128 (1996)

    • Related Report
      1996 Annual Research Report
  • [Publications] 向川康博 他: "2枚の顔写真を用いた任意方向の顔画像の生成" 情報処理学会論文誌. 37・4. 635-644 (1996)

    • Related Report
      1996 Annual Research Report

URL: 

Published: 1996-04-01   Modified: 2016-04-21  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi