• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Construction of Prototype Multi-modal Interface by Lifelike Agent

Research Project

Project/Area Number 09555119
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeSingle-year Grants
Section展開研究
Research Field 情報通信工学
Research InstitutionSeikei University

Principal Investigator

MORISHIMA Shigeo  Faculty of Engineering, Seikei Univ., Professor, 工学部, 助教授 (10200411)

Co-Investigator(Kenkyū-buntansha) YAMADA Hiroshi  Nihon University, Associate Professor, 文理学部, 助教授 (80191328)
Project Period (FY) 1997 – 1999
Project Status Completed (Fiscal Year 1999)
Budget Amount *help
¥9,100,000 (Direct Cost: ¥9,100,000)
Fiscal Year 1999: ¥2,000,000 (Direct Cost: ¥2,000,000)
Fiscal Year 1998: ¥2,300,000 (Direct Cost: ¥2,300,000)
Fiscal Year 1997: ¥4,800,000 (Direct Cost: ¥4,800,000)
KeywordsKansei Information / Face Image Processing / Cyberspace / Agent / Lip Synchronization / Wire Frame Model / Expression Synthesis / Expression Analysis / インタフェース / マルチモーダル / コミュニケーション / メディア変換 / 感情認識
Research Abstract

Prototype communication system between multiple clients in face-to-face style is constructed. Lifelike agent appears in cyberspace driven by other client interactively. This system is composed of one server and multiple clients, and each client has a camera and microphone. Voice captured from microphone is transmitted to server frame by frame. And then voice signal is analyzed and converted to mouth shape parameters by neural network. This mouth shape parameter is transmitted to each client through network and, expression and lip movement of lifelike agent copying other side client are controlled by this parameter. Voice signal is also transmitted to each client and played back by speaker at each client system synchronizing with synthesized face image. Basic expression can be selected by pushing function key to change the face image of lifelike agent displayed at other clients. Two mode is prepared in this system ; walk through mode and fly through mode. In the walk through mode, each user can communicate by eye contact with others and change the location and direction of agent eye. In the fly mode, user can be an observer in communication.
By using this prototype communication system, the experiment in which three users communicate each other is performed. Synthesis rate is 10 frames per second. And natural communication environment can be achieved.

Report

(4 results)
  • 1999 Annual Research Report   Final Research Report Summary
  • 1998 Annual Research Report
  • 1997 Annual Research Report
  • Research Products

    (25 results)

All Other

All Publications (25 results)

  • [Publications] 森島繁生,八木康史: "顔の認識・合成のための標準ツール"システム/制御/情報. 44・3. 119-126 (2000)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] S.Morishima, T.Yotsukura: "Face-to-face Communication Avatar Driven by Voice"Proceedings of IEEE ICIP. 27AS1-3 (1999)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Shigeo Morishima: "Real-time Voice Driven Facial Animation System"Proceedings of IEEE ICSMC. FP13-5 (1999)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] 寺田員人,森島繁生 他: "コンピュータグラフィックスを用いた矯正治療による表情変化"歯科審美. 12・1. 37-51 (1999)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] 四倉達夫,藤井英史,森島繁生: "サイバースペース上の仮想人物による実時間対話システムの構築"情報処理学会論文誌. 40・2. 677-686 (1999)

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] S.Morishima, T.Yotsura: "Face-to-Face Communication Aratar Driven by Voice"Proceedings of IEEE ICIP. 27AS1-3 (1999)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] Shigeo Morishima: "Real-time Voice Driver Facial Animation System"Proceedings of IEEE ICSMC. FP13-5 (1999)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] T.Yotsukura, Ei Fujii, S.Morishima: "Realtime Face-to-face Communication System in Cyberspace Using Voice Driver Aratar with Texture Mapped Face"Transaction of Information Processing Society of Japan. Vol.40. No.2. 677-686 (1999)

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      1999 Final Research Report Summary
  • [Publications] 森島繁生 八木康史: "顔の認識・合成のための標準ツール"システム/制御/情報. 44・3. 119-126 (2000)

    • Related Report
      1999 Annual Research Report
  • [Publications] S.Morishima,T.Yotsukura: "Face-to-Face Communication Avatar Driven by Voice"Proceedings of IEEE ICIP. 27AS1-3 (1999)

    • Related Report
      1999 Annual Research Report
  • [Publications] Shigeo Morishima: "Real-time Voice Driven Facial Animation System"Proceedings of IEEE ICSMC. FP13-5 (1999)

    • Related Report
      1999 Annual Research Report
  • [Publications] 寺田員人、森島繁生 他: "コンピュータグラフィックスを用いた矯正治療による表情変化"歯科審美. 12・1. 37-51 (1999)

    • Related Report
      1999 Annual Research Report
  • [Publications] 四倉達夫、藤井英史、森島繁生: "サイバースペース上の仮想人物による実時間対話システムの構築"情報処理学会論文誌. 40・2. 677-686 (1999)

    • Related Report
      1999 Annual Research Report
  • [Publications] 四倉達夫、藤井英史、森島繁生: "サイバースペース上の仮想人物による実時間対話システムの構築" 情報処理学会論文誌. 40巻2号. 677-686 (1999)

    • Related Report
      1998 Annual Research Report
  • [Publications] Shigeo Morishima: "Real-time Talking Head Driven by Voice and its Application to Communication and Entertainment" Proc.of International Conf.on Auditory Visual Speach Processing. 2巻. 195-200 (1998)

    • Related Report
      1998 Annual Research Report
  • [Publications] Shigeo Morishima, T.Ishikawa: "Facial Muscle Parameter Decision from 2D Froutal Image" IEEE Proc.of ICPR. 14巻. 160-162 (1998)

    • Related Report
      1998 Annual Research Report
  • [Publications] Shigeo Morishima: "3D Facial Expression Tracking and Regeneration from Single Camera Image" International Archives of Photorammetry and Remete Seusing. XXXII. 900-905 (1998)

    • Related Report
      1998 Annual Research Report
  • [Publications] 四倉達夫・森島繁生: "音声から画像へのメディア変換を用いたサイバースペース上での多人数コミュニケーション" インタラクション′98 論文集. 2巻. 133-134 (1998)

    • Related Report
      1998 Annual Research Report
  • [Publications] 坂口竜己、山田寛、森島繁生: "顔画像を基にした3次元感情モデルの構築とその評価" 電子情報通信学会論文誌A. J80-8. 1547-1554 (1997)

    • Related Report
      1998 Annual Research Report
  • [Publications] 四倉達夫, 森島繁生: "音声からの画像へのメディア変換を用いたサイバースペース上での多人数コミュニケーションシステム" インタラクション'98論文集. 133-134 (1998)

    • Related Report
      1997 Annual Research Report
  • [Publications] 金子正秀, 森島繁生: "顔画像によるアドバンストエージェント" 映像情報メディア学会誌. 51・8. 1169-1174 (1997)

    • Related Report
      1997 Annual Research Report
  • [Publications] 長谷川 修, 森島繁生: "「顔」の情報処理" 電子情報通信学会論文誌A. J80-8. 1231-1249 (1997)

    • Related Report
      1997 Annual Research Report
  • [Publications] 坂口竜己, 山田寛, 森島繁生: "顔画像を基にした3次元感情モデルの構築とその評価" 電子情報通信学会論文誌A. J80-8. 1279-1284 (1997)

    • Related Report
      1997 Annual Research Report
  • [Publications] 坂口竜己, 森島繁生: "画像の2次元離散コサイン変換を利用した実時間顔表情認識" 電子情報通信学会論文誌DII. J80-6. 1547-1554 (1997)

    • Related Report
      1997 Annual Research Report
  • [Publications] 森島繁生: "A Physics-based Talking Head for Interface Agent" Proceedings of IJCAI Workshop on Animated Agent. 91-97 (1997)

    • Related Report
      1997 Annual Research Report

URL: 

Published: 1997-04-01   Modified: 2016-04-21  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi