• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Modeling bidirectional multimodal emotion communication for man-to-machine dialog

Research Project

Project/Area Number 25871139
Research Category

Grant-in-Aid for Young Scientists (B)

Allocation TypeMulti-year Fund
Research Field Multimedia database
Intelligent informatics
Research InstitutionShibaura Institute of Technology (2015-2016)
The Institute of Physical and Chemical Research (2013-2014)

Principal Investigator

Arimoto Yoshiko  芝浦工業大学, 工学部, 准教授 (60586957)

Project Period (FY) 2013-04-01 – 2017-03-31
Project Status Completed (Fiscal Year 2016)
Budget Amount *help
¥4,290,000 (Direct Cost: ¥3,300,000、Indirect Cost: ¥990,000)
Fiscal Year 2015: ¥910,000 (Direct Cost: ¥700,000、Indirect Cost: ¥210,000)
Fiscal Year 2014: ¥1,690,000 (Direct Cost: ¥1,300,000、Indirect Cost: ¥390,000)
Fiscal Year 2013: ¥1,690,000 (Direct Cost: ¥1,300,000、Indirect Cost: ¥390,000)
Keywordsヒューマンエージェントインタラクション / マルチモーダル情報処理 / 感情 / 生理反応 / 表情 / 音声 / マルチモーダル対話 / 対話音声コーパス / 同調 / 模倣 / 感情表出
Outline of Final Research Achievements

This research aims to realize bidirectional emotional interaction between man and machine, to elucidate the behavior of the computer which makes the speaker as expressive as in the dialog with human and which makes them immerse in the dialog with the computer. Using the human-to-human dialog corpus, the synchronization of the emotional expressions (facial expressions, speech, autonomic nervous system responses) between the interlocutors was analyzed. As a result, although both the synchronized / the asynchronized emotional experience were observed between the interlocutors, only the synchronized emotional expression/reaction was observed between them. The result suggested that the mimicry of emotional expression/reaction was independent from the synchronization of emotional experience. The other research was conducted with the man-machine dialog corpus using character agent, and those material were analyzed to examine the synchronization of emotional expression with computer.

Report

(5 results)
  • 2016 Annual Research Report   Final Research Report ( PDF )
  • 2015 Research-status Report
  • 2014 Research-status Report
  • 2013 Research-status Report
  • Research Products

    (14 results)

All 2017 2016 2015 2014 2013 Other

All Journal Article (2 results) (of which Peer Reviewed: 2 results,  Open Access: 2 results) Presentation (11 results) (of which Int'l Joint Research: 3 results,  Invited: 2 results) Remarks (1 results)

  • [Journal Article] Multimodal features for automatic emotion estimation during face-to-face conversation2015

    • Author(s)
      Y. Arimoto and K. Okanoya
    • Journal Title

      Journal of Phonetic Society of Japan

      Volume: 19(1) Pages: 53-67

    • Related Report
      2015 Research-status Report
    • Peer Reviewed / Open Access
  • [Journal Article] Mutual emotional understanding in a face-to-face communication environment: How speakers understand and react to listeners' emotion in a game task dialog2015

    • Author(s)
      Y. Arimoto and K. Okanoya
    • Journal Title

      Acoustical Science and Technology

      Volume: 36 Issue: 4 Pages: 370-373

    • DOI

      10.1250/ast.36.370

    • NAID

      130005086497

    • ISSN
      0369-4232, 1346-3969, 1347-5177
    • Related Report
      2015 Research-status Report 2014 Research-status Report
    • Peer Reviewed / Open Access
  • [Presentation] 対話音声の感情認識における音響的特徴と言語的特徴の効果2017

    • Author(s)
      横山雅季, 有本泰子, 森大毅
    • Organizer
      日本音響学会2017年春季研究発表会
    • Place of Presentation
      東京, 日本
    • Year and Date
      2017-03-15
    • Related Report
      2016 Annual Research Report
  • [Presentation] クロスコーパスラベリングによる感情空間への感情カテゴリのマッピング -音声知覚および音響分析による検討-2016

    • Author(s)
      有本泰子, 森大毅
    • Organizer
      電子情報通信学会 HCGシンポジウム2016
    • Place of Presentation
      高知, 日本
    • Year and Date
      2016-12-07
    • Related Report
      2016 Annual Research Report
  • [Presentation] 音声からの知覚による感情カテゴリと感情次元の関係2016

    • Author(s)
      有本泰子, 森大毅
    • Organizer
      日本音響学会2016年秋季研究発表会講演
    • Place of Presentation
      富山, 日本
    • Year and Date
      2016-09-14
    • Related Report
      2016 Annual Research Report
  • [Presentation] Comparison of Emotional Understanding in Modality-controlled Environments using Multimodal Online Emotional Communication Corpus2016

    • Author(s)
      Y. Arimoto and K. Okanoya
    • Organizer
      The International Conference on Language Resources and Evaluation (LREC2016)
    • Place of Presentation
      Portoroz, (Slovenia)
    • Year and Date
      2016-05-23
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Accuracy of Automatic Cross-Corpus Emotion Labeling for Conversational Speech Corpus Commonization2016

    • Author(s)
      H. Mori and Y. Arimoto
    • Organizer
      The International Conference on Language Resources and Evaluation (LREC2016)
    • Place of Presentation
      Portoroz, (Slovenia)
    • Year and Date
      2016-05-23
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Comparison of Emotional Understanding in Modality-controlled Environments using Multimodal Online Emotional Communication Corpus2016

    • Author(s)
      Y. Arimoto and K. Okanoya
    • Organizer
      The International Conference on Language Resources and Evaluation (LREC2016)
    • Place of Presentation
      Portoroz, Slovenia
    • Year and Date
      2016-05-23
    • Related Report
      2015 Research-status Report
    • Int'l Joint Research
  • [Presentation] コミュニケーション場面におけるリアルな感情表出の分析2015

    • Author(s)
      有本泰子
    • Organizer
      日本音響学会2015年秋季研究発表会
    • Place of Presentation
      会津大学 (福島県会津若松市)
    • Year and Date
      2015-09-16
    • Related Report
      2015 Research-status Report
    • Invited
  • [Presentation] Emotional synchrony and covariation of behavioral/physiological reactions between interlocutors2014

    • Author(s)
      Y. Arimoto and K. Okanoya
    • Organizer
      the 17th Oriental COCOSDA (International Committee for the Co-ordination and Standardization of Speech Databases and Assessment Techniques) Conference 2014
    • Place of Presentation
      Cape Panwa Hotel, Phuket, Thai
    • Year and Date
      2014-09-10 – 2014-09-12
    • Related Report
      2014 Research-status Report
  • [Presentation] オンラインコミュニケーションにおけるモダリティ統制下の情動理解2014

    • Author(s)
      有本泰子, 江田康太郎, 菊池英明, 岡ノ谷一夫
    • Organizer
      日本音響学会2014年秋季研究発表会
    • Place of Presentation
      北海学園大学豊平キャンパス, 北海道
    • Year and Date
      2014-09-03 – 2014-09-05
    • Related Report
      2014 Research-status Report
  • [Presentation] 対話場面におけるマルチモーダル感情コミュニケーション2014

    • Author(s)
      有本泰子
    • Organizer
      ヒューマンストレス産業技術研究会第27回講演会
    • Place of Presentation
      千里ライフサイエンスセンター 5階 サイエンスホール, 大阪府
    • Year and Date
      2014-07-11
    • Related Report
      2014 Research-status Report
    • Invited
  • [Presentation] 音声チャットを利用したオンラインゲーム感情音声コーパス2013

    • Author(s)
      有本泰子, 河津宏美
    • Organizer
      日本音響学会2013年秋季研究発表会
    • Place of Presentation
      豊橋技科大
    • Related Report
      2013 Research-status Report
  • [Remarks] 感情評定値付きオンラインゲーム音声チャットコーパス(OGVC)

    • URL

      https://sites.google.com/site/ogcorpus/

    • Related Report
      2013 Research-status Report

URL: 

Published: 2014-07-25   Modified: 2019-07-29  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi