• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Development of Dialogue Management Mechanism for Embodied Conversational Agent in Engaging with Multiple Users

Research Project

Project/Area Number 23700183
Research Category

Grant-in-Aid for Young Scientists (B)

Allocation TypeMulti-year Fund
Research Field Intelligent informatics
Research InstitutionRitsumeikan University

Principal Investigator

HUANG HUNG-HSUAN  立命館大学, 情報理工学部, 准教授 (00572950)

Project Period (FY) 2011 – 2013
Project Status Completed (Fiscal Year 2013)
Budget Amount *help
¥4,420,000 (Direct Cost: ¥3,400,000、Indirect Cost: ¥1,020,000)
Fiscal Year 2013: ¥650,000 (Direct Cost: ¥500,000、Indirect Cost: ¥150,000)
Fiscal Year 2012: ¥1,040,000 (Direct Cost: ¥800,000、Indirect Cost: ¥240,000)
Fiscal Year 2011: ¥2,730,000 (Direct Cost: ¥2,100,000、Indirect Cost: ¥630,000)
Keywordsユーザ・インターフェース / 人工知能 / 音声対話システム / ユーザインターフェース
Research Abstract

There are more and more real-world deployed applications of embodied conversational agents (ECA's). These agent systems are usually used by groups of visitors rather than individuals. This situation is more complex than single user one, specific features are required. This project aims to build an information providing agent for collaborative decision making tasks. A Wizard-of-Oz (WOZ) experiment was conducted for collecting human/agent interaction data. By analyzing it, the model for identifying the addressee of user utterances have been developed with machine learning techniques on visual (face direction and movements) and acoustic (intensity, pitch, and speed) data of the corpus. We then incorporated it into a fully autonomous multiparty ECA. The system identifies the addressee and uses this information in language understanding and dialogue management. Finally, an evaluation experiment shows that the proposed addressee identification mechanism works well in a real-time system.

Report

(4 results)
  • 2013 Annual Research Report   Final Research Report ( PDF )
  • 2012 Research-status Report
  • 2011 Research-status Report
  • Research Products

    (31 results)

All 2014 2013 2012 2011 Other

All Journal Article (5 results) (of which Peer Reviewed: 5 results) Presentation (26 results)

  • [Journal Article] A Multiparty Conversation System with an Addressee Identification Mechanism based on Nonverbal Information2014

    • Author(s)
      中野有紀子,馬場直哉,黄 宏軒,林 佑樹
    • Journal Title

      Transactions of the Japanese Society for Artificial Intelligence

      Volume: 29 Issue: 1 Pages: 69-79

    • DOI

      10.1527/tjsai.29.69

    • NAID

      130003382440

    • ISSN
      1346-0714, 1346-8030
    • Related Report
      2013 Annual Research Report 2013 Final Research Report
    • Peer Reviewed
  • [Journal Article] 人対会話エージェントとの多人数会話における頭部方向と音声情報を用いた受話者推定機構2013

    • Author(s)
      馬場直哉,黄 宏軒,中野有紀子
    • Journal Title

      人工知能学会論文誌

      Volume: 28(2) Pages: 149-159

    • NAID

      130003362320

    • Related Report
      2013 Final Research Report
    • Peer Reviewed
  • [Journal Article] 人対会話エージェントとの多人数会話における頭部方向と音声情報を用いた受話者推定機構2013

    • Author(s)
      馬場直哉,黄 宏軒,中野有紀子
    • Journal Title

      人工知能学会論文 誌

      Volume: 28 Pages: 149-159

    • NAID

      130003362320

    • Related Report
      2012 Research-status Report
    • Peer Reviewed
  • [Journal Article] Evaluating a Virtual Agent Who Responses Attentively to Multiple Players in a Quiz Game2012

    • Author(s)
      Hung-Hsuan Huang and Toyoaki Nishida
    • Journal Title

      ヒューマンインターフェース学会論文誌

      Volume: 14(4) Pages: 351-365

    • NAID

      10031159747

    • Related Report
      2013 Final Research Report
    • Peer Reviewed
  • [Journal Article] Evaluating a Virtual Agent Who Responses Attentively to Multiple Players in a Quiz Game2012

    • Author(s)
      Hung-Hsaun Huang and Toyoaki Nishida
    • Journal Title

      ヒューマン・インタフェース学会論文誌

      Volume: 14 Pages: 351-365

    • NAID

      10031159747

    • Related Report
      2012 Research-status Report
    • Peer Reviewed
  • [Presentation] Detecting Address Estimation Errors from Users'Reactions in Multi-user Agent Conversation2014

    • Author(s)
      Hotta, R., Huang, H.H., Shochi Otogi, and Kawagoe, K
    • Organizer
      HCI International Conference (HCII 2014)
    • Place of Presentation
      Crete (Greece)
    • Year and Date
      2014-06-26
    • Related Report
      2013 Final Research Report
  • [Presentation] Detecting Address Estimation Errors from Users’Reactions in Multi-user Agent Conversation2014

    • Author(s)
      Ryo Hotta, Hung-Hsuan Huang, Shochi Otogi, and Kyoji Kawagoe
    • Organizer
      HCI International Conference (HCII 2014)
    • Place of Presentation
      Crete, Greece
    • Related Report
      2013 Annual Research Report
  • [Presentation] Finding the Timings for a Guide Agent to Intervene Inter-user Conversation in Considering Their Gaze Behaviors2013

    • Author(s)
      Otogi, S., Huang, H.H., Hotta, R., and Kawagoe, K
    • Organizer
      6th Workshop on Eye Gaze in Intelligent Human Machine Interaction, 15th International Conference on Multimodal Interaction (ICMI 2013)
    • Place of Presentation
      Sydney, (Australia)
    • Year and Date
      2013-12-13
    • Related Report
      2013 Final Research Report
  • [Presentation] 複数人ユーザとエージェントの会話における受話者推定の誤り時の状況の考察2013

    • Author(s)
      堀田 怜,乙木 翔地,黄 宏軒,川越 恭二
    • Organizer
      HAI シンポジウム 2013
    • Place of Presentation
      岐阜大学岐阜駅前サテライトキャンパス (岐阜県)
    • Year and Date
      2013-12-13
    • Related Report
      2013 Final Research Report
  • [Presentation] Implementation and Evaluation of Multimodal Addressee Identification, Mechanism for Multiparty Conversation Systems2013

    • Author(s)
      Nakano, Y., Baba, N., Huang, H.H., and Hayashi, Y
    • Organizer
      15th International Conference on Multimodal Interaction (ICMI 2013)
    • Place of Presentation
      Sydney, (Australia)
    • Year and Date
      2013-12-10
    • Related Report
      2013 Final Research Report
  • [Presentation] Finding the Timings for a Guide Agent to Intervene User-user Conversation to Provide Information Actively2013

    • Author(s)
      Otogi, S., Huang, H.H., Hotta, R., and Kawagoe, K
    • Organizer
      the 13th International Conference on Intelligent Virtual Agents (IVA 2013)
    • Place of Presentation
      Edinburgh (UK)
    • Year and Date
      2013-08-29
    • Related Report
      2013 Final Research Report
  • [Presentation] 複数人と会話するエージェントの受話者推定の誤り検知のための特徴量の検討2013

    • Author(s)
      堀田 怜,乙木 翔地,黄 宏軒,川越 恭二
    • Organizer
      第27回人工知能学会全国大会
    • Place of Presentation
      富山国際会議場(富山県)
    • Year and Date
      2013-06-05
    • Related Report
      2013 Final Research Report
  • [Presentation] 複数人ユーザ会話におけるエージェントの割り込みタイミングの推定手法の提案2013

    • Author(s)
      乙木 翔地,堀田 怜,黄 宏軒,川越 恭二
    • Organizer
      第27回人工知能学会全国大会
    • Place of Presentation
      富山国際会議場(富山県)
    • Year and Date
      2013-06-05
    • Related Report
      2013 Final Research Report
  • [Presentation] 複数人ユーザに対応可能な情報提供会話エージェント2013

    • Author(s)
      馬場直哉,黄 宏軒,林 佑樹,中野有紀子
    • Organizer
      第75回情報処理学会全国大会
    • Place of Presentation
      東北大学(宮城県)
    • Year and Date
      2013-03-07
    • Related Report
      2013 Final Research Report
  • [Presentation] 複数人ユーザ会話におけるエージェントの割り込みタイミングの推定手法の検討2012

    • Author(s)
      乙木 翔地,堀田 怜,黄 宏軒,川越 恭二
    • Organizer
      HAI シンポジウム2012
    • Place of Presentation
      京都工芸繊維大学(京都府)
    • Year and Date
      2012-12-08
    • Related Report
      2013 Final Research Report
  • [Presentation] Addressee Identification for Human-Human-Agent Multiparty Conversations in Different Proxemics2012

    • Author(s)
      Baba, N., Huang, H.H., Nakano, Y
    • Organizer
      4th Workshop on Eye Gaze in Intelligent Human Machine Interaction : Eye Gaze and Multimodality,14th International Conference on Multimodal Interaction (ICMI 2012)
    • Place of Presentation
      Santa Monica (USA)
    • Year and Date
      2012-10-26
    • Related Report
      2013 Final Research Report
  • [Presentation] グループ会話対応型会話エージェントにおける受話者推定システム2011

    • Author(s)
      馬場直哉,黄 宏軒,中野有紀子
    • Organizer
      HAI シンポジウム2011
    • Place of Presentation
      京都工芸繊維大学(京都府)
    • Year and Date
      2011-12-03
    • Related Report
      2013 Final Research Report
  • [Presentation] Making Virtual Conversational Agent Aware of the Addressee of Users' Utterances in Multi-user Conversation from Nonverbal Information2011

    • Author(s)
      Huang, H.H., Baba, N., Nakano, Y
    • Organizer
      13th International Conference on Multimodal Interaction (ICMI'11)
    • Place of Presentation
      Alicante (Spain)
    • Year and Date
      2011-11-16
    • Related Report
      2013 Final Research Report
  • [Presentation] Identifying Utterances Addressed to an Agent in Multiparty Human-Agent Conversations2011

    • Author(s)
      Baba, N., Huang, H.H., Nakano, Y
    • Organizer
      the 11th International Conference on Intelligent Virtual Agents (IVA'11)
    • Place of Presentation
      Reykjavik (Iceland)
    • Year and Date
      2011-09-16
    • Related Report
      2013 Final Research Report
  • [Presentation] Making Virtual Conversational Agent Aware of the Addressee of Users' Utterances in Multi-user Conversation from Nonverbal Information2011

    • Author(s)
      Hung-Hsuan Huang, Naoya Baba, Yukiko Nakano
    • Organizer
      13th International Conference on Multimodal Interaction (ICMI'11)
    • Place of Presentation
      Alicante, Spain
    • Related Report
      2011 Research-status Report
  • [Presentation] Predicting the Addressee in Multiparty Human-Agent Conversations based on Multimodal Nonverbal Information2011

    • Author(s)
      Naoya Baba, Hung-Hsuan Huang, and Yukiko Nakano
    • Organizer
      the 11th International Conference on Intelligent Virtual Agents (IVA'11)
    • Place of Presentation
      Reykjavik, Iceland
    • Related Report
      2011 Research-status Report
  • [Presentation] グループ会話対応型会話エージェントにおける受話者推定システム2011

    • Author(s)
      馬場直哉,黄 宏軒,中野有紀子
    • Organizer
      HAIシンポジウム2011
    • Place of Presentation
      京都工芸繊維大学(京都府)
    • Related Report
      2011 Research-status Report
  • [Presentation] Finding the Timings for a Guide Agent to Intervene Inter-user Conversation in Considering Their Gaze Behaviors

    • Author(s)
      Shochi Otogi, Hung-Hsuan Huang, Ryo Hotta, , and Kyoji Kawagoe
    • Organizer
      6th Workshop on Eye Gaze in Intelligent Human Machine Interaction, 15th International Conference on Multimodal Interaction (ICMI 2013)
    • Place of Presentation
      Sydney, Australia
    • Related Report
      2013 Annual Research Report
  • [Presentation] Implementation and Evaluation of Multimodal Addressee Identification Mechanism for Multiparty Conversation Systems

    • Author(s)
      Yukiko Nakano, Naoya Baba, Hung-Hsuan Huang, and Yuki Hayashi
    • Organizer
      15th International Conference on Multimodal Interaction (ICMI 2013)
    • Place of Presentation
      Sydney, Australia
    • Related Report
      2013 Annual Research Report
  • [Presentation] Finding the Timings for a Guide Agent to Intervene User-user Conversation to Provide Information Actively

    • Author(s)
      Shochi Otogi, Hung-Hsuan Huang, Ryo Hotta, , and Kyoji Kawagoe
    • Organizer
      13th International Conference on Intelligent Virtual Agents (IVA 2013)
    • Place of Presentation
      Edinburgh, UK
    • Related Report
      2013 Annual Research Report
  • [Presentation] 複数人ユーザとエージェントの会話における受話者推定の誤り時の状況の考察

    • Author(s)
      堀田 怜,乙木 翔地,黄 宏軒,川越 恭二
    • Organizer
      HAIシンポジウム2013
    • Place of Presentation
      岐阜大学岐阜駅前サテライトキャンパス,岐阜県
    • Related Report
      2013 Annual Research Report
  • [Presentation] 複数人と会話するエージェントの受話者推定の誤り検知のための特徴量の検討

    • Author(s)
      堀田 怜,乙木 翔地,黄 宏軒,川越 恭二
    • Organizer
      第27回人工知能学会全国大会
    • Place of Presentation
      富山国際会議場,富山県
    • Related Report
      2013 Annual Research Report
  • [Presentation] 複数人ユーザ会話におけるエージェントの割り込みタイミングの推定手法の提案

    • Author(s)
      乙木 翔地,堀田 怜,黄 宏軒,川越 恭二
    • Organizer
      第27回人工知能学会全国大会
    • Place of Presentation
      富山国際会議場,富山県
    • Related Report
      2013 Annual Research Report
  • [Presentation] Addressee Identification for Human-Human-Agent Multiparty Conversations in Different Proxemics

    • Author(s)
      Naoya Baba, Hung-Hsuan Huang, and Yukiko Nakano
    • Organizer
      4th Workshop on Eye Gaze in Intelligent Human Machine Interaction: Eye Gaze and Multimodality,14th International Conference on Multimodal Interaction (ICMI 2012)
    • Place of Presentation
      Santa Monica (USA)
    • Related Report
      2012 Research-status Report
  • [Presentation] 複数人ユーザに対応可能な情報提供会話エージェント

    • Author(s)
      馬場直哉,黄 宏軒,林 佑樹,中野有紀子
    • Organizer
      第75回情報処理学会全国大会
    • Place of Presentation
      東北大学(宮城県)
    • Related Report
      2012 Research-status Report
  • [Presentation] 複数人ユーザ会話におけるエージェントの割り込みタイミングの推定手法の検討

    • Author(s)
      乙木 翔地,堀田 怜,黄 宏軒,馬場 直哉,中野 有紀子,川越 恭二
    • Organizer
      HAIシンポジウム2012
    • Place of Presentation
      京都工芸繊維大学(京都府)
    • Related Report
      2012 Research-status Report

URL: 

Published: 2011-08-05   Modified: 2019-07-29  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi