生理的情報に基づく擬人化エージェントとの効果的なインタラクションモデル

• Project/Area Number: 17680013
• Research Category: Grant-in-Aid for Young Scientists (A)
• Allocation Type: Single-year Grants
• Research Field: Intelligent informatics
• Research Institution: National Institute of Informatics
• Principal Investigator: HELMUT Plendinger (HELMUT PRENDINGER / HELMUT Prendinger) National Institute of Informatics, コンテンツ科学研究系, 准教授 (40390596)
• Project Period (FY): 2005 – 2007
• Project Status: Completed (Fiscal Year 2007)
• Budget Amount: ¥24,440,000 (Direct Cost: ¥18,800,000、Indirect Cost: ¥5,640,000); Fiscal Year 2007: ¥7,410,000 (Direct Cost: ¥5,700,000、Indirect Cost: ¥1,710,000); Fiscal Year 2006: ¥5,460,000 (Direct Cost: ¥4,200,000、Indirect Cost: ¥1,260,000); Fiscal Year 2005: ¥11,570,000 (Direct Cost: ¥8,900,000、Indirect Cost: ¥2,670,000)
• Keywords: ユーザインタフェース / 人工知能 / エージェント / 感性情報学 / モデル化 / ユーザインターフェース / インテリジェントユーザインタフェース / マルチモダリティ / 擬人化エージェント

Research Abstract

本プロジェクトの目的は、ユーザの感情状態や興味に的確に対応する新しいタイプのコミュニケーションモデルを実現することであった。このプロジェクトの重要な特徴は以下のとおりである。
・ ユーザの感情の状態を認識するために生体信号とテキストを利用する。
・ ユーザの興味の状態(「視覚的注意」能力)を認識するために眼球運動分析を利角する。
・ ユーザとのインタラクションに仮想エージェントを用いる。
本プロジェクトの重要な成果は以下のとおりである。
・ 実時間で生体信号を処理し、それらを頑健かつ正確にユーザの感情状態として解釈するという感情ベースの新たなインタラクションモデル。実現されたシステムでは、インタラクティブなゲームにおいて、仮想エージェントがユーザの感情に適切な反応を提供する。実証研究により、このシステムの有効性を示した。
ユーザとの実時間インタラクションを実現するために、Multimodal Presentation Markup Language(MPML)の新バージョンを設計および実装する必要があった。その結果として開発したMPML3Dオーサリング言語は、反応(reactive)型モデルに基づいており、それによりユーザや他の仮想エージェントインタフェースから入力がプレゼンテーションの最中にいつでも可能なため、自然で自由なインタラクションを実現できる。MPML3Dは現在、最も強力で柔軟性のあるオーサリング言語であり、専門家以外の人でも使いやすいものである。
・ 開発した「Visual Attentive Presentation Agents(注視プレゼンテーションエージェント)」システムは、ロスアンゼルス(アメリカ合衆国)で開催された第6回International Conference on Intelligent Virtual Agents(IVA'06)においてGALA Award2006を受賞した(「Lifelike Agent Application」部門)。
・ 国際的学術論文8本(ヨーロッパのもの1本を含む)、国際会議論文28本、国際ワークショップ論文4本、国際デモ論文1件。

Research Products

• [Journal Article] Linguistic interpretation of emotions for affect sensing (2008)
  • Author(s): Mostafa Shaikh
  • Journal Title: Proceedings of The Third International Joint Conference on Natural Language Processing(IJCNLP-08), Hyderabad, India Pages: 895-900
  • Peer Reviewed
• [Journal Article] Phase-based gesture motion parametrization and transitions for convers ational agents with MPML3D (2008)
  • Author(s): Klaus Briigmann
  • Journal Title: Proceedings 2nd International Conference on INtelligent TEchnologies for interactive enterTAINment(INTETAIN-08), Playa del Carmen, Cancun Mexico (ACM Digital Library)
  • Peer Reviewed
• [Journal Article] Emotion sensitive news agent: An approach towards user centric emotion sensing from the news (2007)
  • Author(s): Mostafa Shaikh
  • Journal Title: The 2007 IEEE/WIC/ACM International Conference on Web Intelligence(WI-07), Silicon Valley, USA Pages: 614-620
  • Peer Reviewed
• [Journal Article] Automated generation of non-verbal behavior for virtual embodied characters (2007)
  • Author(s): Werner Breitfub
  • Journal Title: Proceedings 9th International Conference on Multimodal Interfaces(ICMI-07), Nagoya, Japan Pages: 319-322
  • Peer Reviewed
• [Journal Article] Interest estimation based on dynamic Bayesian networks for visual attentive presentation agents (2007)
  • Author(s): Boris Brandherm
  • Journal Title: Proceedings 9th International Conference on Multimodal Interfaces(ICMI-07), Nagoya, Japan Pages: 346-349
  • Peer Reviewed
• [Journal Article] Rules of emotions: a linguistic interpretation of an emotion model for affect sensing from texts (2007)
  • Author(s): Mostafa Shaikh
  • Journal Title: Affective Computing and Intelligent Interaction Proceedings 2nd International Conference on Affective Computing and Intelligent Interaction(ACII-07), Lisbon, Portugal 4738/2007 Pages: 737-738
  • Peer Reviewed
• [Journal Article] Assessing sentiment of text by semantic dependency and contextual valence analysis (2007)
  • Author(s): Mostafa Shaikh
  • Journal Title: Affective Computing and Intelligent Interaction Proceedings 2nd International Conference on Affective Computing and Intelligent Interaction(ACII-07), Lisbon, Portugal 4738/2007 Pages: 191-202
  • Peer Reviewed
• [Journal Article] Textual affect sensing for sociable and expressive online communication (2007)
  • Author(s): Alena Neviarouskaya
  • Journal Title: Second International Conference, ACII 2007 Lisbon, Portugal, September 12-14, 2007 Proceedings 4738/2007 Pages: 218-229
  • Peer Reviewed
• [Journal Article] An expressive avatar for instant messaging endowed with emotional intelligence (2007)
  • Author(s): Alena Neviarouskaya
  • Journal Title: Proceedings 7th International Conference on Intelligent Virtual Agents(IVA-07), Paris, France 4722/2007 Pages: 395-396
  • Peer Reviewed
• [Journal Article] T2D: Generating dialogues between virtual agents automatically (2007)
  • Author(s): Paul Piwek
  • Journal Title: Proceedings 7th International Conference on Intelligent Virtual Agents(IVA-07), Paris, France 4722/2007 Pages: 161-174
  • Peer Reviewed
• [Journal Article] Attentive presentation agents (2007)
  • Author(s): Tobias Eichner
  • Journal Title: Proceedings 7th International Conference on Intelligent Virtual Agents(IVA-07)}, Paris, France 4722/2007 Pages: 283-295
  • Peer Reviewed
• [Journal Article] Automatic generation of multi-modal dialogue from text based on discourse structure analysis (2007)
  • Author(s): Helmut Prendinger
  • Journal Title: Proceedings 1st IEEE International Conference on Semantic Computing(ICSC-07), Irvine, CA, USA Pages: 27-36
  • Peer Reviewed
• [Journal Article] A novel method for automatically generating multi-modal dialogue (2007)
  • Author(s): Helmut Prendinger
  • Journal Title: International Journal of Semantic Computing 1 Pages: 319-334
  • Peer Reviewed
• [Journal Article] Recognition of affect conveyed by text messaging in online communication (2007)
  • Author(s): Alena Neviarouskaya
  • Journal Title: Online Communities and Social Computing Proceedings 12th International Conference on Hu man-Computer Interaction, Beijing, China(LNCS) 4564/2007 Pages: 141-150
  • Peer Reviewed
• [Journal Article] Graceful anytime interruptibility for virtual agents (2007)
  • Author(s): Klaus Brugmann
  • Journal Title: ACM International Conference Proceeding Series Proceedings ACM SIGCHI International Conference on Advances in Computer Entertainment Technology(ACE-07), Salzburg, Austria 203 Pages: 284-285
  • Peer Reviewed
• [Journal Article] Gaze-based infotainment agents (2007)
  • Author(s): Helmut Prendinger
  • Journal Title: Proceedings ACM SIGCHI International Conference on Advances in Computer Entertainment Technology (ACE-07), Salzburg, Austria 203 Pages: 284-285
  • Peer Reviewed
• [Journal Article] Highly realistic 3D presentation agents with visual attention capability (2007)
  • Author(s): Arjen Hoekstra
  • Journal Title: Smart Graphics 8th International Symposium, SG 2007, Kyoto, Japan, June 25-27, 2007. Proceedings 4569/2007 Pages: 73-84
  • Peer Reviewed
• [Journal Article] Symmetric Multi-modality revisited: Unveiling users' physiological activity (2007)
  • Author(s): Helmut Prendinger
  • Journal Title: The LEEE Transactions on Industrial Electronics 54 Pages: 692-698
  • Peer Reviewed
• [Journal Article] User interest estimation using cross-modal computation (2007)
  • Author(s): Takamasa Koshizen
  • Journal Title: International Journal of Commputational Intelligence Research 3 Pages: 177-190
  • Peer Reviewed
• [Journal Article] MPML3D : a reactive framework for the Multimodal Presentation Markup Language (2007)
  • Author(s): M.Nischt, H.Prendinger, E.Andre, M.Ishizuka
  • Journal Title: Proceedings 6th International Conference on Intelligent Virtual Agents (IVA-06), Marina del Rey, CA, USA (Springer LNAI 4133, Berlin Heidelberg) Pages: 218-229
• [Journal Article] Automatic preference detection by analyzing the gaze 'cascade effect' (2007)
  • Author(s): N.Bee, H.Prendinger, E.Andre, M.Ishizuka
  • Journal Title: Electronic Proceedings 2nd COGAIN Annual Conference on Communication by Gaze Interaction, Turin, Italy Pages: 61-64
• [Journal Article] SenseNet : A linguistic tool to visualize numerical-valence based sentiment of textual data (2007)
  • Author(s): M.Shaikh, H.Prendinger, M.Ishizuka
  • Journal Title: Proceedings 5th International Conference on Natural Language Processing (ICON-07), Hyderabad, India Pages: 147-152
• [Journal Article] Analysis of affect expressed through the evolving language of online communication (2007)
  • Author(s): A.Neviarouskaya, H.Prendinger, M.Ishizuka
  • Journal Title: Proceedings 2007 International Conference on Intelligent User Interfaces (IUI-07), Hawaii, USA (ACM Press, New York) Pages: 278-281
• [Journal Article] Cascading hand and eye movement for augmented reality videoconferencing (2007)
  • Author(s): I.Barakonyi, H.Prendinger, D.Schmalstieg, M.Ishizuka
  • Journal Title: Proceedings 2nd IEEE Symposium on 3D User Interfaces 2007 (3DUI-07), Charlotte, North Carolina, USA (IEEE Computer Society) Pages: 71-78
• [Journal Article] Eye movements as indices for the utility of life-like interface agents : A pilot study (2007)
  • Author(s): H.Prendinger, C.Ma, M.Ishizuka
  • Journal Title: Interacting with Computers Journal (Elsevier) 19・2 Pages: 281-292
• [Journal Article] Narrowing the social gap among people involved in global dialog : automatic emotion detection in blog posts (2007)
  • Author(s): A.Neviarouskaya, H.Prendinger, M.Ishizuka
  • Journal Title: Proceedings International Conference on Weblogs and Social Media (ICWSM-07), Boulder, Colorado, USA, (Omnipress) Pages: 293-294
• [Journal Article] A learning system for user modeling by combined cognitive and affective modeling for user interest estimation (2006)
  • Author(s): T.Koshizen, Y.Hasegawa, H.Tusjino, M.Kon, K.Aihara, H.Prendinger
  • Journal Title: Proceedings 7th International Conference on Cognitive Modelling (ICCM-06), Trieste, Italy (Edizioni Goliardiche, Trieste) Pages: 184-189
• [Journal Article] Creating three-dimensional animated characters : An experience report and recommendations of good practice (Invited paper) (2006)
  • Author(s): M.Nischt, H.Prendinger, E.Andre, M.Ishizuka
  • Journal Title: The European Journal for the Informatics Professional (Cepis) VII・2 Pages: 36-41
• [Journal Article] Web presentation system using RST events (Short paper) (2006)
  • Author(s): A.Nakasone, H.Prendinger, M.Ishizuka
  • Journal Title: Proceedings 5th International Conference on Autonomous Agents and Multiagent Systems (AAMAS-06), Hakodate, Japan (ACM Press, New York) Pages: 955-957
• [Journal Article] AutoSelect : What You Want Is What You Get. Real-time processing of visual attention and affect (2006)
  • Author(s): N.Bee, H.Prendinger, A.Nakasone, E.Andre, M.Ishizuka
  • Journal Title: Proceedings Tutorial and Research Workshop on Perception and Interactive Technologies (PIT-06), Kloster Irsee, Germany (Springer LNAI 4021, Berlin Heidelberg) Pages: 218-229
• [Journal Article] A study in users' physiological response to an empathic interface agent (2006)
  • Author(s): H.Prendinger, C.Becker, M.Ishizuka
  • Journal Title: International Journal of Humanoid Robotics (World Scientific) 3・3 Pages: 371-391
• [Journal Article] Describing and generating multimodal contents featuring affective lifelike agents with MPML (2006)
  • Author(s): Mitsuru Ishizuka, Helmut Prendinger
  • Journal Title: New Generation Computing Journal Vol.24 Pages: 97-128
• [Journal Article] A cognitively based approach to affect sensing from text (2006)
  • Author(s): Mostafa Shaikj, Helmut Prendinger, Mitsuru Ishizuka
  • Journal Title: Proceedings ACM International Conference on Intelligent Interfaces (IUI-06), Sydney, Australia(ACM Press) Pages: 349-351
• [Journal Article] A new user-machine interactive scheme using cross-modal computation for deep interest estimation. Towards quantifying user satisfaction (2006)
  • Author(s): Motohri Kon, Takamasa Koshizen, Helmut Prendinger
  • Journal Title: Proceedings of Workshop on Effective Multimodal Dialogue Interfaces, in conjunction with the ACM International Conference on Intelligent User Interfaces (IUI-06), Sydney, Australia Pages: 25-34
• [Journal Article] A creative abduction approach to scientific and knowledge discovery (2005)
  • Author(s): Helmut Prendinger, Mitsuru Ishizuka
  • Journal Title: Knowledge-Based Systems Journal Vol.18,No.7 Pages: 321-326
• [Journal Article] Human physiology as a basis for designing and evaluating affective communication with life-like characters (2005)
  • Author(s): Helmut Prendinger, Mitsuru Ishizuka
  • Journal Title: IEICE Transactions on Information and Systems, Special Section on Life-like Agent and its Communication Vol.E88-D, No.11 Pages: 2453-2460
• [Journal Article] Affective communication in online chat using physiological sensors and animated text (2005)
  • Author(s): Hua Wang, Helmut Prendinger, Mitsuru Ishizuka, Takeo Igarashi
  • Journal Title: Journal of the Human Interface Society, Special Issue on "Social Interface" Vol.7,No.1 Pages: 39-45
• [Journal Article] The Empathic Companion : A character-based interface that addresses users' affective states (2005)
  • Author(s): Helmut Prendinger, Mitsuru Ishizuka
  • Journal Title: International Journal of Applied Artificial Intelligence Vol.19,Nos.3-4 Pages: 267-285
• [Journal Article] Using human physiology to evaluate subtle expressivity of a virtual quizmaster in a mathematical game (2005)
  • Author(s): Helmut Prendinger, Junichiro Mori, Mitsuru Ishizuka
  • Journal Title: International Journal of Human-Computer Studies Vol.62 Pages: 231-245
• [Journal Article] Evaluating affective feedback of the 3D agent Max in a competitive cards game (2005)
  • Author(s): Christian Becker, Helmut Prendinger, Mitsuru Ishizuka, Ipke Wachsmuth
  • Journal Title: Proceedings 1st International Conference on Affective Computing and Intelligent Interaction (ACII-05), Springer [NCS 3784, Beijing, China Pages: 466-473
• [Journal Article] Emotion estimation and reasoning based on affective textual interaction (2005)
  • Author(s): Chunling Ma, Helmut Prendinger, Mitsuru Ishizuka
  • Journal Title: Proceedings 1st International Conference on Affective Computing and Intelligent Interaction (ACII-05), Springer [NCS 3784, Beijing, China Pages: 622-628
• [Journal Article] Understanding the effect of life-like interface agents through users' eye movements (2005)
  • Author(s): Helmut Prendinger, Chunling Ma, Jin Yingzi, Arturo Nakasone, Mitsuru Ishizuka
  • Journal Title: Proceedings 7th International Conference on Multimodal Interfaces (ICMI-05), Trento, Italy(ACM Press) Pages: 108-115
• [Journal Article] A chat system based on emotion estimation from text and embodied conversational messengers (2005)
  • Author(s): Chunling Ma, Helmut Prendinger, Mitsuru Ishizuka
  • Journal Title: Proceedings 4th International Conference on Entertainment Computing (ICEC-05), Springer LNCS 3711, Kobe, Sanda, Japan Pages: 535-538
• [Journal Article] Emotion recognition from electromyography and skin conductance (2005)
  • Author(s): Arturo Nakasone, Helmut Prendinger, Mitsuru Ishizuka
  • Journal Title: Proceedings 5th International Workshop on Biosignal Interpretation (BSI-05), Tokyo, Japan Pages: 219-222
• [Journal Article] Recognizing, modeling, and responding to users' affective states (2005)
  • Author(s): Helmut Prendinger, Junichiro Mori, Mitsuru Ishizuka
  • Journal Title: Proceedings 10th International Conference on User Modeling (UM-05), Springer LNAI 3538, Edinburgh, Scotland Pages: 60-69
• [Journal Article] Evaluating the interaction with synthetic agents using attention and affect tracking (2005)
  • Author(s): Helmut Prendinger, Chunling Ma, Jin Yingzi, Kazutaka Kushida, Mitsuru Ishizuka
  • Journal Title: Proceedings 4th International Conference on Autonomous Agents and Multi Agent Systems ACM Press, Utrecht, The Netherlands Pages: 1099-1100
• [Journal Article] Eye movement as an indicator of users' involvement with embodied interfaces at the low level (2005)
  • Author(s): Chunling Ma, Helmut Prendinger, Mitsuru Ishizuka
  • Journal Title: Proceedings AISB-05 Symposium on "Conversational Informatics for Supporting Social Intelligence and Interaction - Situational and Environmental Information Enforcing Involvement in Conversation", Herfordshire, UK Pages: 136-143
• [Journal Article] Estimating user affect and focus of attention from physiological signals in life-like character based interaction scenarios (2005)
  • Author(s): Helmut Prendinger
  • Journal Title: The 13th Annual Workshop of the SIG Adaptivity and User Modeling in Interactive Systems (ABIS-05), at LWA-05, Saarbrucken, Germany Pages: 61-62
• [Journal Article] Designing for affective interaction with embodied interfaces. (2005)
  • Author(s): Helmut Prendinger
  • Journal Title: The 8th Biennial Israeli Symposium on Foundations of AI (BISFAI-05), Haifa, Israel Pages: 25-25
• [Journal Article] Physiologically interactive gaming with the 3D agent Max (2005)
  • Author(s): Curistian Becker, Arturo Nalasone, Helmut Prendinger, Mitsuru Ishizuka, Ipke Wachsmuth
  • Journal Title: Proceedings of JSAI 2005 Workshop on Conversational Informatics, in conj.with JSAI-05, Kita-Kyushu, Japan(ISBN 4-915905-21-7 c-3004(JSAI)) Pages: 37-42
• [Journal Article] Does non-verbal behavior of an embodied agent matter? (2005)
  • Author(s): Helmut Prendinger, Chunling Ma, Junichiro Mori, Mitsuru Ishizuka
  • Journal Title: Proceedings of the 2005 International Conference on Active Media Technology (AMT-05), IEEE, Takamatsu, Kagawa, Japan Pages: 535-540
• [Journal Article] Empathy for Max(Preliminary project report) (2005)
  • Author(s): Christian Becker, Helmut Prendinger, Mitsuru Ishizuka, Ipke Wachsmuth
  • Journal Title: Proceedings of the 2005 International Conference on Active Media Technology (AMT-05), IEEE, Takamatsu, Kagawa, Japan Pages: 541-545
• [Journal Article] A chat system based on emotion estimation from text and embodied conversational messengers (Preliminary report) (2005)
  • Author(s): Chunling Ma, Alexander Osherenko, Helmut Prendinger, Mitsuru Ishizuka
  • Journal Title: Proceedings of the 2005 International Conference on Active Media Technology (AMT-05), IEEE, Takamatsu, Kagawa, Japan Pages: 546-548
• [Remarks]
  • URL: http://research.nii.ac.jp/~prendinger/Kakenhi_FY2005-07/Kakenhi_FY2005-07.html
• [Remarks]
  • URL: http://research.nii.ac.jp/~prendinger/