会話エージェント研究共有プラットフォームの構築と利用技術の研究

2009 Fiscal Year Annual Research Report

• Project/Area Number: 19100001
• Research Institution: Kyoto University
• Principal Investigator: 西田 豊明 Kyoto University, 情報学研究科, 教授 (70135531)
• Co-Investigator(Kenkyū-buntansha): 河原 達也 京都大学, 学術情報メディアセンター, 教授 (00234104) ; 黒橋 禎夫 京都大学, 情報学研究科, 教授 (50263108) ; 角 康之 京都大学, 情報学研究科, 准教授 (30362578) ; 中野 有紀子 成蹊大学, 理工学部, 准教授 (40422505) ; 大本 義正 京都大学, 情報学研究科, 助教 (90511775)
• Keywords: エージェント / 会話エージェント / 対話エージェント / 音声言語 / 言語処理

Research Abstract

GECAシステムに言語処理システムSynGraphをGECAシステムに追加する作業を進めた.GECAを用いた複数ユーザインタラクション機能の改良,SKGを用いたコンテンツ供給機能との接続を行った.GECAを用いてインタラクティブに機器操作説明を行う会話エージェントシステムを試作した.人間どうしの自然な音声会話において,聞き手が興味を示しているあいづちの形態的・韻律的パターンを同定し,会話音声からあいづちを自動検出する方法について研究を行った.WWWコーパスの機械解析結果から計算される文要素関の共起尺度を観測変数として用い,変数間の相関を因子分析モデルによって分析した.CEBEと呼ぶ没入型WOZ環境の開発を進めた.現実空間をコンテンツ化するため,全方位カメラを用いて撮影した実空間を元に仮想空間を簡便に構築するシステムを開発した.3人会話データから特徴的な会話プロトコルの抽出を試みた.多人数会話エージェントの開発に向けた基礎データの収集と分析を行い,会話状況を検知できることを示した.社会的関係を特徴づけるパラメータを提案し,文化,社会的関係,そして非言語行動との相互関係を表現するモデルを構築した.人間の会話行動推定モデルを統合し,多人数会話における対話制御方式の提案と実装を行った.非言語的なインタラクション系列の模倣学習.教師なし学習と教師あり学習を組み合わせて,非言語的なインタラクションの模倣学習を実現するための基本アルゴリズムを構成した.会話エージェントとのインタラクションの自然さの指標作成.信号処理をもちいて生理指標からストレスの度合いを計測することにより,インタラクションの自然さを評価するシステムを構築した.エージェントによる場の盛り上がりと個人の盛り上がりの自動判別手法を実装した.

Research Products

• [Journal Article] Statistical Transformation of Language and Pronunciation Models for Spontaneous Speech Recognition (2010)
  • Author(s): Akita Y., Kawahara, T.
  • Journal Title: IEEE Trans. Audio, Speech & Language Processing (採録決定)
  • Peer Reviewed
• [Journal Article] Online unsupervised classification with model comparison in the Variational Bayes framework for voice activity detection (2010)
  • Author(s): Cournapear, D., Watanabe, S., Nakamura, A., Kawahara, T
  • Journal Title: IEEE J.Selected Topics in Signal Processing (採録決定)
  • Peer Reviewed
• [Journal Article] Bayes risk-based dialogue management for document retrieval system with speech interface (2010)
  • Author(s): Misu, T., Kawahara, T.
  • Journal Title: Speech Communication 52 Pages: 61-71
  • Peer Reviewed
• [Journal Article] 複数ユーザインタラクションにおける外発的な盛り上がりの雰囲気の判定方法と影響の検討 (2010)
  • Author(s): 大本義正, 三宅峰, 西田豊明
  • Journal Title: 電子情報通信学会情報システムソサイエティ和文論文誌 (採録決定)
  • Peer Reviewed
• [Journal Article] Constrained Motif Discovery in Time Series (2009)
  • Author(s): Mohammad, Y., Nishida, T.
  • Journal Title: New Generation Computing 27 Pages: 319-346
  • Peer Reviewed
• [Journal Article] From observation to simulation : generating culture-specific behavior for interactive systems (2009)
  • Author(s): Rehm, M., Nakano, Y.I., Andre, E., Nishida, T., Bee, N., Endrab, B., Wissner, M., Lipi, A.A., Huang, H.H.
  • Journal Title: AI & Society 24 Pages: 267-280
  • Peer Reviewed
• [Journal Article] Toward a multi-culture adaptive virtual tour guide agent with a modular approach (2009)
  • Author(s): Huang, H.H., Cerekovic, A., Pandzic, I.S., Nakano, Y.I., Nishida, T.
  • Journal Title: AI & Society 24 Pages: 225-235
  • Peer Reviewed
• [Journal Article] Juliusを用いた音声認識インタフェースの作成 (2009)
  • Author(s): 李晃伸, 河原達也
  • Journal Title: ヒューマンインタフェース学会誌 11 Pages: 31-38
  • Peer Reviewed
• [Presentation] Social Intelligence Design for Knowledge Circulation (2010)
  • Author(s): Toyoaki Nishida
  • Organizer: atabases in Networked Information Systems(DNIS)
  • Place of Presentation: University of Aizu
  • Year and Date: 2010-03-29
• [Presentation] 因子分析による日本語文末表現の機能・叙法分析 (2010)
  • Author(s): 玉城伸仁, 黒橋禎夫
  • Organizer: 言語処理学会第16回年次大会(2010)
  • Place of Presentation: 東京大学
  • Year and Date: 2010-03-11
• [Presentation] 知識空間の可視化と共有支援システムの開発とその利用 (2010)
  • Author(s): 神田賢一, 大本義正, 西田豊明
  • Organizer: 第72回情報処理学会全国大会
  • Place of Presentation: 東京大学
  • Year and Date: 2010-03-10
• [Presentation] Webからの情報抽出に基づく雑談的な対話の生成 (2010)
  • Author(s): 吉野幸一郎, 河原達也
  • Organizer: 言語処理学会第16回年次大会(2010)
  • Place of Presentation: 東京大学
  • Year and Date: 2010-03-09
• [Presentation] ポスター会話中の音リアクションイベントに基づくホットスポットの抽出 (2010)
  • Author(s): 須見康平, 河原達也
  • Organizer: 第72回情報処理学会全国大会
  • Place of Presentation: 東京大学
  • Year and Date: 2010-03-09
• [Presentation] エージェントを介した遠隔地コミュニケーションの実現に向けたWOZシステムの開発 (2010)
  • Author(s): 大橋洋輝, 高橋暁弘, 岡田将吾, 大本義正, 西田豊明
  • Organizer: 第72回情報処理学会全国大会
  • Place of Presentation: 東京大学
  • Year and Date: 2010-03-09
• [Presentation] 交渉時のエージェントの性格パラメータの影響の検討 (2010)
  • Author(s): 三宅峰, 大本義正, 西田豊明
  • Organizer: 第72回情報処理学会全国大会
  • Place of Presentation: 東京大学
  • Year and Date: 2010-03-09
• [Presentation] A Socio-Cultural Model Based on Empirical Data of Cultural and Social Relationship (2010)
  • Author(s): Lipi, A.A., Nakano, Y., Rehm, M.
  • Organizer: International Workshop on Agents in Cultural Context, 2010
  • Place of Presentation: 京都大学
  • Year and Date: 2010-02-22
• [Presentation] Performance of an Artificial Emotion System of a Virtual Agent in the Cultural Context (2010)
  • Author(s): Kiselev, A., Abdikeyev, N., Nishida, T.
  • Organizer: International Workshop on Agents in Cultural Context, 2010
  • Place of Presentation: 京都大学
  • Year and Date: 2010-02-22
• [Presentation] Capture and Express Behavior Environment (CEBE) for realizing enculturating human-agent interaction (2010)
  • Author(s): Ohmoto, Y., Takahahshi, A., Ohashi, H., Nishida, T.
  • Organizer: International Workshop on Agents in Cultural Context, 2010
  • Place of Presentation: 京都大学
  • Year and Date: 2010-02-22
• [Presentation] 聞き手の興味・関心を示すあいづちの生起する会話文脈の分析 (2010)
  • Author(s): 高梨克也, 常志強, 河原達也
  • Organizer: 第58回 人工知能学会 言語・音声理解と対話処理研究会
  • Place of Presentation: 神戸市
  • Year and Date: 2010-02-12
• [Presentation] Estimating User's Engagement from Eye-gaze Behaviors in Human-Agent Conversations (2010)
  • Author(s): Nakano, Y.I., Ishii, R.
  • Organizer: 2010 International Conference on Intelligent User Interfaces(IUI2010)
  • Place of Presentation: Hong Kong, China
  • Year and Date: 2010-02-09
• [Presentation] New perspectives on spoken language understanding : Does machine need to fully understand speech? (2009)
  • Author(s): Kawahara, T.
  • Organizer: IEEE Workshop on Automatic Speech Recognition and Understanding
  • Place of Presentation: 京都
  • Year and Date: 2009-12-10
• [Presentation] Improving Emotional Communication Skills of the Virtual Agent for Customer-Engineer Interactions (2009)
  • Author(s): Kiselev, A., Abdikeev, N., Nishida, T.
  • Organizer: the 8th International Workshop on Social Intelligence Design(SID 2009)
  • Place of Presentation: 京都
  • Year and Date: 2009-11-09
• [Presentation] the 5th International Conference on Active Media Technology(AMT 2009) (2009)
  • Author(s): Cerekovic, A., Huang, H.H., Furukawa, T., Yamaoka, Y., Pandzic, I.S., Nishida, T., Nakano, Y.I.
  • Organizer: Intelligent Virtual Agents 2009
  • Place of Presentation: Beijin, China
  • Year and Date: 2009-10-23
• [Presentation] Unsupervised simultaneous learning of gestures, actions and their associations for Human-Robot Interaction (2009)
  • Author(s): Mohammad, Y., Nishida, T., Okada, S.
  • Organizer: IROS 2009
  • Place of Presentation: St. Louis, MO, USA
  • Year and Date: 2009-10-15
• [Presentation] Recent development of open-source speech recognition engine Julius (2009)
  • Author(s): Lee, A., Kawahara, T.
  • Organizer: APSIPA-ASC
  • Place of Presentation: 札幌
  • Year and Date: 2009-10-04
• [Presentation] The Lessons Learned in Developing Multi-user Attentive Quiz Agents (2009)
  • Author(s): Huang, H.H., Furukawa, T., Ohashi, H., Cerekovic, A., Yamaoka, Y., Pandzic, I.S., Nakano, Y.I., Nishida, T.
  • Organizer: Intelligent Virtual Agents 2009
  • Place of Presentation: Amsterdam, the Netherlands
  • Year and Date: 2009-09-14
• [Presentation] A WFST-based log-linear framework for speaking-style transformation (2009)
  • Author(s): Neubig, G., Mori, S., Kawahara, T.
  • Organizer: INTERSPEECH
  • Place of Presentation: Brighton UK
  • Year and Date: 2009-09-08
• [Presentation] Optimization of dereverberation parameters based on likelihood of speech recognizer (2009)
  • Author(s): Gomez, R., Kawahara, T.
  • Organizer: INTERSPEECH
  • Place of Presentation: Brighton UK
  • Year and Date: 2009-09-08
• [Presentation] Acoustic event detection for spotting hot spots in podcasts (2009)
  • Author(s): Sumi, K., Kawahara, T., Ogata, J., Goto, M.
  • Organizer: INTERSPEECH
  • Place of Presentation: Brighton UK
  • Year and Date: 2009-09-08
• [Presentation] Automatic transcription system for meetings of the Japanese National Congress (2009)
  • Author(s): Akita, Y., Mimura, M., Kawahara, T.
  • Organizer: INTERSPEECH
  • Place of Presentation: Brighton UK
  • Year and Date: 2009-09-07
• [Presentation] Some Pitfalls for Developing Enculturated Conversational Agents (2009)
  • Author(s): Rehm, M., Andre E., Nakano, Y.I.
  • Organizer: 13th International Conference on Human-Computer Interaction(HCII09)
  • Place of Presentation: San Diego, CA, USA
  • Year and Date: 2009-07-22
• [Presentation] Multimodal Corpus Analysis as a Method for Ensuring Cultural Usability of Embodied Conversational Agents (2009)
  • Author(s): Nakano, Y.I., Rehm, M.
  • Organizer: 13th International Conference on Human-Computer Interaction(HCII09)
  • Place of Presentation: San Diego, CA, USA
  • Year and Date: 2009-07-22
• [Presentation] A Parameter-based Model for Generating Culturally Adaptive Nonverbal Behaviors in Embodied Conversational Agents (2009)
  • Author(s): Lipi, A.A., Nakano, Y.I., Rehm, M.
  • Organizer: 13th International Conference on Human-Computer Interaction(HCII09)
  • Place of Presentation: San Diego, CA, USA
  • Year and Date: 2009-07-22
• [Presentation] Toward the Knowledge Circulation between Institutions and Public Audiences with Virtual Agents (2009)
  • Author(s): Huang, H.H., Kubota, H., Y., Nishida, T.
  • Organizer: The Twenty Second International Conference on Industrial, Engineering & Other Applications of Applied Intelligent Systems(IEA/AIE 2009)
  • Place of Presentation: Hainan, Taiwan
  • Year and Date: 2009-06-27
• [Presentation] Measuring Naturalness during Close Encounters Using Physiological Signal Processing (2009)
  • Author(s): Mohammad, Y., Nishida, T.
  • Organizer: The Twenty Second International Conference on Industrial, Engineering & Other Applications of Applied Intelligent Systems(IEA/AIE 2009)
  • Place of Presentation: Hainan, Taiwan
  • Year and Date: 2009-06-25
• [Presentation] Robust Singular Spectrum Transform (2009)
  • Author(s): Mohammad, Y., Nishida, T.
  • Organizer: The Twenty Second International Conference on Industrial, Engineering & Other Applications of Applied Intelligent Systems(IEA/AIE 2009)
  • Place of Presentation: Hainan, Taiwan
  • Year and Date: 2009-06-24
• [Presentation] Communicating with Multiple Users for Embodied Conversational Agents (2009)
  • Author(s): Huang, H.H., Cerekovic, A., Furukawa, T., Yamaoka, Y., Ohashi, H., Pandzic, I.S., Nakano, Y.I., Nishida, T.
  • Organizer: 10th International Conference on Telecommunications(ConTEL2009)
  • Place of Presentation: Zagreb, Croztia
  • Year and Date: 2009-06-08
• [Presentation] Incorporating Intentional and Emotional Behaviors into a Virtual Human for Better Customer-Engineer-Interaction (2009)
  • Author(s): Hacker, B.A., Wankerl, T., Kiselev, A., Huang, H.H., Merckel, L., Okada, S., Schlichter, J., Abdikkev, A. Nishida, T.
  • Organizer: 10th International Conference on Telecommunications(ConTEL2009)
  • Place of Presentation: Zagreb, Croztia
  • Year and Date: 2009-06-08
• [Presentation] Conversational Informatics for Situated Communication (2009)
  • Author(s): Toyoaki Nishida
  • Organizer: The 10th International Conference on Telecommunications(ConTEL 2009)
  • Place of Presentation: Zagreb, Croztia
  • Year and Date: 2009-06-08
• [Presentation] Language model transformation applied to lightly supervised training of acoustic model for congress meetings (2009)
  • Author(s): Kawahara, T., Mimura, M., Akita, Y.
  • Organizer: IEEE-ICASSP
  • Place of Presentation: Taipei, Taiwan
  • Year and Date: 2009-04-23
• [Presentation] Social Intelligence Design for Intelligent Decision Technologies (2009)
  • Author(s): Toyoaki Nishida
  • Organizer: KES IDT 2009
  • Place of Presentation: Himeji, Japan
  • Year and Date: 2009-04-23