• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Understanding and generation of referring expressions using gaze

Research Project

Project/Area Number 21300049
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeSingle-year Grants
Section一般
Research Field Intelligent informatics
Research InstitutionTokyo Institute of Technology

Principal Investigator

TOKUNAGA Takenobu  東京工業大学, 大学院・情報理工学研究科, 教授 (20197875)

Co-Investigator(Kenkyū-buntansha) KURIYAMA Naoko  東京工業大学, 大学院・社会理工学研究科, 助教 (90361782)
TERAI Asuka  東京工業大学, グローバルエッジ研究院, テニュア・トラック助教 (70422540)
Project Period (FY) 2009 – 2011
Project Status Completed (Fiscal Year 2011)
Budget Amount *help
¥18,850,000 (Direct Cost: ¥14,500,000、Indirect Cost: ¥4,350,000)
Fiscal Year 2011: ¥2,210,000 (Direct Cost: ¥1,700,000、Indirect Cost: ¥510,000)
Fiscal Year 2010: ¥3,770,000 (Direct Cost: ¥2,900,000、Indirect Cost: ¥870,000)
Fiscal Year 2009: ¥12,870,000 (Direct Cost: ¥9,900,000、Indirect Cost: ¥2,970,000)
Keywords参照表現 / 自然言語処理 / 協調作業対話 / 視線情報 / 幾何パズル / 対話システム / 状況依存対話 / 言語生成・理解
Research Abstract

We constructed a Japanese referring expression corpus, which serves as fundamental data for research on computational model of understanding and generation of referring expressions. In our experimental setting, two dialogue participants collaboratively solve geometrical puzzles. In each dialogue, arrangement of puzzle pieces, operations on puzzle pieces and eye gaze of the participants were recorded in synchronization with dialogue speech. Referring expressions were extracted from transcribed speech texts and annotated with their referents and attributes. The constructed corpus includes 131 dialogues in total. Using this corpus, computational models for understanding and generation of referring expressions were constructed based on a machine learning framework. The models were evaluated by using the constructed corpus to show the effectiveness of the models.

Report

(4 results)
  • 2011 Annual Research Report   Final Research Report ( PDF )
  • 2010 Annual Research Report
  • 2009 Annual Research Report
  • Research Products

    (39 results)

All 2011 2010 2009

All Journal Article (2 results) (of which Peer Reviewed: 2 results) Presentation (37 results)

  • [Journal Article] REX-J : Japanese referring expression corpus of situated dialogs2010

    • Author(s)
      Phlipp Spanger, Masaaki Yasuhara, Ryu Iida, Takenobu Tokunaga, Asuka Terai, Naoko Kuriyama
    • Journal Title

      Language Resources and Evaluation

      Volume: 46 Issue: 3 Pages: 461-491

    • DOI

      10.1007/s10579-010-9134-8

    • Related Report
      2011 Final Research Report
    • Peer Reviewed
  • [Journal Article] REX-J : Japanese referring expression corpus of situated dialogs2010

    • Author(s)
      Philipp Spanger, Masaaki Yasuhara, Ryu lida, Takenobu Tokunaga, Asuka Teral, Naoko Kuriyama
    • Journal Title

      Language Resources and Evaluation (Online First)

      Volume: 10. 1007/s10579-010-9134-8

    • Related Report
      2010 Annual Research Report
    • Peer Reviewed
  • [Presentation] 参照表現ベイジアンネットワークによる参照解決の評価2011

    • Author(s)
      船越孝太郎, 中野幹生, 徳永健伸, 飯田龍
    • Organizer
      情報処理学会自然言語処理研究会
    • Place of Presentation
      石垣市
    • Year and Date
      2011-11-29
    • Related Report
      2011 Final Research Report
  • [Presentation] 参照表現ベイジアンネットワークによる参照解決の評価2011

    • Author(s)
      船越孝太郎, 中野幹生, 徳永健伸, 飯田龍
    • Organizer
      情報処理学会自然言語処理研究会
    • Place of Presentation
      石垣市商工会館
    • Year and Date
      2011-11-22
    • Related Report
      2011 Annual Research Report
  • [Presentation] Multi-modal Reference Resolution in Situated Dialogue by Integrating Linguistic and Extra-Linguistic Clues2011

    • Author(s)
      Ryu Iida, Masaaki Yasuhara, Takenobu Tokunaga
    • Organizer
      Proceedings of the 5th International Joint Conference on Natural Language Processing(IJCNLP2011)
    • Place of Presentation
      Chiang Mai
    • Year and Date
      2011-11-09
    • Related Report
      2011 Final Research Report
  • [Presentation] Multi-modal reference resolution in situated dialogue by integrating ling uistic and extralinguistic clues2011

    • Author(s)
      Ryu Iida, Masaaki Yasuhara, Takenobu Tokunaga
    • Organizer
      Proceedings of the 5th International Joint Conference on Natural Language Processing
    • Place of Presentation
      Chiang Mai, Thailand
    • Year and Date
      2011-11-09
    • Related Report
      2011 Annual Research Report
  • [Presentation] Gaze matching of referring expressions in collaborative problem solving2011

    • Author(s)
      Naoko Kuriyama, Asuka Terai, Masaaki Yasuhara, Takenobu Tokunaga, Kimihiko Yamagishi, Takashi Kusumi
    • Organizer
      Proceedings of International Workshop on Dual Eye Tracking in CSCW(DUET 2011)
    • Place of Presentation
      Aarhus
    • Year and Date
      2011-09-25
    • Related Report
      2011 Final Research Report
  • [Presentation] Using metaphors in collaborative problem solving : An eye-movement analysis2011

    • Author(s)
      Asuka Terai, Naoko Kuriyama, Masaaki Yasuhara, Takenobu Tokunaga, Kimihiko Yamagishi, Takashi Kusumi
    • Organizer
      Proceedings of International Workshop on Dual Eye Tracking in CSCW(DUET 2011)
    • Place of Presentation
      Aarhus
    • Year and Date
      2011-09-25
    • Related Report
      2011 Final Research Report
  • [Presentation] Gaze matching of referring expressions in collaborative problem solving2011

    • Author(s)
      Naoko Kuriyama, Asuka Terai, Masaaki Yasuhara, Takenobu Tokunaga, Kimihiko Yamagishi, Takashi Kusumi
    • Organizer
      Proceedings of International Workshop on Dual Eye Tracking in CSCW
    • Place of Presentation
      Arhus, Denmark
    • Year and Date
      2011-09-25
    • Related Report
      2011 Annual Research Report
  • [Presentation] Using metaphors in collaborative problem solving : An eye-movement analysis2011

    • Author(s)
      Asuka Terai, Naoko Kuriyama, Masaaki Yasuhara, Takenobu Tokunaga, Kimihiko Yamagishi, Takashi Kusumi
    • Organizer
      Proceedings of International Workshop on Dual Eye Tracking in CSCW
    • Place of Presentation
      Arhus, Denmark
    • Year and Date
      2011-09-25
    • Related Report
      2011 Annual Research Report
  • [Presentation] 共同問題解決における参照表現と視線一致率の関係-参照表現の属性に基づく分析2011

    • Author(s)
      栗山直子, 寺井あすか, 安原正晃, 徳永健伸, 山岸侯彦, 楠見孝
    • Organizer
      日本認知科学会第28回大会
    • Place of Presentation
      東京
    • Year and Date
      2011-09-23
    • Related Report
      2011 Final Research Report
  • [Presentation] 共同問題解決における参照表現と視線一致率の関係-参照表現の属性に基づく分析2011

    • Author(s)
      栗山直子, 寺井あすか, 安原正晃, 徳永健伸, 山岸侯彦, 楠見孝
    • Organizer
      日本認知科学会第28回大会
    • Place of Presentation
      東京大学
    • Year and Date
      2011-09-23
    • Related Report
      2011 Annual Research Report
  • [Presentation] 共同問題解決における比喩の利用-視線情報に基づく検討2011

    • Author(s)
      寺井あすか, 栗山直子, 安原正晃, 徳永健伸, 山岸侯彦, 楠見孝
    • Organizer
      日本心理学会第75回大会
    • Place of Presentation
      東京
    • Year and Date
      2011-09-17
    • Related Report
      2011 Final Research Report
  • [Presentation] 共同問題解決における比喩の利用-視線情報に基づく検討2011

    • Author(s)
      寺井あすか, 栗山直子, 安原正晃, 徳永健伸, 山岸侯彦, 楠見孝
    • Organizer
      日本心理学会第75回大会
    • Place of Presentation
      日本大学
    • Year and Date
      2011-09-17
    • Related Report
      2011 Annual Research Report
  • [Presentation] コンピュータを介した共同問題解決における発話同定~2者の視線一致とマウスポインタの役割の関係~2011

    • Author(s)
      栗山直子, 寺井あすか, 徳永健伸, 山岸侯彦, 楠見孝
    • Organizer
      電子情報通信学会ヒューマンコミュニケーション基礎研究
    • Place of Presentation
      京都
    • Year and Date
      2011-08-26
    • Related Report
      2011 Final Research Report
  • [Presentation] コンピュータを介した共同問題解決における発話同定~2者の視線一致とマウスポインタの役割の関係~2011

    • Author(s)
      栗山直子, 寺井あすか, 徳永健伸, 山岸侯彦, 楠見孝
    • Organizer
      電子情報通信学会ヒューマンコミュニケーション基礎研究
    • Place of Presentation
      京都大学
    • Year and Date
      2011-08-26
    • Related Report
      2011 Annual Research Report
  • [Presentation] ベイジアンネットワークを用いた参照表現モデルの提案2011

    • Author(s)
      船越孝太郎, 中野幹生, 徳永健伸, 飯田龍
    • Organizer
      情報処理学会自然言語処理研究会
    • Place of Presentation
      山形
    • Year and Date
      2011-07-15
    • Related Report
      2011 Final Research Report
  • [Presentation] ベイジアンネットワークを用いた参照表現モデルの提案2011

    • Author(s)
      船越孝太郎, 中野幹生, 徳永健伸, 飯田龍
    • Organizer
      情報処理学会自然言語処理研究会
    • Place of Presentation
      山形大学
    • Year and Date
      2011-07-15
    • Related Report
      2011 Annual Research Report
  • [Presentation] 視線情報を利用した協調作業対話における参照解析2011

    • Author(s)
      安原正晃, 飯田龍, 徳永健伸
    • Organizer
      言語処理学会第17回年次大会発表論文集
    • Place of Presentation
      豊橋
    • Year and Date
      2011-03-10
    • Related Report
      2011 Final Research Report
  • [Presentation] 視線情報を利用した協調作業対話における参照解析2011

    • Author(s)
      安原正晃・飯田龍・徳永健伸
    • Organizer
      言語処理学会第17回年次大会
    • Place of Presentation
      豊橋
    • Year and Date
      2011-03-10
    • Related Report
      2010 Annual Research Report
  • [Presentation] 言語生成研究の動向2011

    • Author(s)
      徳永健伸
    • Organizer
      言語処理学会第17回年次大会
    • Place of Presentation
      豊橋
    • Year and Date
      2011-03-07
    • Related Report
      2010 Annual Research Report
  • [Presentation] 視線情報を含むマルチモーダル協調作業対話コーパスの構築と利用2010

    • Author(s)
      安原正晃, 石川真也, 飯田龍, 徳永健伸
    • Organizer
      情報処理学会自然言語処理研究会
    • Place of Presentation
      広島
    • Year and Date
      2010-11-19
    • Related Report
      2011 Final Research Report
  • [Presentation] 視線情報を含むマルチモーダル協調作業対話コーピスの構築と利用2010

    • Author(s)
      安原正晃・石川真也・飯田龍・徳永健伸
    • Organizer
      情報処理学会自然言語処理研究会
    • Place of Presentation
      広島
    • Year and Date
      2010-11-19
    • Related Report
      2010 Annual Research Report
  • [Presentation] 共同問題解決時の二者の視線一致が共有知識の形成に及ぼす効果2010

    • Author(s)
      栗山直子, 寺井あすか, 安原正晃, 徳永健伸, 山岸侯彦, 楠見孝
    • Organizer
      日本認知科学会第27回大会
    • Place of Presentation
      神戸
    • Year and Date
      2010-09-18
    • Related Report
      2011 Final Research Report 2010 Annual Research Report
  • [Presentation] Construction of bilingual multimodal corpora of referring expressions in collaborative problem solving2010

    • Author(s)
      Tokunaga Takenobu, Yasuhara Masaaki, Terai Asuka, David Morris, Anja Belz
    • Organizer
      Proceedings of the Eighth Workshop on Asian Language Resources
    • Place of Presentation
      Beijing
    • Year and Date
      2010-08-23
    • Related Report
      2011 Final Research Report
  • [Presentation] Construction of bilingual multimodal corpora of referring expressions in collaborative problem solving2010

    • Author(s)
      Tokunage Takenobu, Yasuhara Masaaki, Terai Asuka, David Morris, Anja Belz
    • Organizer
      The 8^<th> Workshop on Asian Language Resources
    • Place of Presentation
      北京(中国)
    • Year and Date
      2010-08-23
    • Related Report
      2010 Annual Research Report
  • [Presentation] Effect of gaze matching on collaborative problem solving2010

    • Author(s)
      Kuriyama Naoko, Terai Asuka, Yasuhara Masaaki, Tokunaga Takenobu, Yamagishi Kimihiko and Kusumi Takashi
    • Organizer
      Proceedings of 7th International Conference on Cognitive Science
    • Place of Presentation
      Beijing
    • Year and Date
      2010-08-18
    • Related Report
      2011 Final Research Report
  • [Presentation] Effect of gaze matching on collaborative problem solving.2010

    • Author(s)
      Kuriyama Naoko, Terai Asuka, Yasuhara Masaaki, Tokunage Takenobu, Yamagishi Kimihiko, Kusumi Takashi
    • Organizer
      7th International Conference on Cognitive Science
    • Place of Presentation
      北京(中国)
    • Year and Date
      2010-08-18
    • Related Report
      2010 Annual Research Report
  • [Presentation] Incorporating Extra-linguistic Information into Reference Resolution in Collaborative Task Dialogue2010

    • Author(s)
      Ryu Iida, Shumpei Kobayashi, Takenobu Tokunaga
    • Organizer
      Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics
    • Place of Presentation
      Uppsala
    • Year and Date
      2010-07-14
    • Related Report
      2011 Final Research Report
  • [Presentation] Incorporating extra-linguistic information into reference resolution in collaborative task dialogue2010

    • Author(s)
      Ryu lida, Shumpei Kobayashi, Takenobu Tokunaga
    • Organizer
      The 48th Annual Meeting of the Association for Computational Linguistics
    • Place of Presentation
      Uppsala (Sweeden)
    • Year and Date
      2010-07-14
    • Related Report
      2010 Annual Research Report
  • [Presentation] Towards an Extrinsic Evaluation of Referring Expressions in Situated Dialogs2010

    • Author(s)
      Philipp Spanger, Iida Ryu, Tokunaga Takenobu, Terai Asuka, Kuriyama Naoko
    • Organizer
      Proceedings of the Sixth International Natural Language Generation Conference(INLG 2010)
    • Place of Presentation
      Dublin
    • Year and Date
      2010-07-09
    • Related Report
      2011 Final Research Report
  • [Presentation] Towards an extrinsic evaluation of referring expressions in situated dialogs2010

    • Author(s)
      Philipp Spanger, lida Ryu, Tokunaga Takenobu, Terai Asuka, Kuriyama Naoko
    • Organizer
      The 6th International Natural Language Generation Conference
    • Place of Presentation
      Trim (Ireland)
    • Year and Date
      2010-07-09
    • Related Report
      2010 Annual Research Report
  • [Presentation] 非言語情報を用いた協調作業対話における照応解析2010

    • Author(s)
      小林俊平, 飯田龍, 徳永健伸, 船越孝太郎, 中野幹生
    • Organizer
      第16回言語処理学会年次大会
    • Place of Presentation
      東京
    • Year and Date
      2010-03-11
    • Related Report
      2011 Final Research Report 2009 Annual Research Report
  • [Presentation] 共同問題解決における視線の一致が解決過程に及ぼす影響2009

    • Author(s)
      栗山直子, 寺井あすか, 安原正晃, 徳永健伸, 山岸侯彦, 楠見孝
    • Organizer
      日本認知科学会第26回大会
    • Place of Presentation
      藤沢
    • Year and Date
      2009-09-12
    • Related Report
      2011 Final Research Report 2009 Annual Research Report
  • [Presentation] 視線情報に基づく共同問題解決における比喩の効果の実験的検討2009

    • Author(s)
      寺井あすか, 栗山直子, 安原正晃, 徳永健伸, 山岸侯彦, 楠見孝
    • Organizer
      日本心理学会第73回
    • Place of Presentation
      京都
    • Year and Date
      2009-08-27
    • Related Report
      2011 Final Research Report 2009 Annual Research Report
  • [Presentation] Using extra linguistic information for generating demonstrative pronouns in a situated collaboration task, Workshop on Production of Referring Expressions2009

    • Author(s)
      P., Spanger, Yasuhara, M., Iida, R., Tokunaga, T.
    • Organizer
      Bridging the gap between computational and empirical approaches to reference
    • Place of Presentation
      Amsterdam
    • Year and Date
      2009-07-29
    • Related Report
      2011 Final Research Report
  • [Presentation] Using extra linguistic information for generating demonstrative pronouns in a situated collaboration task2009

    • Author(s)
      P., Spanger, Yasuhara, M., Lida, R., Tokunaga, T.
    • Organizer
      Workshop on Production of Referring Expressions : Bridging the gapbetween computational and empirical approaches to reference
    • Place of Presentation
      Amsterdam
    • Year and Date
      2009-07-29
    • Related Report
      2009 Annual Research Report
  • [Presentation] 日本語参照表現コーパスの構築と分析2009

    • Author(s)
      徳永健伸, P., Spanger, 安原正晃, 飯田龍
    • Organizer
      情報処理学会自然言語処理研究会
    • Place of Presentation
      東京
    • Year and Date
      2009-05-22
    • Related Report
      2011 Final Research Report
  • [Presentation] 日本語参照表現コーパスの構築と分析2009

    • Author(s)
      徳永健伸, P., Spanger, 安原正晃, 飯田龍
    • Organizer
      情報処理学会 自然言語処理研究会
    • Place of Presentation
      東京
    • Year and Date
      2009-05-22
    • Related Report
      2009 Annual Research Report

URL: 

Published: 2009-04-01   Modified: 2016-04-21  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi