Learning Robust Word Representations for Natural Language Processing

• Project/Area Number: 24800041
• Research Category: Grant-in-Aid for Research Activity Start-up
• Allocation Type: Single-year Grants
• Research Field: Intelligent informatics
• Research Institution: Nara Institute of Science and Technology
• Principal Investigator: DUH Kevin 奈良先端科学技術大学院大学, 情報科学研究科, 助教 (80637322)
• Project Period (FY): 2012-08-31 – 2014-03-31
• Project Status: Completed (Fiscal Year 2013)
• Budget Amount: ¥2,730,000 (Direct Cost: ¥2,100,000、Indirect Cost: ¥630,000); Fiscal Year 2013: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000); Fiscal Year 2012: ¥1,560,000 (Direct Cost: ¥1,200,000、Indirect Cost: ¥360,000)
• Keywords: 自然言語処理 / 機械学習

Research Abstract

Language is a highly productive phenomenon; new words and expressions are constantly being invented. Current Natural Language Processing techniques have difficulty handling new words, so their performance on real-world text suffers. To address this, we develop robust models of word semantics, focusing on Deep Learning methods for learning vector word representations. Furthermore, we show improvements in parsing and translation performance using systems that incorporate these word representations.

Research Products

• [Presentation] Deep Learningの基礎と言語処理への応用 （チュートリアル） (2014)
  • Author(s): Kevin Duh
  • Organizer: 言語処理学会第２０回年次大会
  • Place of Presentation: 北海道大学
  • Invited
• [Presentation] Modeling and Learning Semantic Co-Compositionality through Prototype Projection and Neural Networks (2013)
  • Author(s): M Tsubaki, K Duh, M Shimbo, and Y Matsumoto
  • Organizer: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP)
  • Place of Presentation: Seattle, USA.
  • Year and Date: 2013-10-18
• [Presentation] Adaptation Data Selection using Neural Language Models : Experiments in Machine Translation (2013)
  • Author(s): K Duh, G Neubig, K Sudoh, and H Tsukada
  • Organizer: Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)
  • Place of Presentation: Sofia, Bulgaria
  • Year and Date: 2013-08-06
• [Presentation] An Empirical Investigation of Word Representations for Parsing the Web (2013)
  • Author(s): S Hisamoto, K Duh, and Y Matsumoto
  • Organizer: 言語処理学会第19回年次大会発表論文集
  • Place of Presentation: 名古屋大学
  • Year and Date: 2013-03-14
• [Presentation] Adaptation Data Selection using Neural Language Models: Experiments in Machine Translation (2013)
  • Author(s): Kevin Duh, Graham Neubig, Katsuhito Sudoh, Hajime Tsukada
  • Organizer: The 51th Annual Meeting of the Association for Computational Linguistics
  • Place of Presentation: National Palace of Culture, Sofia, Bulgaria
• [Presentation] Modeling and Learning Semantic Co-Compositionality through Prototype Projections and Neural Networks (2013)
  • Author(s): Masashi Tsubaki, Kevin Duh, Masashi Shimbo, Yuji Matsumoto
  • Organizer: The 2013 Conference on Empirical Methods in Natural Language Processing
  • Place of Presentation: Grand Hyatt, Seattle, USA
• [Presentation] An Empirical Investigation of Word Representations for Parsing the Web
  • Author(s): Sorami Hisamoto, Kevin Duh, Yuji Matsumoto
  • Organizer: 言語処理学会第19回年次大会
  • Place of Presentation: 名古屋大学