Neural Machine Translation Based on Bilingual Resources Extracted from Multimodal Data
| Project/Area Number |
19K20343
|
|---|
| Research Category |
Grant-in-Aid for Early-Career Scientists
|
| Allocation Type | Multi-year Fund |
|---|
| Review Section |
Basic Section 61030:Intelligent informatics-related
|
|---|
| Research Institution | Kyoto University (2020-2021)
Osaka University (2019) |
|---|
Principal Investigator |
Chu Chenhui 京都大学, 情報学研究科, 特定准教授 (70784891)
|
|---|
| Project Period (FY) |
2019-04-01 – 2022-03-31
|
| Project Status |
Completed (Fiscal Year 2021)
|
| Budget Amount *help |
¥4,160,000 (Direct Cost: ¥3,200,000、Indirect Cost: ¥960,000)
Fiscal Year 2021: ¥910,000 (Direct Cost: ¥700,000、Indirect Cost: ¥210,000)
Fiscal Year 2020: ¥910,000 (Direct Cost: ¥700,000、Indirect Cost: ¥210,000)
Fiscal Year 2019: ¥2,340,000 (Direct Cost: ¥1,800,000、Indirect Cost: ¥540,000)
|
|---|
| Keywords | 機械翻訳 / マルチモーダル / ニューラル機械翻訳 / マルチモーダルデータ / 対訳資源 |
|---|
| Outline of Research at the Start |
In machine translation (MT), the translation knowledge is acquired from parallel corpora (sentence-aligned bilingual texts). However, domain specific parallel corpora are usually scarce or nonexistent in most languages, and thus MT performs poorly in such scenarios. We aim to address this problem based on the state-of-the-art neural MT. Our core idea is extracting parallel data from multimodal data consisting of images and multilingual describing text, which is widely available from the web and social media and studying NMT using the extracted parallel data.
|
| Outline of Final Research Achievements |
In this project, we mainly studied the following topics for multimodal neural machine translation (NNMT). 1). MNMT with comparable sentences. We constructed an MNMT with comparable sentences dataset and organized a shared task in the 8th Workshop on Asian Translation (WAT 2021). Our system achieved the best performance in this shared task. 2). MNMT with semantic image regions and word-region alignment. We studied MNMT with semantic image regions and word-region alignment and published them in two famous international journals Neurocomputing and TASLP. 3). Video-guided MT (VMT). We proposed VMT with a spatial hierarchical attention network, which can address both verb and noun sense disambiguation.
|
| Academic Significance and Societal Importance of the Research Achievements |
機械翻訳における自然言語の意味曖昧性解消を目的として、マルチモーダルニューラル機械翻訳(MNMT)が主に研究されている。本プロジェクトでは、低資源な設定でコンパラブル文を用いたMNMTという新しい仕組みを考案し、画像を用いたMNMTにおいてはセマンティック画像領域と単語領域アライメントを用いたMNMTを提案し、映像を用いたMNMTにおいては空間階層注意ネットワークを提案し、機械翻訳における視覚情報の利用の有効性を示した。開発したMNMTシステムは映画、ドラマ、アニメやニュースなどの字幕の自動翻訳の精度向上に貢献できるし、大阪万博などの国際的イベントでの自動翻訳ニューズにも応えられる。
|
Report
(4 results)
Research Products
(40 results)
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
[Presentation] Overview of the 8th Workshop on Asian Translation2021
Author(s)
Toshiaki Nakazawa, Hideki Nakayama, Chenchen Ding, Raj Dabre, Shohei Higashiyama, Hideya Mino, Isao Goto, Win Pa Pa, Anoop Kunchukuttan, Shantipriya Parida, Ondrej Bojar, Chenhui Chu, Akiko Eriguchi, Kaori Abe, Yusuke Oda, Sadao Kurohashi
Organizer
In Proceedings of the 8th Workshop on Asian Translation (WAT2021)
Related Report
Int'l Joint Research
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
