| 研究実績の概要 |
Machine translation (MT) is crucial to promote globalization. In Japan, because of the rapid increase of foreign tourists and the host of the 2020 Tokyo Olympic Games, translation needs are rapidly growing, making MT indispensable. In MT, the translation knowledge is acquired from parallel corpora (sentence-aligned bilingual texts). However, domain specific parallel corpora are usually scarce or nonexistent in most languages, and thus MT performs poorly in such scenarios. Improving MT quality in this low resource scenario is a challenging unsolved problem.
To address this problem, we aim to extract parallel data from multimodal data consisting of images and multilingual description text, which is widely available from the web and social media and studying neural MT (NMT) using the extracted parallel data. In FY 2019, we mainly studied the following towards our goal.
1. NMT with semantic image regions. We developed a multimodal NMT system with double attention that attends to both words and semantic image regions and published our work at NL 241 and YANS 2019.
2. Cross-lingual visual grounding. To identify parallel data from multimodal data, we studied a cross-lingual visual grounding model on both English and French data. This work is under submission.
|
