Language-independent, multi-modal, and data-efficient approaches for speech synthesis and translation

• Project/Area Number: 21K11951
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Review Section: Basic Section 61010:Perceptual information processing-related
• Research Institution: National Institute of Informatics
• Principal Investigator: Cooper Erica 国立情報学研究所, コンテンツ科学研究系, 特任助教 (30843156)
• Co-Investigator(Kenkyū-buntansha): Kruengkrai Canasai 国立情報学研究所, コンテンツ科学研究系, 特任助教 (10895907)
• Project Period (FY): 2021-04-01 – 2024-03-31
• Project Status: Granted (Fiscal Year 2021)
• Budget Amount: ¥4,160,000 (Direct Cost: ¥3,200,000、Indirect Cost: ¥960,000); Fiscal Year 2023: ¥1,300,000 (Direct Cost: ¥1,000,000、Indirect Cost: ¥300,000); Fiscal Year 2022: ¥1,430,000 (Direct Cost: ¥1,100,000、Indirect Cost: ¥330,000); Fiscal Year 2021: ¥1,430,000 (Direct Cost: ¥1,100,000、Indirect Cost: ¥330,000)
• Keywords: speech synthesis / multi-lingual / machine translation / deep learning / neural networks

Outline of Research at the Start

Language technology has improved due to advances in neural-network-based approaches; for example, speech synthesis has reached the quality of human speech. However, neural models require large quantities of data. Speech technologies bring social benefits of accessibility and communication - to ensure broad access to these benefits, we consider language-independent methods that can make use of less data. We propose 1) articulatory class based end-to-end speech synthesis; 2) multi-modal machine translation with text and speech; and 3) neural architecture search for data-efficient architectures.