2023 Fiscal Year Final Research Report
Multi-lingual multi-speaker voice conversion system by non-parallel learning method
| Project/Area Number |
20H04207
|
|---|
| Research Category |
Grant-in-Aid for Scientific Research (B)
|
| Allocation Type | Single-year Grants |
|---|
| Section | 一般 |
|---|
| Review Section |
Basic Section 61010:Perceptual information processing-related
|
|---|
| Research Institution | Japan Advanced Institute of Science and Technology |
|---|
Principal Investigator |
Akagi Masato 北陸先端科学技術大学院大学, 先端科学技術研究科, 名誉教授 (20242571)
|
|---|
| Co-Investigator(Kenkyū-buntansha) |
鵜木 祐史 北陸先端科学技術大学院大学, 先端科学技術研究科, 教授 (00343187)
|
|---|
| Project Period (FY) |
2020-04-01 – 2024-03-31
|
| Keywords | パラ言語情報 / 非言語情報 / 音声変換 / 非並行型学習 |
|---|
| Outline of Final Research Achievements |
This study aims to enhance paralinguistic and non-linguistic information in multilingual speech through Voice Conversion (VC), with the manipulation of speaker identity in speech as one of its central objectives. To achieve this, we propose a non-parallel learning method for cross-lingual VC and explore the construction of a multi-speaker attribute conversion system based on this learning approach. Specifically, the issues addressed include (A) handling speaker information when the source and target languages of VC are different, (B) achieving multi-speaker-to-multi-speaker attribute conversion, (C) describing speaker characteristics when considering the use of unseen speakers, and (D) ensuring the quality and intelligibility of synthesized speech after conversion. By addressing these challenges within the framework of deep learning and optimizing the entire process through appropriate objective functions, we attempt to achieve comprehensive optimization.
|
| Free Research Field |
音声情報処理
|
| Academic Significance and Societal Importance of the Research Achievements |
話者のパラ言語および非言語情報を抽出し合成音声に付加することができる音声-音声翻訳のための多言語間音声変換システムを開発するために，その第一歩として，非言語情報の一つである話者属性（性別，年齢，声質等）の自由な変換操作を目指して，多言語間での音声変換のための非並行型学習法を提案し，これにもとづいた変換システムを検討する。これにより，ある言語で話をした話者の声と同じ声質で別の言語の音声を合成できる，しかも使用言語および使用話者を選ばないシステムの構築が可能となり，入力音声に含まれる話者属性を出力音声でも維持できることで，コミュニケーションの質を向上させることができる。
|
