研究課題/領域番号 |
21K17775
|
研究種目 |
若手研究
|
配分区分 | 基金 |
審査区分 |
小区分61010:知覚情報処理関連
|
研究機関 | 国立情報学研究所 |
研究代表者 |
Wang Xin 国立情報学研究所, コンテンツ科学研究系, 特任助教 (60843141)
|
研究期間 (年度) |
2021-04-01 – 2024-03-31
|
研究課題ステータス |
交付 (2022年度)
|
配分額 *注記 |
4,550千円 (直接経費: 3,500千円、間接経費: 1,050千円)
2023年度: 1,560千円 (直接経費: 1,200千円、間接経費: 360千円)
2022年度: 1,560千円 (直接経費: 1,200千円、間接経費: 360千円)
2021年度: 1,430千円 (直接経費: 1,100千円、間接経費: 330千円)
|
キーワード | speech privacy / speaker anonymization / speech waveform modeling / neural network / deep learning |
研究開始時の研究の概要 |
Human speech contains not only verbal contents but also private information about the speaker such as the speaker identity. This proposal is on protecting the speaker’s privacy in speech data for two scenarios: 1) Speech anonymization: when the speaker shares the speech data in untrusted cyberspace, this speech data should be protected so that the audience can understand the speech but cannot infer who the speaker is; 2) Speech de-anonymization: when the speaker further shares the speech data to trusted audience, the original natural speech can be reconstructed from protected version.
|
研究実績の概要 |
The second year's work consists of three parts: Part 1) Based on the previous year's work, the second VoicePrivacy challenge was organized by us and other universities. We defined new evaluation frameworks and conducted solid evaluations. In addition to many findings, we found that the new baseline, which was the research outcome of the previous year, outperformed the legacy baseline. We also saw submissions that outperformed the new baseline, which indicates the advancement of the research field brought by the VoicePrivacy challenge.
Part 2) Based on the framework of the voice privacy challenge, we did a deep analysis of the common approaches to generating anonymized speaker identity representation (i.e., pseudo speaker embedding). Through a large-scale experiment, we identified good strategies to choose and assign the pseudo-speaker, including random gender selection and utterance-level anonymization. We also found that a simple percentile-based pitch conversion reduced the risk against the strongest (Semi-Informed) attacker. These findings were published in a top IEEE journal.
Part 3) We followed the research plan and extended the language-independent speaker anonymization framework. Although the framework is language-independent, its performance degrades when processing unseen languages. We found that using multilingual training data for the waveform generator was helpful. We also proposed a correlation-alignment-based strategy to alleviate channel mismatch. Additionally, we extended the framework to hide gender information. Both works were published in top conferences.
|
現在までの達成度 (区分) |
現在までの達成度 (区分)
2: おおむね順調に進展している
理由
The efforts of the VoicePrivacy Challenge 2022 produced good outcomes. The challenge attracted 43 registered teams from 17 countries, which led to 16 successful submissions. We also organized a special session in the Interspeech 2022 satellite workshop and had presentations from participants and ourselves. The results are released on VoicePrivacy Challenge's official website: https://www.voiceprivacychallenge.org/results-2022/.
The experimental study analyzing the shortcomings and optimal strategy for speaker anonymization under (Part 2 of the research outcome) was published in a top IEEE journal.
We followed the research plan and investigated the language-independent speaker anonymization framework (Part 3 of the research outcome), and the work was accepted by the Interspeech 2022 conference (CORE rank A) and ICASSP 2023 conference (CORE rank B).
|
今後の研究の推進方策 |
Following the research plan made in the previous year, we will work on the language-independent speaker anonymization framework. Although it performs well in different languages (research outcome of 1st year) and other speaker attributes (Part 3 of the research outcome), there are issues left: 1) The quality of the anonymized voice is still inferior to the natural voice. Findings from the research outcome (Part 2) indicate that the selection-based generate pseudo speaker embedding is one bottleneck. We plan to investigate generative approaches for better performance. 2) The optimization of the speaker anonymization framework lacks a solid mathematical description. We plan to derive a unified mathematical description to consider multiple goals of the optimization and improve the current framework accordingly.
The final year research plan also includes work on the VoicePrivacy Challenge series: 1) post-challenge analysis on VoicePrivacy Challenge 2022 and how the progress of the research field has been made since the previous challenge. 2) whether stronger attacker models can recognize the speaker identity in the anonymized speech waveforms.
|