2022 Fiscal Year Research-status Report
Language-independent speaker anonymization with multiple privacy-related attributes
| Project/Area Number |
22K21319
|
|---|
| Research Institution | National Institute of Informatics |
|---|
Principal Investigator |
Miao Xiaoxiao 国立情報学研究所, コンテンツ科学研究系, 特任研究員 (10962508)
|
|---|
| Project Period (FY) |
2022-08-31 – 2024-03-31
|
| Keywords | speaker anonymization / language independent / gender netural |
|---|
| Outline of Annual Research Achievements |
The first year's work on speaker anonymization includes three parts:
Part 1) Following our plan, we propose modifying the speaker embedding and pitch to conceal the speaker's gender. This approach has been accepted by ICASSP 2023. Our code and audio samples is available. (see https://github.com/ nii- yamagishilab/speaker_sex_attribute_privacy)
Part 2) I contributed to the VoicePrivacy 2022 challenge and built a self-supervised learning (SSL) -based speaker anonymization system (SAS). It uses an SSL-based content encoder to extract general context representations regardless of the language of the input speech. This model is released for free (see https://www.voiceprivacychallenge.org).
Part 3) We made changes to the SSL-based SAS program mentioned earlier, with the goal of making it easier for users to operate. We added several new features, including the ability to adjust pitch and select speakers to anonymize in a flexible way. This updated program was then applied to Japanese speech data. A broadcasting company in Japan has expressed interest in using this model for a real TV program.
|
| Current Status of Research Progress |
Current Status of Research Progress
1: Research has progressed more than it was originally planned.
Reason
1) As planned for the first year, we modified speaker embedding and pitch to conceal the speaker's gender while preserving speech content. We tested it for gender recognition and downstream tasks and conducted listening tests. The related work has been accepted by ICASSP 2023.
2) We contributed to the VoicePrivacy 2022 challenge by developing a self-supervised learning (SSL) based speaker anonymization system (SAS).
3) The above SAS has been proven to be effective for Japanese speaker anonymization after updating several new features. There is a possibility that it may be used by a broadcasting company in the future.
|
| Strategy for Future Research Activity |
The original research plans were: 2nd year: anonymization of dialect, age, or other speaker-related information; 2) 3rd year: subjective and objective evaluations of our proposed approach to confirm that different privacy-related attributes can be successfully protected.
After examining the results of the first year, we looked into protecting speaker privacy by considering gender as one of the attributes. For the second year, we will extended our latestet proposed system, called language-independent speaker anonymization approach to modify other speaker privacy-related attributes such as dialect and age. The advantage of this new framework is that it does not require language-dependent components like ASR, and can be easily used for unseen language speaker anonymization. Subsequently, we will execute the plan for the third year, which involves conducting both subjective and objective evaluations to conduct an in-depth analysis.
|
| Causes of Carryover |
We would like to use the grant to conduct listening tests to analyze proposed approaches, refine our papers, and participate in international conferences where we can present our work. Presenting our work to receive feedback and staying in touch with the latest developments at the top speech conferences and workshops (SLT, ICASSP, INTERSPEECH, ASRU) are very helpful to improve our research. In order to achieve this, it is necessary to have professional paid proofreading to obtain high-quality papers in terms of English writing. And also listening tests are required for the proposed research to draw reasonable conclusions in terms of the subjective perceptual listening evaluation.
|
