Speech security on human-computer interaction

2023 Fiscal Year Annual Research Report

• Project/Area Number: 22K21304
• Research Institution: Japan Advanced Institute of Science and Technology
• Principal Investigator: MAWALIM CandyOlivia 北陸先端科学技術大学院大学, 先端科学技術研究科, 助教 (10963720)
• Project Period (FY): 2022-08-31 – 2024-03-31
• Keywords: speaker verification / spoof attacks / intelligibility / auditory model

Outline of Annual Research Achievements

This fiscal year, our research addressed the previously unclear target of anonymization and the diverse threats posed by speech attacks. To tackle spoofing in speaker verification systems, we proposed novel methods using linear frequency cepstral coefficients and spectro-temporal modulation representation. These findings were presented at the APSIPA 2023 and iSAI-NLP 2023 conferences. We also investigated human speech perception through an auditory periphery model. This research, published in the Journal of Applied Acoustics, provides a foundation for future studies on detecting speech and speaker identity changes caused by speech synthesis techniques. Finally, we are expanding our research scope beyond English by initiating the development of a Thai language spoof database.

Research Products

• [Journal Article] A Ranking Model for Evaluation of Conversation Partners Based on Rapport Levels (2023)
  • Author(s): Hayashi Takato、Mawalim Candy Olivia、Ishii Ryo、Morikawa Akira、Fukayama Atsushi、Nakamura Takao、Okada Shogo
  • Journal Title: IEEE Access Volume: 11 Pages: 73024～73035
  • DOI: 10.1109/ACCESS.2023.3287984
  • Peer Reviewed / Open Access
• [Journal Article] Non-intrusive speech intelligibility prediction using an auditory periphery model with hearing loss (2023)
  • Author(s): Mawalim Candy Olivia、Titalim Benita Angela、Okada Shogo、Unoki Masashi
  • Journal Title: Applied Acoustics Volume: 214 Pages: 109663～109663
  • DOI: 10.1016/j.apacoust.2023.109663
  • Peer Reviewed / Open Access
• [Presentation] Analysis of Spectro-Temporal Modulation Representation for Deep-Fake Speech Detection (2023)
  • Author(s): Haowei Cheng, Candy Olivia Mawalim, Kai Li, Lijun Wang, and Masashi Unoki
  • Organizer: 2023 Asia-Pasific Signal and Information Processing Association Annual Summit and Conference
  • Int'l Joint Research
• [Presentation] Auditory Model Optimization with Wavegram-CNN and Acoustic Parameter Models for Nonintrusive Speech Intelligibility Prediction in Hearing Aids (2023)
  • Author(s): Candy Olivia Mawalim, Benita Angela Titalim, Shogo Okada, and Masashi Unoki
  • Organizer: The 31st European Signal Processing Conference (EUSIPCO 2023)
  • Int'l Joint Research
• [Presentation] Speech signal processing for privacy and security protection (2023)
  • Author(s): Candy Olivia Mawalim
  • Organizer: APSIPA Workshop on Signal and Information Processing in Indonesia
  • Invited
• [Presentation] Incorporating the Digit Triplet Test in A Lightweight Speech Intelligibility Prediction for Hearing Aids (2023)
  • Author(s): Xiajie Zhou, Candy Olivia Mawalim, and Masashi Unoki
  • Organizer: 2023 Asia-Pasific Signal and Information Processing Association Annual Summit and Conference
  • Int'l Joint Research
• [Presentation] AI for Speech Processing: “Threats and Opportunities of Voice-Interactive Applications” (2023)
  • Author(s): Candy Olivia Mawalim
  • Organizer: AI Talks ITB #10 (Online Webinar)
  • Invited
• [Presentation] Inter-person Intra-modality Attention Based Model for Dyadic Interaction Engagement Prediction (2023)
  • Author(s): Xiguang Li, Shogo Okada, and Candy Olivia Mawalim
  • Organizer: The 25th HCI International Conference
  • Int'l Joint Research
• [Presentation] Investigating the Effect of Linguistic Features on Personality and Job Performance Predictions (2023)
  • Author(s): Hung Le, Sixia Li, Candy Olivia Mawalim, Hung-Hsuan Huang, Chee Wee Leong, and Shogo Okada
  • Organizer: The 25th HCI International Conference
  • Int'l Joint Research
• [Presentation] ThaiSpoof: A Database for Spoof Detection in Thai Language (2023)
  • Author(s): Kasorn Galajit, Thunpisit Kosolsriwiwat, Candy Olivia Mawalim, Pakinee Aimmanee, Waree Kongprawechnon, Win Pa Pa, Anuwat Chaiwongyen, Teeradaj Racharak, Hayati Yassin, Jessada Karnjana, Surasak Boonkla, and Masashi Unoki
  • Organizer: The 18th International Joint Symposium on Artificial Intelligence and Natural Language Processing and The International Conference on Artificial Intelligence and Internet of Things (iSAI-NLP 2023)
  • Int'l Joint Research
• [Presentation] Voice Contribution on LFCC feature and ResNet-34 for Spoof Detection (2023)
  • Author(s): Khaing Zar Mon, Kasorn Galajit, Candy Olivia Mawalim, Jessada Karnjana, Tsuyoshi Isshiki, and Pakinee Aimmanee
  • Organizer: The 18th International Joint Symposium on Artificial Intelligence and Natural Language Processing and The International Conference on Artificial Intelligence and Internet of Things (iSAI-NLP 2023)
  • Int'l Joint Research