A study on an implicit health-monitoring technique through the dialog with a speech assistant

2019 Fiscal Year Final Research Report

• Project/Area Number: 18K18136
• Research Category: Grant-in-Aid for Early-Career Scientists
• Allocation Type: Multi-year Fund
• Review Section: Basic Section 61060:Kansei informatics-related
• Research Institution: Tohoku University
• Principal Investigator: CHIBA Yuya 東北大学, 工学研究科, 助教 (30780936)
• Project Period (FY): 2018-04-01 – 2020-03-31
• Keywords: ユーザ状態推定 / 音声感情認識 / 音声対話システム

Outline of Final Research Achievements

In this study, we examined estimation methods of user's emotion by using acoustic non-verbal signals at first. The experimental results showed that data augmentation based on emotional speech synthesis significantly improves recognition accuracy comparing with the conventional method. Then, we proposed an approach based on multi-stream attention-based BLSTM with segmental features, and further improved the recognition performance. The proposed method obtained 73.4% of recognition accuracy, which is comparable to human evaluation (75.5 %). In addition, we examined a dialog management method prompting to use continuously. We confirmed that the difference of dialog strategies between groups of different relationships by the analysis of verbal/non-verbal information using a multimodal chat-talk corpus of human-human conversation.

Free Research Field

音声対話システム

Academic Significance and Societal Importance of the Research Achievements

当初の目的である健康状態の推定には至らなかったものの，雑音重畳や統計的音声合成によりデータ増強を行うことでユーザ状態推定の頑健性を向上できることを示した．この手法は音声感情認識だけでなく，様々なユーザ状態推定に応用可能であるため，音声を用いたアプリケーションの多くで有用である．また，識別器の改善により音声感情認識自体も人間による判断に匹敵する性能が得られることを示した．加えて，対話型アプリケーションが継続的に使われるための言語/非言語的ふるまいの変化に関して，包括的な対話制御モデルを構築するための手がかりとなる結果を得た．