Analyzing group decision making process based on multimodal conversation modeling

2017 Fiscal Year Final Research Report

• Project/Area Number: 15K00300
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Research Field: Intelligent informatics
• Research Institution: Japan Advanced Institute of Science and Technology (2017); Tokyo Institute of Technology (2015-2016)
• Principal Investigator: Okada Shogo 北陸先端科学技術大学院大学, 先端科学技術研究科, 准教授 (00512261)
• Project Period (FY): 2015-04-01 – 2018-03-31
• Keywords: マルチモーダルインタラクション / 社会的信号処理 / 機械学習 / データマイニング / グループディスカッション

Outline of Final Research Achievements

This research project focuses on group discussion for a problem solving and develops a framework for analyzing the group decision making process based on verbal and nonverbal (multimodal) information which observed from group members.We defined the quality of group output as an index set of social science (“product dimension”), which proposed by Hackman. The annotation data of the quality of group output has been collected. The machine learning model it developed to predict the product dimension from multimodal information including dialog transcription, head motion, speech prosody and turn taking. Novel co-occurrence data mining is proposed to capture the group interaction and multimodal patterns. Through the machine learning modeling and data mining,the specific multimodal features observed in group discussion process with high/low quality can be discovered automatically. Best prediction accuracy of product dimension is 82 % in binary classification task (high or low of quality).

Free Research Field

知能情報処理