2019 Fiscal Year Research-status Report
Multilingual Socio-Emotional Communication Support for Improving Foreign Student Adjustment and Mental Health Outcomes
| Project/Area Number |
18K18085
|
|---|
| Research Institution | The University of Tokyo |
|---|
Principal Investigator |
ハウタサーリ アリ 東京大学, 大学院情報学環・学際情報学府, 特任講師 (70752236)
|
|---|
| Project Period (FY) |
2018-04-01 – 2021-03-31
|
| Keywords | socio-emotional / multilingual / communication |
|---|
| Outline of Annual Research Achievements |
Second iteration of development of a software tool for supporting foreign students’ socio-emotional communication was completed. The approach taken in the prototype development was to design for a method to display an abstraction of the level of arousal in emotional text-based messages to non-native speakers who may have difficulties detecting the emotional nuances in second language messages. Presently, the developed system uses voice-input in text chats to display sender's utterance intensity to receivers with a novel method using speech bubbles as emotional cues.
Details of the initial system design and evaluation results were published in IPSJ INTERACTION as a premium track presentation. The process of combining the new prototype with the emotional font display developed in FY2018 is under way.
Development of a mobile application to collect non-native English speakers' valence and arousal evaluations of English emotional words was initiated. This application will be used to collect data for a corpus to improve the accuracy of emotional valence detection based on users' native language in the developed socio-emotional communication support system.
|
| Current Status of Research Progress |
Current Status of Research Progress
1: Research has progressed more than it was originally planned.
Reason
The socio-emotional communication support system development is proceeding more smoothly than anticipated thanks to effective collaboration with other researchers and developers. The initial plan was to use a multilingual sentiment corpora to detect the emotional valence in text-based messages, but a bidirectional-LSTM trained with human-annotated emotional sentences was implemented first in FY2018, and the plan was to instead improve the detection accuracy with the multilingual sentiment corpora.
In the second iteration of the prototype, the aim was to develop a method for capturing message senders' level of arousal more accurately, which was found to be challenging with only human-annotated text as training data. Voice-input is used to capture and display an abstraction of this information as a speech bubble. Presently, the process of combining emotional font outputs based on verbal content of a message, and speech bubbles representing the level of arousal in one novel emotional communication support system is on-going.
In order to effectively collect evaluations of the emotionality of English language words from non-native English speakers, including in Mechanical Turk, development of a mobile application with novel interface for gathering emotional evaluations from users was initiated.
Due to the global COVID-19 pandemic a laboratory experiment planned for the end of March was postponed.
|
| Strategy for Future Research Activity |
- Conduct a multilingual communication experiment using the developed system in order to evaluate the efficacy of the proposed approach on interpersonal relationship building in a controlled laboratory setting.
- Continue the creation of the proposed non-native sentiment corpora for further system development by utilizing a mobile application to gather human evaluations, and analyze the discrepancies in emotional word evaluations between native and non-native speakers of English.
- Longitudinal user studies are planned to be conducted at two Japanese universities.
- Development of a cross-cultural training program based on the research results.
|
| Causes of Carryover |
The research results were presented in an online domestic conference, which reduced travel costs for the fiscal year. The next publications are planned to be presented in international conferences, and the travel costs will be allocated for this purpose.
Personnel expenditure was reduced due to postponement of a laboratory experiment at the end of March, and are allocated to be used for controlled laboratory experiments, as well as software development, conducted in FY2020.
|
Research Products
(2 results)
