2019 Fiscal Year Annual Research Report
Deep Learning the Human Mind - Recognising and Augmenting Cognitive Performance Fluctuations
| Project/Area Number |
18H03278
|
|---|
| Research Institution | Keio University |
|---|
Principal Investigator |
クンツェ カイ 慶應義塾大学, メディアデザイン研究科(日吉), 教授 (00648040)
|
|---|
| Project Period (FY) |
2018-04-01 – 2022-03-31
|
| Keywords | hci / cognitive recognition / behavior modeling |
|---|
| Outline of Annual Research Achievements |
This proposal aims at modelling and augmenting performance fluctuations in the human mind in real life scenarios. Towards this goal we were able to record and analyze several large scale data recordings (publications in preparation) at UbiComp international conference, a premier venue for human computer interaction. We gathered data and ground truth questionnaires from over 76 participants in over 20 sessions. In this recording we are especially focusing on interest and attention.
Additionally, we recorded the performers and audience in several dance performances to establish theories about synchrony in behavior related to stress and attention (7 performers, around 20 visitors for 3 perfromances). In this recording we are interested in how performers and visitors interact (as well as attention related cues). In terms of controlled experiments we were looking into dyadic conversation and established a method to detect synchrony in the lab using smart eyewear (published at ISWC). We also established a larger dataset for analyzing thermal fluctuation in the users face related to cognitive load changes (publication in preparation). We also worked on a new paradigm for computer human integration with a group of international researchers, in which Deep Learning the Human Mind is an important concept. This lead to a Dagstuhl Seminar on the topic of Cognitive Augmentation. The first directions are published in CHI, the premier HCI conference.
We also organized a workshop collocated with Ubicomp 2010.
|
| Current Status of Research Progress |
Current Status of Research Progress
1: Research has progressed more than it was originally planned.
Reason
We are on schedule with our research works. We have several real life datasets in a theater, education and conference setting recorded as well as their groundtruth.
In addition we could establish a working group with international researchers leading also to a submission and acceptance of a Dagstuhl seminar on Cognitive Augmentation https://www.dagstuhl.de/20342 strongly related to this project. Our paradigm shift for HCI is also described in a CHI publication (premier HCI conference acceptance under 20 % ).
Overall, the project is on track and could also establish synergies that will hopefully influence the field in itself (human computer interaction).
|
| Strategy for Future Research Activity |
We have now several larger scale, in-the-wild datasets recorded about cognitive fluctuations dealing with learning (course work), theater performance and conference attendance. We already prepared an initial data analysis (see publications as well as papers under review). In this year we focus on analysis, we will now train and evaluate machine learning models (especially deep learning models) using those datasets.
Additionally, we will test these models in realistic scenarios (following the large scale dataset recordings from last year).We are in preparation to make the datasets publicly available and continue to present the results in high level venues.
|
