Deep Learning the Human Mind - Recognising and Augmenting Cognitive Performance Fluctuations
| Project/Area Number |
18H03278
|
|---|
| Research Category |
Grant-in-Aid for Scientific Research (B)
|
| Allocation Type | Single-year Grants |
|---|
| Section | 一般 |
|---|
| Review Section |
Basic Section 61020:Human interface and interaction-related
|
|---|
| Research Institution | Keio University |
|---|
Principal Investigator |
クンツェ カイ 慶應義塾大学, メディアデザイン研究科(日吉), 教授 (00648040)
|
|---|
| Project Period (FY) |
2018-04-01 – 2022-03-31
|
| Project Status |
Granted (Fiscal Year 2020)
|
| Budget Amount *help |
¥16,640,000 (Direct Cost: ¥12,800,000、Indirect Cost: ¥3,840,000)
Fiscal Year 2020: ¥4,030,000 (Direct Cost: ¥3,100,000、Indirect Cost: ¥930,000)
Fiscal Year 2019: ¥4,290,000 (Direct Cost: ¥3,300,000、Indirect Cost: ¥990,000)
Fiscal Year 2018: ¥4,940,000 (Direct Cost: ¥3,800,000、Indirect Cost: ¥1,140,000)
|
|---|
| Keywords | HCI / Behavior Analysis / Cognitive Recognition / ヒューマンインターフェース / ヒューマンインタフェース / ウェアラブル機器 / ヒューマンインターフェイス |
|---|
| Outline of Annual Research Achievements |
We utilize the Open Eyewear Platform system and modified it for the project(adding more robust thermal sensors and especially modifying software including psychology cognitive performance tests.
We setup the deep learning server architecture and already preformed test recordings from the eyewear to estimate cognitive fluctuations.
Additionally, we programmed a Virtual Reality environment for the lab focused tests and start running several studies related to sleep and fatigue using this experimental equipment. We opted for Virtual Reality as it is easier to control than real life (in the wild).We programmed a Virtual Reality environment for the lab focused tests and start running several studies related to sleep and fatigue.
An experimental Setup related to alertness and cognitive load and engagement is designed and we proceed with the conduction of the study (3 participants already recorded, n should be bigger than 20).We also evaluate EMG and eye gaze input in virtual reality to record subtle interactions of the user (for estimating their cognitive performance).
|
| Current Status of Research Progress |
Current Status of Research Progress
1: Research has progressed more than it was originally planned.
Reason
The the technology assessment and hardware performed better and with less problems then expected (integration of the sensing and machine learning approaches was easier than expected so we could already proceed to data recordings and experiments).
Additionally, the initial results for which sensors to use are also very positive. The Thermal sensors show a clear signal for cognitive load and engagement for the baseline psychology tests. We are a bit surprised and will investigate (as we did expect much more noise and trouble with this assessment). Of course, if we plan to use them in the wild the picture will be different.
|
| Strategy for Future Research Activity |
In the next steps, we continue with the two setups to evaluate cognitive fluctuations in VR and in the wild (real life). The controlled setup for VR will start to be evaluated soon and we want to see how useful the deep learning approach is for the type of tasks we described in the proposal.
We start with the evaluation for alterness and engagement in the wild (real life) and look into task depended performance in the VR case.
The experiment will record eye movements, facial temperature (using our own build sensors) and estimate alterness, fatigue using deep learning architecture. We will us the ground truth of alterness (Psychomotor Vigilance Test).
|
Report
(1 results)
Research Products
(3 results)
