Eyetifact: Development of a Platform for Eyewear Data Conversion and Its Application to Activity Recognition in the Wild

2019 Fiscal Year Final Research Report

• Project/Area Number: 17K12728
• Research Category: Grant-in-Aid for Young Scientists (B)
• Allocation Type: Multi-year Fund
• Research Field: Human interface and interaction
• Research Institution: Osaka Prefecture University
• Principal Investigator: Ishimaru Shoya 大阪府立大学, 研究推進機構, 客員研究員 (10788730)
• Project Period (FY): 2017-04-01 – 2020-03-31
• Keywords: 心的状態認識 / 読書行動認識 / アイトラッキング / 赤外線サーモグラフィ / 皮膚電気活動 / 眼電位 / リカレントニューラルネットワーク / 人工知能

Outline of Final Research Achievements

This project aims to develop a system that learns the relationship between data recorded by multiple sensors and converts data obtained from one sensor into the equivalent data obtained by another advanced sensor. In the first year, as basic research for sensor data conversion, we investigated the relationship between internal states and sensor signals including eye movements and nose temperatures. In the next year, we developed data collection software and evaluated its scalability. In the final year, we added EEG to the list of the measurements and estimated the level of interest while reading from eye movements. Furthermore, we proposed a method for recognizing reading behavior in the real world by using electrooculography glasses and published the method with a dataset.

Free Research Field

Human-Computer Interaction

Academic Significance and Societal Importance of the Research Achievements

本研究の成果は，安価で日常的に記録ができるアイウェアのデータを大量に記録して高機能のものに変換するという方法で，これまでボトルネックであった「いかにして大量の行動データを集めるか」という問題を解消することができる．社会的には，教育と医療への影響が期待できる．例えば，本研究成果によって認知的行動の活動計測が実用化される．万歩計を見ながら生活することで運動のモチベーションを維持できるのと同様に，1日の読書量や会話量を計測することで，勉強のやる気を維持したり，行動の変化から病気の早期発見や治療，予防に役立てることができる．