Research and development of a system for evaluating emotion with facial skin temperature

2022 Fiscal Year Final Research Report

• Project/Area Number: 20K12020
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Review Section: Basic Section 61060:Kansei informatics-related
• Research Institution: The University of Electro-Communications
• Principal Investigator: Mizuno Tota 電気通信大学, 大学院情報理工学研究科, 准教授 (00337875)
• Project Period (FY): 2020-04-01 – 2023-03-31
• Keywords: 赤外線サーモグラフィ / 実画像 / 顔面皮膚温度 / 血流

Outline of Final Research Achievements

We have been developing research on a system for evaluating emotion using facial skin temperature. Conventional estimation using thermal images of the face is based on the temperature difference between the nasal area, which is a sympathetic index, and the forehead area, which is less affected by autonomic nervous system activity. However, the conventional method requires a far-infrared camera, which is not easy to use and costly. Therefore, we investigated a method to estimate autonomic nerve activity from real images. As a result, we developed a method that enables feature extraction of blood flow through capillaries on the subcutaneous surface even from real images. Using this method, we examined whether it is possible to evaluate autonomic nervous system activity during computational problems and reading, and found that blood flow changes when emotions change.

Free Research Field

HCI，生体情報工学

Academic Significance and Societal Importance of the Research Achievements

近年，職場における勤務者の疲労度や集中度を計測し，適切な休息を取り入れて職場の環境向上に努めるなど，人間がより快適な生活を送るために感情推定の必要性が高まっている．疲労度や集中度を計測する上では，これまでに，心拍，脈波，脳波，眼球運動，顔面熱画像といった生体情報を取得して，感情推定を行う手法が各種検討されてきた．しかし，これら生体情報を取得する際に，特殊な装置が必要であることが多いため，評価が可能であってもなかなか普及しない．そこで，我々は，既に普及しているUSBカメラやスマートデバイス等にすでに搭載されているカメラを用いて，感情を推定可能なシステムの構築を行った．