Development of Facial Expression Recognition Technology in Immersive Virtual Environments
Project/Area Number |
16H05870
|
Research Category |
Grant-in-Aid for Young Scientists (A)
|
Allocation Type | Single-year Grants |
Research Field |
Human interface and interaction
|
Research Institution | Keio University |
Principal Investigator |
Maki Sugimoto 慶應義塾大学, 理工学部(矢上), 教授 (50517399)
|
Project Period (FY) |
2016-04-01 – 2020-03-31
|
Project Status |
Completed (Fiscal Year 2021)
|
Budget Amount *help |
¥19,890,000 (Direct Cost: ¥15,300,000、Indirect Cost: ¥4,590,000)
Fiscal Year 2019: ¥4,420,000 (Direct Cost: ¥3,400,000、Indirect Cost: ¥1,020,000)
Fiscal Year 2018: ¥5,330,000 (Direct Cost: ¥4,100,000、Indirect Cost: ¥1,230,000)
Fiscal Year 2017: ¥4,680,000 (Direct Cost: ¥3,600,000、Indirect Cost: ¥1,080,000)
Fiscal Year 2016: ¥5,460,000 (Direct Cost: ¥4,200,000、Indirect Cost: ¥1,260,000)
|
Keywords | 表情認識 / ユビキタス光センシング / HMD / 表情計測 / 反射型光センサ / 機械学習 / 表情識別 / 頭部装着型ディスプレイ / 幾何形状推定 / バーチャルリアリティ / 光センシング / ユーザインターフェース |
Outline of Final Research Achievements |
The aim of this project was to develop facial expression recognition technology using an embedded optical sensor array and machine learning, which enables facial expression recognition in a head-mounted display (HMD). Facial expression recognition in an HMD with an ordinal camera system is challenging due to occlusions. We developed a method for detecting basic facial expressions and extended it to estimate the intensity of each facial expression. We have shown that synthesized expressions can be rendered on avatars in a virtual environment. In addition to identifying the basic expression classes, we confirmed that it is possible to reconstruct high-dimensional 3D positions of feature points on a facial surface from the low-dimensional sensor values. Furthermore, we demonstrated that facial expression recognition accuracy was improved by making a training dataset that incorporates gaze and head direction as a machine learning method robust to the diversity of facial expressions.
|
Academic Significance and Societal Importance of the Research Achievements |
本研究では学習データの多様性を考慮した機械学習を行なうことで,没入型バーチャル環境における表情認識精度を頑健にできることを示した.こうした,没入型バーチャル環境に対してユーザの身体情報を反映する技術基盤は,我々が没入型バーチャル環境においてソーシャルコミュニケーションを行なう上で必要となるものであり,実環境とバーチャル環境をシームレスに接続することに貢献するものである.また,本研究課題で開発を行なった組み込み型光センサと機械学習を用いる手法は,低次元のセンサ情報から高次元のユーザの状態を推定することにも活用できるため,様々な身体情報の認識を行なっていく上で重要な技術になると期待される.
|
Report
(5 results)
Research Products
(14 results)
-
-
-
[Journal Article] 没入型バーチャル環境における表情認識技術2021
Author(s)
杉本 麻樹
-
Journal Title
Journal of the Virtual Reality Society of Japan
Volume: 26
Issue: 3
Pages: 28-29
DOI
NAID
ISSN
1342-6680, 2435-8746
Year and Date
2021-09-30
Related Report
Open Access
-
-
-
-
-
-
-
-
-
-
-