2021 Fiscal Year Final Research Report
Development of Gaze and Head Direction Detection and Lip-Reading Technology Using Pupil and Nostril Positions for Small Devices
| Project/Area Number |
19K04293
|
|---|
| Research Category |
Grant-in-Aid for Scientific Research (C)
|
| Allocation Type | Multi-year Fund |
|---|
| Section | 一般 |
|---|
| Review Section |
Basic Section 20020:Robotics and intelligent system-related
|
|---|
| Research Institution | Shizuoka University |
|---|
Principal Investigator |
|
|---|
| Project Period (FY) |
2019-04-01 – 2022-03-31
|
| Keywords | 瞳孔 / 鼻孔 / 口領域 / 視線検出 / 顔方向 / 口唇 / 小型デバイス / 発話解析 |
|---|
| Outline of Final Research Achievements |
This study attempts to develop a technology for detecting gaze and head direction and lip reading that can be used in small devices. The three-dimensional coordinates of the users' pupils and nostrils were detected in images from two black-and-white cameras and the gaze and head direction and the optimal mouth area for each user were detected from the relative positions of the pupils and nostrils. The most accurate classification of vowels was achieved when the mouth region images were normalized using images of the users facing sideways. Introducing CNN enabled us to estimate the positions of the pupils when the users' eyes were closed, even when the users moved their heads. While implementing the system on a small device, it was possible to detect the users' gazing point on a small display attached to a smartphone mockup.
|
| Free Research Field |
人間工学
|
| Academic Significance and Societal Importance of the Research Achievements |
本研究では，開発した技術の小型デバイスへの実装までには至らなかったが，本技術が使用されれば，料理をしながら，化粧をしながらとった小型デバイスの「ながら操作」が可能となる．また，頭部方向検出による「頭部ジェスチャー」も可能になるため，視線と組み合わせた多種多様な操作が期待できる．読唇技術では，公共施設やバス，電車など，雑音により音声認識が困難な場所や，会議や講演会といった静寂さを求められる環境などにおける「サイレント音声認識」が可能になり，発話することなく，文字入力が可能となるため，より利便性の向上が図れる．
|
