2021 Fiscal Year Final Research Report
Vocabulary acquisition and 3D avatar approach for Japanese sign language communication
| Project/Area Number |
19K12023
|
|---|
| Research Category |
Grant-in-Aid for Scientific Research (C)
|
| Allocation Type | Multi-year Fund |
|---|
| Section | 一般 |
|---|
| Review Section |
Basic Section 61010:Perceptual information processing-related
|
|---|
| Research Institution | Osaka Prefecture University |
|---|
Principal Investigator |
|
|---|
| Co-Investigator(Kenkyū-buntansha) |
岩村 雅一 大阪府立大学, 工学(系)研究科(研究院), 准教授 (80361129)
井上 勝文 大阪府立大学, 工学(系)研究科(研究院), 准教授 (50733804)
|
|---|
| Project Period (FY) |
2019-04-01 – 2022-03-31
|
| Keywords | Sign lang. recognition / 3D conv. neural networks / Deep learning / Attention Network |
|---|
| Outline of Final Research Achievements |
To improve the performance of existing word-level Sign Language Recognition (W-SLR), in our first approach, a system with a multi-stream structure focusing on global information, local information, and skeletal information was proposed. The local information comprises of handshape and facial expression. The skeleton information captures hand position relative to the body. By combining these three streams, the proposed method achieves higher recognition performance than the state-of-the-art methods.
In the second work, the original I3D network which was originally proposed for action recognition problems has been modified to improve the WSLR performance. The improvement includes an improved inception module named dilated inception module (DIM) and an attention mechanism-based temporal attention module (TAM) to identify the essential features of gestures.
|
| Free Research Field |
Pattern Recognition, Image Processing, etc.
|
| Academic Significance and Societal Importance of the Research Achievements |
Word-level Sign Language Recognition (W-SLR) systems overcome the communication barrier between people with speech impairment and those who can hear. In our approach, we combined these local and relative position of body parts and achieved higher performance on most W-SLR datasets.
|
