Sign Language Recognition based on Subunits of manual signals and non-manual signals

2015 Fiscal Year Final Research Report

• Project/Area Number: 25350666
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Research Field: Rehabilitation science/Welfare engineering
• Research Institution: Nagoya Institute of Technology
• Principal Investigator: Kitamura Tadashi 名古屋工業大学, 工学(系)研究科(研究院), 教授 (60114865)
• Co-Investigator(Kenkyū-buntansha): SAKO Shinji 名古屋工業大学, 大学院工学研究科, 助教 (30396791)
• Project Period (FY): 2013-04-01 – 2016-03-31
• Keywords: 手話認識 / 手指動作 / 非手指動作 / Kinect

Outline of Final Research Achievements

A sign language consists of manual signals and non-manual signals. Manual signals convey the meaning of the word by three elements, e.g. the shapes, the movements and the positions of the hands. Non-manual signals such as facial expressions, head tilting etc. work to add grammatical functions.
We carried out recognition experiments for manual signals using database recorded by 2 persons, 223 words, twice. The results have shown that the recognition accuracies for 24 hand shapes, 8 positions, 40 movements, are 28.5%, 78.3%, 60.0% respectively. The word recognition accuracy rate was 33.8% using combination of weighted scores of each element. The recognition accuracy rate for top 10 was 79.7%. The recognition experiment of non-manual signals for 3 different motions of head tilting was performed. In the experiments 66 sentences were used for training the Hidden Markov Models of the motions. The recognition rate was 79% for 125 test sentences.

Free Research Field

情報工学