Spotting Finger Spelled Words from Sign Language Video

2015 Fiscal Year Final Research Report

• Project/Area Number: 25282173
• Research Category: Grant-in-Aid for Scientific Research (B)
• Allocation Type: Partial Multi-year Fund
• Section: 一般
• Research Field: Rehabilitation science/Welfare engineering
• Research Institution: University of Tsukuba
• Principal Investigator: Fukui Kazuhiro 筑波大学, システム情報系, 教授 (40375423)
• Co-Investigator(Kenkyū-buntansha): Okazaki Akio 筑波技術大学, 産業技術学部, 教授 (20516679) ; Kato Nobuko 筑波技術大学, 産業技術学部, 教授 (90279555)
• Project Period (FY): 2013-04-01 – 2016-03-31
• Keywords: 指文字 / 手話 / スポッティング / 正準相関分析 / 部分空間表現

Outline of Final Research Achievements

In this research, we developed a new method for spotting specific finger alphabet words from an input sign language video. We addressed the spotting task by employing the basic idea of temporal regularized canonical correlation analysis (TRCCA). The classification accuracy of TRCCA is enhanced by incorporating two functions: 1) parallel processing with multiple time scales, 2) implicit feature mapping by nonlinear orthogonalization. The enhanced TRCCA is called “kernel orthogonal TRCCA (KOTRCCA)”. We demonstrated its effectiveness through the experiments in the task of spotting 8 kinds of finger alphabet words from a sign language vide. In addition, as another approach to spotting, we developed a randomized extension of the DTW, termed randomized time warping (RTW), which generates time elastic (TE) features by randomly sampling the sequential data while retaining the temporal information. We also demonstrated the applicability of RTW through experiments on three public datasets.

Free Research Field

コンピュータビジョン，パターン認識