Development of a sign language dictionary using a sign language recognition system and a proposal for an encyclopedia in sign language

2022 Fiscal Year Final Research Report

• Project/Area Number: 18K18517
• Research Category: Grant-in-Aid for Challenging Research (Exploratory)
• Allocation Type: Multi-year Fund
• Review Section: Medium-sized Section 2:Literature, linguistics, and related fields
• Research Institution: National Institute of Technology, Toyota College
• Principal Investigator: KIMURA Tsutomu 豊田工業高等専門学校, 情報工学科, 教授 (80225044)
• Co-Investigator(Kenkyū-buntansha): 神田 和幸 国立民族学博物館, 人類基礎理論研究部, 外来研究員 (70132123)
• Project Period (FY): 2018-06-29 – 2023-03-31
• Keywords: 手話認識 / 手話辞書 / ディープラーニング / 深層学習 / 手話学習

Outline of Final Research Achievements

We have developed an online sign language dictionary system with a sign language recognition function. The system is designed to be used through a web browser. When a user performs a sign language gesture using a webcam, the system recognizes it and displays a list of candidate words. When the user clicks on a word, a sample video is played back for confirmation. Currently, the system can recognize 175 words. Furthermore, we are currently developing a sign language learning system by incorporating the sign language word recognition engine developed in this research. These systems will be released to the public in FY2023.

Free Research Field

福祉工学

Academic Significance and Societal Importance of the Research Achievements

手話を学習するためには，手話サークルに通うなど，対面で習う必要がある．テレビなどでも学習できるが，自身の手話が相手に伝わるかどうかはわからない．英語学習であれば，音声認識を使ったシステムもあるが，手話にはそれがなかった．本研究は，手話学習者に大いに役立てると考えている．さらに，本研究で開発した手話認識用学習ネットワークは，単語だけでなく，手話文，指文字にも利用できる．今後，手話認識システムの開発にも応用できる．