Research on Sign Language Description, Recognition and Gamification of Self-Learning Support System with Explainable Decision of Gesture Correctness
| Project/Area Number |
18K11364
|
|---|
| Research Category |
Grant-in-Aid for Scientific Research (C)
|
| Allocation Type | Multi-year Fund |
|---|
| Section | 一般 |
|---|
| Review Section |
Basic Section 61010:Perceptual information processing-related
|
|---|
| Research Institution | Aichi Institute of Technology |
|---|
Principal Investigator |
|
|---|
| Project Period (FY) |
2018-04-01 – 2021-03-31
|
| Project Status |
Completed (Fiscal Year 2020)
|
| Budget Amount *help |
¥4,420,000 (Direct Cost: ¥3,400,000、Indirect Cost: ¥1,020,000)
Fiscal Year 2020: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000)
Fiscal Year 2019: ¥1,430,000 (Direct Cost: ¥1,100,000、Indirect Cost: ¥330,000)
Fiscal Year 2018: ¥1,820,000 (Direct Cost: ¥1,400,000、Indirect Cost: ¥420,000)
|
|---|
| Keywords | 障害者福祉 / 手話 / 指文字 / ジェスチャ認識 / 学習支援 / ゲーミフィケーション / 学習 |
|---|
| Outline of Final Research Achievements |
This research has engaged on the technology for easy and effective self-learning of sign language. Finger alphabets have been used as learning tasks.
Proposed self-learning support system displays a model finger alphabet CG and created 3D CG of a user's hand-and-fingers measured by sensors simultaneously, and the user can imitate the shape of the alphabet intuitively. The system also decides the correctness of the imitated finger alphabet, explains reasons and points out parts on CG if wrong.
Description rules of finger alphabets' shape and motion, recognition methods of them and an indexing method of continuous alphabets for the system have been proposed and performance experiments with various sensors showed the effectiveness and future tasks of the proposed methods. Gamificational learning support functions for promoting iterative learning have also been introduced into the system, and questionnaire survey revealed the effectiveness and challenges of the proposed functions.
|
| Academic Significance and Societal Importance of the Research Achievements |
本研究では指文字を手指の状態として記述する手法（構造化）およびカテゴリ判定ルールを用いた正誤判定手法（認識）を開発した。これにより、深層学習等では明示化しにくい正誤判定の理由説明を容易に可能とし、また自主学習支援システムに組み込むことで学習効率向上への寄与を検証した。
また、本研究は手話認識技術を導入した機器によるコミュニケーション支援ではなく、健聴者に対する手話学習機会提供のためのシステムを提供することで、聴覚障害者と健聴者間のより直接的で豊かなコミュニケーション支援の実現を可能とした。
|
Report
(4 results)
Research Products
(11 results)
