| 研究課題/領域番号 |
19K12023
|
|---|
| 研究機関 | 大阪府立大学 |
|---|
研究代表者 |
ロイ パルサプラティム 大阪府立大学, 研究推進機構, 客員研究員 (10837222)
|
|---|
| 研究分担者 |
岩村 雅一 大阪府立大学, 工学(系)研究科(研究院), 准教授 (80361129)
井上 勝文 大阪府立大学, 工学(系)研究科(研究院), 准教授 (50733804)
|
|---|
| 研究期間 (年度) |
2019-04-01 – 2022-03-31
|
| キーワード | Sign Lang. Recognition / 3D Conv. Network / Multi-stream network / Optical flow / Skelton / Face / Hand |
|---|
| 研究実績の概要 |
The goal of this research project is to obtain the vocabulary of sign language. To do that, we need a highly accurate sign language recognition method. Hence, in this fiscal year, we have explored existing word-level sign language recognition methods and found that their recognition performance is not sufficient for our goal.
The state-of-the-art recognition method of American sign language was initially developed for action recognition and applied to sign language recognition.
However, due to the difference between action recognition and sign language recognition, methods developed for action recognition cannot grasp the detailed features required for distinguishing sign languages. Thus, we proposed a new multi-stream neural network focusing on local regions. Our experiments revealed that the proposed method significantly improved the recognition accuracy by about 15% in the Top-1 accuracy compared with the previous state-of-the-art method.
We presented our research work at IEICE Technical Committee on Pattern Recognition, and Media Understanding (PRMU) in March 2021, and the first author won the Best Presentation Award of the month.
|
| 現在までの達成度 (区分) |
現在までの達成度 (区分)
2: おおむね順調に進展している
理由
In the second year, we could propose the state-of-the-art word-level sign language recognition method. Using this method can accelerate the development of a method to obtain the vocabulary of sign language.
|
| 今後の研究の推進方策 |
As written above, we plan to develop a method to obtain sign language vocabulary using the proposed multi-stream neural network. In the first year, we have collected a sufficient amount of videos of American sign languages. We will analyze the video data to acquire phoneme-like elements for sign languages. Regarding them as the alphabet, we challenge to describe sign language vocabulary by the phoneme-like elements.
|
| 次年度使用額が生じた理由 |
Due to the pandemic of Covid'19, we could not spend the research budget as planned.
|
