2006 Fiscal Year Final Research Report Summary
Research on Practical Sign Language Interpretation by integration of image processing of hands and other information
| Project/Area Number |
16091204
|
|---|
| Research Category |
Grant-in-Aid for Scientific Research on Priority Areas
|
| Allocation Type | Single-year Grants |
|---|
| Review Section |
Science and Engineering
|
|---|
| Research Institution | Ritsumeikan University (2005-2006)
Osaka University (2004) |
|---|
Principal Investigator |
SHIRAI Yoshiaki Ritsumeikan University, Human and Computer Intelligence, Professor (50206273)
|
|---|
| Co-Investigator(Kenkyū-buntansha) |
SHIMADA Nobutaka Ritsumeikan Univ., Human and Computer Intelligence, Associate Professor (10294034)
MIURA Jun Osaka Univ., Mechanical Engineering, Associate Professor (90219585)
|
|---|
| Project Period (FY) |
2004 – 2006
|
| Keywords | sign languate interpretation / image processing / hand shape recognition / face expression measurement / information wellfare |
|---|
| Research Abstract |
1. Color is used for extraction of hands and face regions. Because the color of these regions may be varied depending on persons, we developd a method of determining the skin color region in a color space. In this method, approximate regions are first extracted using an apriori skin color region. This region is refined based on the properties of extracted hands and face regions.
In addition, we develop a method to avoid inclusion of the background colors while hands are moving quickly.
2. When hands and face overlap, the face region is first determined in the skin region, and the rest of the skin region is searched for hands. Because the face region is not determned precisely, the certainty of the face region is used for matching hads templates in the skin region.
3. We developed a method of synthesizing sign language samples from a set of real samples to increase samples for HMM learning. Although impovement of recognition rate is proved by experiments, the amount of improvement is small. One reason is that the features of synthesized samples are not suitable because of improper feature extraction. We imporved our feature extraction method.
4. For recognition of sign language words, we used HMM without branching. Because the features of the same sign language word varies depending on persons, we had to prepare multiple models. We have developed a method of building HMM with branching to facilitate models of multiple sign language features.
5. In order to recognize face expressions for sign language recognition, we developed a method of determining the position and shape of face parts, and detecting useful features of face face expressions. In a small scale experiment, typical expressions such ad question, joy, and anger are identified by the method.
|
