2006 Fiscal Year Final Research Report Summary
Research on an effective method of video communication of sign language in consideration of cognitive characteristics
| Project/Area Number |
16091209
|
|---|
| Research Category |
Grant-in-Aid for Scientific Research on Priority Areas
|
| Allocation Type | Single-year Grants |
|---|
| Review Section |
Science and Engineering
|
|---|
| Research Institution | Kogakuin University |
|---|
Principal Investigator |
NAGASHIMA Yuji Kogakuin University, Faculty of Infomatics, Professor (50138137)
|
|---|
| Project Period (FY) |
2004 – 2006
|
| Keywords | Cognitive Science / Sign Linguistic Engineering / Video Communication / ERP / ICT on Well-being / Corpus / Commnnication Assistive Technology |
|---|
| Research Abstract |
This research has aimed at the communication support using ICT for the hearing impaired people. Provided result is the following items.
(a) We are concerned with language cognition, semantic processing process analysis based on N400 ; event related potential measurement was examined for the understanding the Japanese sign language comprehension mechanisms. As a result, because it is understood that N400 is observed in semantic congruity sentence by sign language and the character, and the potential is large in the character stimulation, and was shown the difference of the processing process by the presentation modality.
(b) We discussed evaluation techniques for the quality of digitally encoded motion picture of sign language. This result guarantees that opinion test can be substituted for intelligibility test. Further investigation shows that motion pictures with high frame rate are more legible than those with high precision. Another result shows that screen size of sign picture doesn'
… More
t affect comprehensibility.
(c) We are concerned with low-bit rate digital encoding of moving picture of sign language. A new encoding method suitable for sign language is presented based on characteristics of sign language. As a result, an encoding image data using our proposed method is more possible at a low bit rate in comparison with an existing method. Moreover, our method in the same parameter condition has been confirmed that it was obtained a clearer image in comparison with an existing method.
(d) We have also worked on a problem of mental workload, which is mental stress that sign language users undergo during interactive communication. We have found that NASA-TLX is effective enough for measuring the degree of mental workload
(e) In the research process, we collected a corpus by video image and electronic data by three-dimensional movement for linguistic analysis of sign language. The electronic data showed it on Web as sign language animation by the BoT3D engine.
Our findings have been appeared in journals such as the one of the transactions on IEICE, and the one of Human Interface Society.
We feel necessity to develop this research. On the basis of result of this research, we continue analysis work from the side of cognitive science through conversation analysis of sign language. Less
|
