Study of Analysis and Description of Visual Communication Based on the Real World Mapping Models

2015 Fiscal Year Final Research Report

• Project/Area Number: 24500644
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Research Field: Rehabilitation science/Welfare engineering
• Research Institution: The Hyogo Institute of Assistive Technology (2014-2015); Kogakuin University (2013); Kyoto Institute of Technology (2012)
• Principal Investigator: Nakazono Kaoru 兵庫県立福祉のまちづくり研究所, その他部局等, 研究員 (60531267)
• Co-Investigator(Kenkyū-buntansha): NAGASHIMA Yuji 工学院大学, 情報工学科, 教授 (50138137)
• Project Period (FY): 2012-04-01 – 2016-03-31
• Keywords: ヒューマンコンピュータインタラクション / ヒューマンコミュニケーション / ビジュアルコミュニケーション / 福祉情報工学 / 手話 / ピクトグラム

Outline of Final Research Achievements

Deaf people use a visual language (sign language) and there are some aspects of their expressions that can be implemented on the design of an animated movie. We studied the simple recognizable design method for pictograms which is imported from the expression manner using visual mapping technique specific to sign language. Furthermore, we constructed layered description model of visual expression from macroscopic to microscopic layer.
First, we extracted sign language specific expression manners from analysis of sign conversation video. Based on these manners, very simple moving pictograms which expresses abstract notions were designed and intelligibility of them comparing to the static pictograms was confirmed by evaluation experiments. Then, we tried to generate animated short movie using TVML, TV Program Language. Furthermore, we studied technique to simplify troublesome TVML programming works by using framework of multi layer animated movie description technique.

Free Research Field

福祉情報工学