A study for displaying timing between verbal and non-verbal information for a real-time speech-to-caption system

2014 Fiscal Year Final Research Report

• Project/Area Number: 23501096
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Research Field: Educational technology
• Research Institution: Tsukuba University of Technology
• Principal Investigator: KUROKI Hayato 筑波技術大学, 産業技術学部, 教授 (00345159)
• Co-Investigator(Renkei-kenkyūsha): IFUKUBE Tohru 東京大学, 高齢社会総合研究機構, 名誉教授 (70002102) ; NAKANO Satoko 広島大学, アクセシビリティセンター, 特任講師 (20359665)
• Project Period (FY): 2011-04-28 – 2015-03-31
• Keywords: ヒューマン・インタフェース / 障害者支援 / 高等教育支援 / 教育工学 / 認知科学

Outline of Final Research Achievements

This study aimed to investigate the suitable ways of the display display sequence and display timing between captions that have speech recognition errors and the speaker's face movement images for our developing real-time speech-to-caption system for the hearing-impaired.
The results showed that the sequence displaying the caption before the speaker's face image improved the comprehension of the captions. The sequence displaying both simultaneously showed an improvement of only a few percent higher than that of the question sentence, and
the sequence displaying the speaker's face image before the caption showed almost no change. In addition, the sequence displaying the caption 1 second before the speaker's face showed the most significant improvement of all the conditions in the hearing-impaired.

Free Research Field

アシスティブテクノロジー