Support on visualization of environmental sound for hearing impaired person -Support by Ambient fonts utilized Manga techniques -

2019 Fiscal Year Final Research Report

• Project/Area Number: 17K01102
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Research Field: Educational technology
• Research Institution: Kinjo College
• Principal Investigator: Arai Hiroshi 金城大学短期大学部, 美術学科, 准教授 (30331557)
• Project Period (FY): 2017-04-01 – 2020-03-31
• Keywords: 障害学習支援 / 環境音 / 臨場感フォント / 漫符 / オノマトペ

Outline of Final Research Achievements

Information obtained by hearing has less than by visual. Therefore, we have few opportunities to be aware of it on our daily life. However, it is very important to be able to obtain information outside the field of view, such as behind and beyond a wall.

The purpose of this research is to develop a system that visualizes environmental sounds and support students with hearing impairments. In the present research, we focused on environmental sounds with dangers that hearing impaired may encounter.This system consists of an 8-channel microphone array and robot audition software "HARK". The system recognizes the direction and distance of the sound source. And the visualization of the recognition results are used "Ambient font" that we developed in the previous study. The “Ambient font” is an expression method utilized manga techniques.

Free Research Field

複合領域　デザイン学

Academic Significance and Societal Importance of the Research Achievements

このシステムにより聴覚障害学生が日常的な危険・注意に関わる場で過ごしやすくなる。本研究ではヘッドマウントディスプレイによる情報提示であったが、ARコンタクトレンズなどデバイスの発達により、より有効活用が期待できる。また副次的な成果として、文字情報に依存度の高いコミュニケーションがとられる、SNSに代表されるWEB上のやりとりにおいて、重層的な臨場感ある情報交換が可能となる。