Deep Learning-based Semantics Information Compensation for Color Vision Deficiency

2023 Fiscal Year Final Research Report

• Project/Area Number: 22K21274
• Research Category: Grant-in-Aid for Research Activity Start-up
• Allocation Type: Multi-year Fund
• Review Section: 1001:Information science, computer engineering, and related fields
• Research Institution: University of Yamanashi
• Principal Investigator: ZHU ZHENYANG 山梨大学, 大学院総合研究部, 助教 (10954927)
• Project Period (FY): 2022-08-31 – 2024-03-31
• Keywords: 色覚補償 / 拡張現実 / 深層学習

Outline of Final Research Achievements

In this study, a color vision compensation system called Color Communication Glasses (CC-Glasses) was developed, which incorporates a deep learning model and augmented reality (AR) glasses to compensate for semantic information loss due to color vision deficiency (CVD). Assuming that users with CVD have difficulty identifying target objects specified by colors, the system sends scenes shot captured by AR glasses to a server that incorporates a deep learning model, and displays the analyzed results on AR glasses. We created a dataset for training the deep learning model used in this study. Moreover, in order to verify the effectiveness of the proposed system, an evaluation experiment involving people with CVD was conducted.

Free Research Field

ビジュアルコンピューティング

Academic Significance and Societal Importance of the Research Achievements

本研究は医学と情報工学の共同研究であり，未だに医学的な根治手段が見つかってない色覚障がいに対する情報技術による支援策を提供する．医工融合研究の重要性に関する発信にもつながる．本研究で開発するAI技術により，色の意味情報損失が補償され，CVD患者が自力で色を正確に認識したり形容することが可能となり，危険にさらされるリスクを避けることができる．また，日常生活補助のみならず，仕事面でも活躍できる分野が広がると考えられる．さらに，CVD患者が周囲とのコミュニケーションがより取りやすくなるため，社会全体においてQoLの向上が期待される．