Computer Vision for Tangible and Intangible Deep Shitsukan

2024 Fiscal Year Final Research Report

• Project Area: Analysis and synthesis of deep SHITSUKAN information in the real world
• Project/Area Number: 20H05951
• Research Category: Grant-in-Aid for Transformative Research Areas (A)
• Allocation Type: Single-year Grants
• Review Section: Transformative Research Areas, Section (IV)
• Research Institution: Kyoto University
• Principal Investigator: Nishino Ko 京都大学, 情報学研究科, 教授 (60814754)
• Co-Investigator(Kenkyū-buntansha): 延原 章平 京都工芸繊維大学, 情報工学・人間科学系, 教授 (00423020) ; 鄭 銀強 東京大学, 大学院情報理工学系研究科, 教授 (30756896)
• Project Period (FY): 2020-11-19 – 2025-03-31
• Keywords: 知覚情報処理 / 視覚情報処理

Outline of Final Research Achievements

Based on the premise that the interaction of people with things, other people, and the space as well as the information necessary to plan and execute actions in the real world lie at the core of Shitsukan, i.e., Deep Shitsukan we seek, we aimed to establish foundations of computational visual perception that can understand tangible instantiations as well as intangible clues of Shitsukan from visual information through research aimed at recovering physical properties and conditions of things, capturing signals reflecting the internal states and intentions of people, and reconstructing spaces and their affordance. This has led to a number of research results consisting of novel algorithms and data that enable the extraction of essential physical quantities and emergent behaviors of people from visual information. These results have been published widely in top international conferences and journals.

Free Research Field

コンピュータビジョン

Academic Significance and Societal Importance of the Research Achievements

物体などの物理量を視覚情報から明示的に復元し、シーンを構成する素材を認識し、人をただ観察するだけでその視線行動や指差しなどの意図発現を捉え、環境の精緻な三次元復元を行い、その中の人物の姿勢を正確に求め、人間も知覚できない方法で表面構成などの物の質を捉えることができる計算機視覚を実現することにより、肉眼でも明らかな質感ばかりでなく、実体はないが感じ取られる質感を定量化する技術基盤を確立した。これらの成果を通し、我々の実世界を彩る質感の本質に迫る手段を広く実現することにより、より豊かな計算機と人間の共生基盤を樹立したばかりではなく、人間の質感認知機構の解明に資すると期待できる。