2020 Fiscal Year Final Research Report
Deep Representation Learning of Shitsukan Using Vision and Language
| Project Area | Understanding human recognition of material properties for innovation in SHITSUKAN science and technology |
|---|
| Project/Area Number |
15H05919
|
|---|
| Research Category |
Grant-in-Aid for Scientific Research on Innovative Areas (Research in a proposed research area)
|
| Allocation Type | Single-year Grants |
|---|
| Review Section |
Complex systems
|
|---|
| Research Institution | Tohoku University |
|---|
Principal Investigator |
|
|---|
| Co-Investigator(Kenkyū-buntansha) |
川嵜 圭祐 新潟大学, 医歯学系, 准教授 (60511178)
山口 光太 東北大学, 情報科学研究科, 助教 (10742596)
|
|---|
| Project Period (FY) |
2015-06-29 – 2020-03-31
|
| Keywords | 深層学習 / コンピュータビジョン / 質感 / 画像と言語 |
|---|
| Outline of Final Research Achievements |
We have obtained step-by-step achievements towards realizing an AI that can recognize the Shitsukan of an object from its single image. We have developed the following methods: i) a CNN-based method for Shitsukan recognition based on learning to rank image pairs; ii) a method for mining visually recognizable Shitsukan concepts from pairs of images and their descriptions on the Web; iii) a method for generating appropriate titles for images of products while capturing the context-aware Shitsukan concepts. The developments of the above lead to the conclusion that Shitsukan recognition is "comprehensive image recognition," and we successfully developed a network with an attention mechanism, which is an offshoot of today's multimodal deep neural networks for image understanding.
|
| Free Research Field |
画像認識，コンピュータビジョン
|
| Academic Significance and Societal Importance of the Research Achievements |
人と同じように物の質感を視覚的に認識するAIの実現は，最近のAI技術の進展を踏まえても難しかった．質感は人にも言語化が難しく，AIが学習すべき教師データを作るのが難しかったことが大きな理由である．本研究は段階的にこの困難さを解決する方法を与えた．AIによる質感認識が難しいことは現在のAIの人とのギャップの1つであると言え，本研究はこのギャップを狭めることに成功し，これはAIそのものの発展に貢献する学術的な意義である．社会的な意義には，以前は人にしかできなかった質感認識が関わる作業を代替することを初めとする工学的応用がある．
|
