Data augmentation method for quantification of visual aesthetics using deep learning

2022 Fiscal Year Final Research Report

• Project/Area Number: 20K12038
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Review Section: Basic Section 61060:Kansei informatics-related
• Research Institution: Chukyo University
• Principal Investigator: Doman Keisuke 中京大学, 工学部, 准教授 (90645748)
• Co-Investigator(Kenkyū-buntansha): 井手 一郎 名古屋大学, 数理・データ科学教育研究センター, 教授 (10332157) ; 目加田 慶人 中京大学, 工学部, 教授 (00282377)
• Project Period (FY): 2020-04-01 – 2023-03-31
• Keywords: 視覚的感性 / 魅力度 / 深層学習 / データ拡張 / 料理写真

Outline of Final Research Achievements

This research has developed a data augmentation method for generating pairs of an image and its attractiveness value, aiming at a machine learning method for accurately quantifying the visual aesthetics (attractiveness) for food photos. This research has also shown that the accuracy of attractiveness estimation can be improved by using the developed method in combination with an image feature extraction method based on eye gaze information. In addition, this research showed that its application to multi-task learning, which is one of the frameworks of deep learning models, further improves the estimation accuracy. In addition, to explore the possibility of applying the method to data other than food photos, this research studied a method for analyzing and estimating the attractiveness of food recipes including food photos, and analyzed the relationship between the recipe title and the degree of attention to the food recipe.

Free Research Field

画像パターン認識

Academic Significance and Societal Importance of the Research Achievements

本研究では，視覚的感性，特に料理写真を見た際に感じる魅力度という人の曖昧な感覚量を深層学習により定量化する手法を開発した．この技術は，これまでデータセットの質・量が足かせとなっていた分野でも深層学習ベースのアプローチを検討する際の一助となり，関連の各学術分野での技術発展を促進することが期待される．また，人の感性を踏まえた振る舞いが可能な人工知能系の基盤構築の一助となることも期待され，社会と人工知能の新たな関係を形成する一助となることも期待される．