| Project/Area Number |
22K12095
|
|---|
| Research Category |
Grant-in-Aid for Scientific Research (C)
|
| Allocation Type | Multi-year Fund |
|---|
| Section | 一般 |
|---|
| Review Section |
Basic Section 61010:Perceptual information processing-related
|
|---|
| Research Institution | Iwate Prefectural University |
|---|
Principal Investigator |
戴 瑩 岩手県立大学, ソフトウェア情報学部, 准教授 (60305290)
|
|---|
| Project Period (FY) |
2022-04-01 – 2025-03-31
|
| Project Status |
Granted (Fiscal Year 2023)
|
| Budget Amount *help |
¥4,160,000 (Direct Cost: ¥3,200,000、Indirect Cost: ¥960,000)
Fiscal Year 2024: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000)
Fiscal Year 2023: ¥1,430,000 (Direct Cost: ¥1,100,000、Indirect Cost: ¥330,000)
Fiscal Year 2022: ¥1,560,000 (Direct Cost: ¥1,200,000、Indirect Cost: ¥360,000)
|
|---|
| Keywords | ingredient recognition / ingredient segmentation / food image / decision-making / food recognition / deep learning / food computing |
|---|
| Outline of Research at the Start |
In this research, we focus on realizing the recognition of the visible ingredients in the food images. For this purpose, a new hierarchical structure for recognizing ingredients is proposed based on 農林水産省の生鮮食品品質表示基準. On the basis of this structure, a novel method of segmenting ingredients from the food images is explored. Then the method of extracting and representing the spotlight regions of the ingredients is investigated. Furthermore, an approach of classifying each ingredient is explored. The effectiveness of the proposed methods are evaluated on the prototype system.
|
| Outline of Annual Research Achievements |
Despite remarkable advances in computer vision and machine learning, food image recognition remains very challenging. Machines find it difficult to identify visible ingredients in food images due to significant variability in the shapes of the same ingredients, which often appear visually similar to those from other ingredient categories. In this research, we aim to address these challenges to achieve the recognition of visible ingredients in food images. We also aim to validate the effectiveness and efficiency of the proposed methods, contributing to the development of applications and services in the fields of health, medicine, cooking, nutrition, and related areas.
In 2023, we constructed a single-ingredient image dataset based on 農林水産省の生鮮食品品質表示基準. This dataset was used to train a single-ingredient classification model for recognizing multiple ingredients in food images. Additionally, we developed a multi-ingredient image dataset to rigorously evaluate the performance of multiple ingredient recognition. We then improved a new approach for segmenting multiple ingredients in food images using k-means clustering based on feature maps extracted from the single-ingredient classification model. Furthermore, these segments were recognized using an introduced decision-making scheme. Experimental results validated the effectiveness and efficiency of our method.
|
| Current Status of Research Progress |
Current Status of Research Progress
2: Research has progressed on the whole more than it was originally planned.
Reason
We constructed and improved the single-ingredient image dataset, comprising 9,982 images across 110 diverse categories, emphasizing variety in ingredient shapes and cooking methods. The multiple-ingredient image dataset contains a total of 2,121 images, each depicting multiple ingredients under various cooking conditions.
We proposed a new framework for ingredient segmentation utilizing feature maps of the CNN-based single-ingredient classification model trained on the individual ingredient dataset with image-level annotation. This resolves the problem of excessively hard and time-consuming work required for pixel-level annotations to achieve semantic segmentation.
To tackle the challenge of processing speed in multi-ingredient recognition, we introduced a novel model pruning method to enhance the efficiency of the classification model.
The experiments particularly highlighted its competitive capability in recognizing multiple ingredients compared to state-of-the-art (SOTA) methods. Furthermore, it was found that the CNN-based pruned model enhances the ingredient segmentation accuracy of food images, marking a significant advancement in the field of food image analysis.
|
| Strategy for Future Research Activity |
In previous studies, we focused on addressing the issues of high intra-class variances and class imbalance in ingredient classification. This year, our aim is to solve the problem of high inter-class similarity in multiple ingredient recognition in food images. We propose a novel framework to recognize multiple ingredients, aiming to improve the performance of ingredient recognition by analyzing ingredients that are prone to being classified into other similar categories and introducing new models for these ingredients.
Furthermore, to validate the effectiveness and efficiency of the proposed methods, we plan to build a prototype system for multiple ingredient recognition in food images in the MATLAB environment.
|
