2023 Fiscal Year Research-status Report
A novel study on visible ingredient identification in food images for food computing
| Project/Area Number |
22K12095
|
|---|
| Research Institution | Iwate Prefectural University |
|---|
Principal Investigator |
戴 瑩 岩手県立大学, ソフトウェア情報学部, 准教授 (60305290)
|
|---|
| Project Period (FY) |
2022-04-01 – 2025-03-31
|
| Keywords | ingredient recognition / ingredient segmentation / food image / decision-making |
|---|
| Outline of Annual Research Achievements |
Despite remarkable advances in computer vision and machine learning, food image recognition remains very challenging. Machines find it difficult to identify visible ingredients in food images due to significant variability in the shapes of the same ingredients, which often appear visually similar to those from other ingredient categories. In this research, we aim to address these challenges to achieve the recognition of visible ingredients in food images. We also aim to validate the effectiveness and efficiency of the proposed methods, contributing to the development of applications and services in the fields of health, medicine, cooking, nutrition, and related areas.
In 2023, we constructed a single-ingredient image dataset based on 農林水産省の生鮮食品品質表示基準. This dataset was used to train a single-ingredient classification model for recognizing multiple ingredients in food images. Additionally, we developed a multi-ingredient image dataset to rigorously evaluate the performance of multiple ingredient recognition. We then improved a new approach for segmenting multiple ingredients in food images using k-means clustering based on feature maps extracted from the single-ingredient classification model. Furthermore, these segments were recognized using an introduced decision-making scheme. Experimental results validated the effectiveness and efficiency of our method.
|
| Current Status of Research Progress |
Current Status of Research Progress
2: Research has progressed on the whole more than it was originally planned.
Reason
We constructed and improved the single-ingredient image dataset, comprising 9,982 images across 110 diverse categories, emphasizing variety in ingredient shapes and cooking methods. The multiple-ingredient image dataset contains a total of 2,121 images, each depicting multiple ingredients under various cooking conditions.
We proposed a new framework for ingredient segmentation utilizing feature maps of the CNN-based single-ingredient classification model trained on the individual ingredient dataset with image-level annotation. This resolves the problem of excessively hard and time-consuming work required for pixel-level annotations to achieve semantic segmentation.
To tackle the challenge of processing speed in multi-ingredient recognition, we introduced a novel model pruning method to enhance the efficiency of the classification model.
The experiments particularly highlighted its competitive capability in recognizing multiple ingredients compared to state-of-the-art (SOTA) methods. Furthermore, it was found that the CNN-based pruned model enhances the ingredient segmentation accuracy of food images, marking a significant advancement in the field of food image analysis.
|
| Strategy for Future Research Activity |
In previous studies, we focused on addressing the issues of high intra-class variances and class imbalance in ingredient classification. This year, our aim is to solve the problem of high inter-class similarity in multiple ingredient recognition in food images. We propose a novel framework to recognize multiple ingredients, aiming to improve the performance of ingredient recognition by analyzing ingredients that are prone to being classified into other similar categories and introducing new models for these ingredients.
Furthermore, to validate the effectiveness and efficiency of the proposed methods, we plan to build a prototype system for multiple ingredient recognition in food images in the MATLAB environment.
|
| Causes of Carryover |
今年度は、国際会議での研究成果発表がオンラインで行われたため、旅費が削減された。また、Journal of Imagingに掲載された論文が招待論文であったため、掲載料が免除された。次年度は、その分をオーストラリアで開催される国際会議ACM MM 2024の旅費に使用することを考えている。
|
