Developing Semantic Image Synthesis Model Using Limited Training Data

2022 Fiscal Year Final Research Report

• Project/Area Number: 20K19816
• Research Category: Grant-in-Aid for Early-Career Scientists
• Allocation Type: Multi-year Fund
• Review Section: Basic Section 61010:Perceptual information processing-related
• Research Institution: University of Tsukuba
• Principal Investigator: Endo Yuki 筑波大学, システム情報系, 助教 (00790396)
• Project Period (FY): 2020-04-01 – 2023-03-31
• Keywords: 深層学習 / 画像生成 / GAN

Outline of Final Research Achievements

Semantic image synthesis is a technique that can generate images from a semantic map annotated with pixel-level labels, such as buildings and trees. In this research, we developed an algorithm that can perform high-quality and diverse semantic image synthesis using only a small amount of labeled training data. Furthermore, we also developed a method for controlling the layout of generated images without using any labeled training data. We obtained research outcomes containing semantic image synthesis diversification (two domestic meetings and two journals/international conferences), few-shot semantic image synthesis (one domestic meeting and one journal/international conference), and zero-shot control of image generation (one domestic meeting and one journal/international conference).

Free Research Field

コンピュータグラフィックス

Academic Significance and Societal Importance of the Research Achievements

本研究成果は、ここ数年で急速に発展している画像生成モデルにおいて、ユーザが介入可能な方法を開拓し、意味ラベルマップなど用いて出力を従来よりも低コストで、柔軟かつ多様に制御可能な方法を示したことに学術的な意義がある。社会的には、コンテンツ産業における創作活動の促進だけでなく、自動運転や医用画像解析の画像認識モデルの精度向上のための訓練データの構築など、本技術の広範な応用が期待できる。