2D-3D Reconstruction for internal organs using Deep Learning Techniques

2023 Fiscal Year Final Research Report

• Project/Area Number: 20K20167
• Research Category: Grant-in-Aid for Early-Career Scientists
• Allocation Type: Multi-year Fund
• Review Section: Basic Section 90110:Biomedical engineering-related
• Research Institution: Osaka University
• Principal Investigator: Wu Shuqiong 大阪大学, 産業科学研究所, 助教 (30775763)
• Project Period (FY): 2020-04-01 – 2024-03-31
• Keywords: 3D reconstruction / machine learning / X-ray / CT / deformation estimation

Outline of Final Research Achievements

In this study, we aim to develop a high-precision technique for reconstructing three-dimensional structures from two-dimensional images. To improve the accuracy of three-dimensional reconstruction, we conducted researches in two directions: deep-learning-based methods and methods based on Sparse-View. We analyzed the weaknesses of an existing learning model X2CTGAN and improved the reconstruction performance by adding new projection constraints and a rotational transformation, resulting in an increase in the SSIM score from approximately 0.6 to 0.8. We will summary these results and submit them to an international conference.
Meanwhile, we also researched methods to reconstruct three-dimensional structures from Sparse to Dense views, and successfully reconstructed in the Z-direction of CT images. The results have already been published as a journal paper.

Free Research Field

Computer Vision

Academic Significance and Societal Importance of the Research Achievements

本研究では、二次元から三次元を再構築する手法を提案した。この手法は、さまざまな場面で活用できると考えられる。例えば、X線画像からCTの再構築、シングルビューからマルチビューの再構築、RGB画像から三次元動作の再構築などに役立つ。さらに、提案したデータ拡張手法により、学習データが不足していても、効果的に学習ができ、汎化性能も向上するようになった。こうしたデータ拡張の手法は、他の研究テーマでも適用可能である。本研究では、基礎技術の研究開発が進み、さまざまなアプリケーションにも適用できると考えられ、それによって大きな社会的な意義があると考えられる。