2022 Fiscal Year Research-status Report
2D-3D Reconstruction for internal organs using Deep Learning Techniques
| Project/Area Number |
20K20167
|
|---|
| Research Institution | Osaka University |
|---|
Principal Investigator |
武 淑瓊 大阪大学, 産業科学研究所, 助教 (30775763)
|
|---|
| Project Period (FY) |
2020-04-01 – 2024-03-31
|
| Keywords | 3D reconstruction / Deep learning / GAN / Deformation estimation / 2D-3D registration |
|---|
| Outline of Annual Research Achievements |
The purpose of this research is to develop a data-driven approach which can reconstruct 3D data from 2D images. A typical application is to reconstruct 3D CT data from x-ray images. If we can realize this goal, we can locate the tumors much more accurately than the usual case during a surgery.
To fulfill the goal, we have proposed two different approaches for the 2D-3D reconstruction. First, we reconstructed the 3D CT data from fewer views than normal cases and then reduced the number of views. From-less-to-more reconstruction was implemented in Z axis, and the results were published recently. We tried to reduce the number of views to approximate 2D. However, the results were not so good.
Therefore, we proposed the second approach where we implemented an existing neural network X2CT-GAN. This model can reconstruct 3D CT data from two orthogonal X-ray images. However, the accuracy is not good enough (SSIM is about 0.62). Although we used data augmentation to improve its performance, the SSIM is about 0.74, which is still too low for clinic applications. This time, we exploited the 3D topology of the training CT data, and added the 3D topology features to the X2CT-GAN network. In our model, the system can reconstruct 3D CT data from two x-ray images which are not necessarily the orthogonal ones.
We compared our approach with the original X2CT-GAN, and found our method achieved better performance. Currently, we are implementing more comparison experiments to demonstrate the effectiveness of the proposed model.
|
| Current Status of Research Progress |
Current Status of Research Progress
2: Research has progressed on the whole more than it was originally planned.
Reason
In our research proposal, we planed to reconstruct the 3D CT data from multi-view X-ray images by traditional machine learning algorithms. However, recently the deep learning techniques especially GAN (Generative Adversarial Network) were developed so quickly, and many published papers demonstrated that the deep learning methods outperform the traditional ones. Therefore, we changed the original plan and decided to apply GAN to our research. Though the technology we used was changed, the research still progresses smoothly.
|
| Strategy for Future Research Activity |
The future research will be implemented in two directions.
First, we will continue the experiments to demonstrate that the proposed GAN with 3D topology features are more effective than the original one. After all experiments, we will publish these results on a journal.
Secondly, we will try other data augmentation models to increase the training data for the proposed GAN, and also we will do more comparison experiments to illustrate the effectiveness of both the data augmentation and 3D reconstruction approaches. Finally we will summary all the research work and publish them as a journal paper.
|
| Causes of Carryover |
学会発表のための諸費用
論文掲載のための諸費用
|
