2021 Fiscal Year Research-status Report
2D-3D Reconstruction for internal organs using Deep Learning Techniques
| Project/Area Number |
20K20167
|
|---|
| Research Institution | Osaka University |
|---|
Principal Investigator |
武 淑瓊 大阪大学, 産業科学研究所, 助教 (30775763)
|
|---|
| Project Period (FY) |
2020-04-01 – 2023-03-31
|
| Keywords | 3D organ reconstruction / deep Learning / 2D-3D reconstruction |
|---|
| Outline of Annual Research Achievements |
In this research, we proposed a novel architecture that reconstructs 3D data from 2D images. Our purpose is to build a data-driven model so that the reconstruction performance will be stable if only we have training data. Our model is also required to work well on small databases because collecting a lot of clinical data is difficult.
As we mentioned that we would try deep learning models in our last report, in FY2021, we applied 3D GAN (Generative Adversarial Network) to the 2D-3D reconstruction, which reconstructs 3D CT from x-ray images of two views. The approach used convolutional neural networks to extract features from two x-ray images, and then mapped these features to a 3D space. The performance was measured by SSIM (structural similarity index measure). SSIM was about 0.62. To improve the SSIM, we augmented the 3D models using statistical information and narrower ranges. In the augmentation, we considered statistical information both on rotation and affine transformation. Finally, the augmented database were 10 times of the original database. The performance (SSIM) was improved to 0.74.However, this performance is need to be further improved because a score (SSIM) more than 0.9 is expected in real clinic applications.
|
| Current Status of Research Progress |
Current Status of Research Progress
3: Progress in research has been slightly delayed.
Reason
In our original plan, in FY2022, we should summary the results and publish papers. However, the performance of the current model was not satisfactory enough. The current performance is 0.74 (SSIM). However, a score above 0.9 is expected. Therefore, in FY2022, we will continue the improvement of our models and comparison experiments.
After we achieve the high performance, we will summary all the results from both data augmentation and 3D reconstruction, and publish papers.
|
| Strategy for Future Research Activity |
First, we will improve our current models from the following aspects:
(1) Add the shape information of the inner organs into the current 3D GAN model.
(2) Use real X-ray images instead of the DRR (digitally reconstructed radiographs) as the input, and check the reconstructed results.
(3) Apply other data augmentation methods such as info-GAN and Autoencoder.
Secondly, we will conduct evaluation experiments to compare our model with other peer works.
Thirdly, we will summary two papers based on this research. The first one is about the proposed data augmentation method, and the second one is about the proposed 2D-3D reconstruction approach.
|
| Causes of Carryover |
人件費・謝金: 500,000；機械学習で使うGPUの購入：1,000,000；他の消耗品：150,000；論文掲載費：400,000；旅費：500,000
論文発表の際、英語校正が必要です。あと、データ整理のため、バイト一人を雇用するつもりです。深層学習を実施するため、GPUが必要です。そして、他の消耗品（マウスやキーボードやハードディスクなど）も必要です。論文掲載費と発表用の旅費が必要です。
|
