High-dimensional organ state reconstruction during therapy using complex sensing

• Project/Area Number: 18K19918
• Research Category: Grant-in-Aid for Challenging Research (Exploratory)
• Allocation Type: Multi-year Fund
• Review Section: Medium-sized Section 90:Biomedical engineering and related fields
• Research Institution: Kyoto University
• Principal Investigator: Nakao Megumi 京都大学, 情報学研究科, 准教授 (10362526)
• Project Period (FY): 2018-06-29 – 2021-03-31
• Project Status: Completed (Fiscal Year 2020)
• Budget Amount: ¥6,370,000 (Direct Cost: ¥4,900,000、Indirect Cost: ¥1,470,000); Fiscal Year 2020: ¥1,690,000 (Direct Cost: ¥1,300,000、Indirect Cost: ¥390,000); Fiscal Year 2019: ¥2,210,000 (Direct Cost: ¥1,700,000、Indirect Cost: ¥510,000); Fiscal Year 2018: ¥2,470,000 (Direct Cost: ¥1,900,000、Indirect Cost: ¥570,000)
• Keywords: 機械学習 / 臓器変形 / 治療支援 / 医用人工知能 / 圧縮センシング / 医用システム

Outline of Final Research Achievements

The purpose of this study is to reconstruct high-dimensional state of patient-specific organs from low-dimensional and local information during treatment by using prior knowledge on organ structures and treatment procedure. A deformable mesh registration method was newly proposed and a statistical deformation model of multiple abdominal organs was constructed. We also built an Image-to-graph convolutional network that estimates the 3D shape from the 2D projection image. We confirmed the estimation accuracy of the 3D shape and deformation of the liver associated with respiration from only a single 2D X-ray image. The shape features of surrounding organs that can accurately estimate the respiratory motion/deformation of pancreatic cancer were identified.

Academic Significance and Societal Importance of the Research Achievements

治療時における生体臓器の幾何学的・力学的状態の理解は生体医工学や放射線治療，コンピュータ外科学において重要なテーマであり，臨床におけるニーズも高い研究課題である．本研究では，治療時の追加計測を前提とせず，計算機で低次元かつ局所的な情報から臓器の形状や変形を再構成可能な枠組みを構築した．統計的変位モデルは生体臓器の3次元形状と変形をより低次元の画像特徴などから推定する目的において事前知識，臓器データベースとして利用可能である． また，2次元X線画像では検出されない膵がんの呼吸性変位を推定可能な周辺臓器の形状特徴量を同定し，臨床利用可能な推定精度を達成することを示した．

Research Products

• [Journal Article] Statistical deformation reconstruction using multi-organ shape features for pancreatic cancer localization (2021)
  • Author(s): Nakao Megumi、Nakamura Mitsuhiro、Mizowaki Takashi、Matsuda Tetsuya
  • Journal Title: Medical Image Analysis Volume: 67 Pages: 101829-101829
  • DOI: 10.1016/j.media.2020.101829
  • Peer Reviewed / Open Access / Int'l Joint Research
• [Journal Article] Statistical shape model-based planning organ-at-risk volume: application to pancreatic cancer patients (2021)
  • Author(s): M. Nakamura, M. Nakao, N. Mukumoto, R. Ashida, H. Hirashima, M. Yoshimura, T. Mizowaki
  • Journal Title: Physics in Medicine & Biology Volume: 66 Issue: 1 Pages: 014001-014001
  • DOI: 10.1088/1361-6560/abcd1b
  • Peer Reviewed
• [Journal Article] Regularized Three-Dimensional Generative Adversarial Nets for Unsupervised Metal Artifact Reduction in Head and Neck CT Images (2020)
  • Author(s): M. Nakao, K. Imanishi, N. Ueda, Y. Imai, T. Kirita, T. Matsuda
  • Journal Title: IEEE Access Volume: 8 Pages: 109453-109453
  • DOI: 10.1109/access.2020.3002090
  • Peer Reviewed / Open Access
• [Journal Article] Surface deformation analysis of collapsed lungs using model-based shape matching (2019)
  • Author(s): Megumi Nakao, Junko Tokuno, Toyofumi Chen-Yoshikawa, Hiroshi Date, Tetsuya Matsuda
  • Journal Title: Int J Comp Assist Rad Surg Volume: 14 Issue: 10 Pages: 1763-1774
  • DOI: 10.1007/s11548-019-02013-0
  • Peer Reviewed / Open Access / Int'l Joint Research
• [Journal Article] Statistical deformation reconstruction using multi-organ shape features for pancreatic cancer localization (2019)
  • Author(s): M. Nakao, M. Nakamura, T. Mizowaki, T. Matsuda
  • Journal Title: arXiv Volume: -
  • Open Access / Int'l Joint Research
• [Journal Article] Performance evaluation of a newly developed three-dimensional model-based global-to-local registration in prostate cancer, Journal of Radiation Research (2019)
  • Author(s): M. Nakamura, M. Nakao, H. Hirashima, H. Iramina, T. Mizowaki
  • Journal Title: Journal of Radiation Research Volume: in press
  • Peer Reviewed / Int'l Joint Research
• [Presentation] Kernel-based modeling of pneumothorax deformation using intraoperative cone-beam CT images (2021)
  • Author(s): M. Nakao, H. Maekawa, K. Mineura, T. Chen-Yoshikawa, and T. Matsuda
  • Organizer: SPIE Medical Imaging
  • Int'l Joint Research
• [Presentation] X-ray2Shape: グラフ畳み込みネットワークを用いた単一X-ray画像からの臓器形状再構成 (2021)
  • Author(s): Fei Tong, 中尾 恵, 武 淑瓊, 中村 光宏, 松田 哲也
  • Organizer: 情報処理学会 コンピュータビジョンとイメージメディア研究会(CVIM)
• [Presentation] 擬似カメラ画像の内部補完学習を用いた拡張内視鏡画像の生成 (2021)
  • Author(s): 新田 潤平, 中尾 恵, 松田 哲也
  • Organizer: 情報処理学会 コンピュータビジョンとイメージメディア研究会(CVIM)
• [Presentation] 気胸変形の学習に基づく肺部分形状メッシュの可変形位置合わせ (2021)
  • Author(s): 前川日南子, 中尾 恵, 峯浦 一貴, 芳川 豊史, 松田 哲也
  • Organizer: 電子情報通信学会技術報告 (MI)
• [Presentation] Deep learning based lung region segmentation with data preprocessing by generative adversarial nets (2020)
  • Author(s): J. Nitta, M. Nakao, K. Imanishi, T. Matsuda
  • Organizer: Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)
• [Presentation] Model-based registration for pneumothorax deformation analysis using intraoperative cone-beam CT images (2020)
  • Author(s): H. Maekawa, M. Nakao, K. Mineura, T.F. Chen-Yoshikawa, T. Matsuda
  • Organizer: Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)
• [Presentation] X-ray2Shape: Reconstruction of 3D liver shape from a single 2D projection image (2020)
  • Author(s): F. Tong, M. Nakao, S. Wu, M. Nakamura, T. Matsuda
  • Organizer: Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)
• [Presentation] 胸腔鏡画像における教師なし画像変換を用いた肺領域抽出 (2020)
  • Author(s): 新田 潤平, 中尾 恵, 今西 勁峰, 松田 哲也
  • Organizer: 電子情報通信学会技術報告 (MI)
• [Presentation] グラフ畳み込みネットワークを用いた単一X-ray画像からの3次元臓器形状の再構成 (2020)
  • Author(s): Fei Tong, 中尾 恵, 武 淑瓊, 中村 光宏, 松田 哲也
  • Organizer: 電子情報通信学会技術報告 (MI)
• [Presentation] Cone-Beam CT画像の画質改善を目的とした3次元敵対的生成ネットワークの提案 (2020)
  • Author(s): 羽瀬 拓視, 中尾 恵, 今西 勁峰, 中村 光宏, 松田 哲也
  • Organizer: 電子情報通信学会技術報告 (MI)
• [Presentation] スパース生体モデリングと治療支援画像生成 (2020)
  • Author(s): 中尾 恵
  • Organizer: 日本医学物理学会学術大会
  • Invited
• [Presentation] X-ray2Shape: Reconstruction of 3D Liver Shape from a Single 2D Projection Image (2020)
  • Author(s): F. Tong, M. Nakao, S. Wu, M. Nakamura, T. Matsuda
  • Organizer: 42nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)
  • Int'l Joint Research
• [Presentation] Model-based registration for pneumothorax deformation analysis using intraoperative cone-beam CT images (2020)
  • Author(s): H. Maekawa, M. Nakao, K. Mineura, T.F. Chen-Yoshikawa, T. Matsuda
  • Organizer: 42nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)
  • Int'l Joint Research
• [Presentation] スパース生体モデリングと治療支援画像生成 (2020)
  • Author(s): 中尾 恵
  • Organizer: 第119回日本医学物理学会学術大会
  • Invited
• [Presentation] 術中CBCT画像を用いた虚脱肺の変形解析の試み (2020)
  • Author(s): 前川 日南子, 中尾 恵, 峯浦 一貴, 芳川 豊史, 松田 哲也
  • Organizer: 電子情報通信学会技術報告 (MI)
• [Presentation] 統計学的アプローチによるPRVマージンの決定 (2019)
  • Author(s): 中村 光宏，中尾 恵，椋本 宜学，芦田 良，平島 英明，吉村 通央，溝脇 尚志
  • Organizer: 日本放射線腫瘍学会
• [Presentation] モデルベース位置合わせによる動物気胸肺の変形解析 (2019)
  • Author(s): 小林 晃太郎, 中尾 恵, 徳野 純子, 陳 豊史, 伊達 洋至, 松田 哲也
  • Organizer: 電子情報通信学会技術報告 (MI)
• [Presentation] D U-Netの畳み込みネットワークを用いた隣接スライスからのCT画像再構成 (2019)
  • Author(s): 武 淑瓊, 中尾 恵, 今西 勁峰, 中村 光宏, 松田 哲也
  • Organizer: 第38回日本医用画像工学会大会
• [Presentation] Internal Structure Registration of Deaeration Deformation in vivo Animal Lung (2019)
  • Author(s): K. Kobayashi, M. Nakao, J. Tokuno, T. F. Chen, H. Date, T. Matsuda
  • Organizer: Int. J. Computer Assisted Radiology and Surgery
  • Int'l Joint Research
• [Presentation] Reconstructing 3D Lung Shape from a Single 2D Image during the Deaeration Deformation Process using Model-based Data Augmentation (2019)
  • Author(s): S. Wu, M. Nakao, J. Tokuno, T. Chen-Yoshikawa, T. Matsuda
  • Organizer: IEEE EMBS International Conference on Biomedical and Health Informatics (BHI)
  • Int'l Joint Research
• [Presentation] 複数周辺臓器の形状特徴に基づく膵癌変位推定法 (2019)
  • Author(s): 岩井 泰児, 中尾 恵, 中村 光宏, 松田 哲也
  • Organizer: 第63回システム制御情報学会
• [Presentation] Sparse Modeling of Mandibular Reconstruction Procedures Using Statistical Geometric Features (2018)
  • Author(s): M. Nakao, T. Matsuda
  • Organizer: 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)
• [Presentation] A Preliminary Study on a Statistical Deformation Model of Multiple Organs for Reconstructing Respiratory Displacement (2018)
  • Author(s): T. Iwai, M. Nakao, M. Nakamura, T. Matsuda
  • Organizer: 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)
• [Presentation] 局所変位観測に基づく弾性率分布のモデルベース推定 (2018)
  • Author(s): 森田 充樹, 中尾 恵, 松田 哲也
  • Organizer: 電子情報通信学会技術報告 (MI)
• [Presentation] 動体追尾放射線治療のための複数周辺臓器の多次元特徴量に基づく膵癌変位推定法 (2018)
  • Author(s): 岩井 泰児, 中尾 恵, 中村 光宏, 松田 哲也
  • Organizer: 電子情報通信学会技術報告 (MI)
• [Remarks] Research Home
  • URL: http://www.bme.sys.i.kyoto-u.ac.jp/~meg/
• [Remarks] Research Home
  • URL: http://www.bme.sys.i.kyoto-u.ac.jp/~meg/index.html
• [Patent(Industrial Property Rights)] 機械学習装置、推定装置、プログラム及び学習済みモデル (2018)
  • Inventor(s): 中尾 恵, 陳 豊史, 徳野 純子, 伊達 洋至, 松田 哲也
  • Industrial Property Rights Holder: 京都大学
  • Industrial Property Rights Type: 特許
  • Industrial Property Number: 2018-171825
  • Filing Date: 2018