Weakly-Supervised Differentiation of Focal Liver Lesions Using Phase-Attention Net

2021 Fiscal Year Annual Research Report

• Project/Area Number: 21H03470
• Allocation Type: Single-year Grants
• Research Institution: Ritsumeikan University
• Principal Investigator: 陳 延偉 立命館大学, 情報理工学部, 教授 (60236841)
• Co-Investigator(Kenkyū-buntansha): 岩本 祐太郎 立命館大学, 情報理工学部, 助教 (30779054) ; 韓 先花 山口大学, 大学院創成科学研究科, 准教授 (60469195) ; 古川 顕 東京都立大学, 人間健康科学研究科, 教授 (80199421) [Withdrawn] ; 金崎 周造 滋賀医科大学, 医学部, 非常勤講師 (90464180)
• Project Period (FY): 2021-04-01 – 2025-03-31
• Keywords: 肝臓がん / 深層学習 / アテンション機構 / 弱教師あり学習 / 教師なし学習 / 肝臓腫瘍セグメンテーション / 早期再発予測

Outline of Annual Research Achievements

肝臓がんは、平均5年生存率が30％程度とがん全体の中でも極めて低く、正確な診断に基づく早期発見が求められる。肝臓がんの診断には、マルチフェーズCT画像（造影剤を注入してから経時的に複数回の撮像で得られたCT画像）がよく用いられる。近年深層学習を代表とする人工知能による肝腫瘍性病変の診断支援が多く研究されているが、既存手法では、個々のフェーズから特徴を抽出し、診断を行っていたため、診断精度は80%に留まっていた。また、事前に腫瘍を検出または分割する必要があった。本研究では、学習により腫瘍の種類毎にフェーズ間の重みを自動的に調整し、鑑別に重要なフェーズを強調可能なフェーズアテンション機構を提案する。マルチフェーズCT画像から有効時空間特徴抽出法を確立するとともに、腫瘍領域のアノテーションを必要としない弱教師学習による肝腫瘍性病変の検出・鑑別法を確立する。以下の研究成果が挙げられた。
(1)フェーズ内アテンション機構とフェーズ間アテンション機構を提案し、マルチフェーズCT画像における肝臓腫瘍を高精度にセグメンテーションすることができた。
(2)フェーズアテンション深層学習モデルを提案し、マルチフェーズCT画像を用いた肝臓がんの術後再発を高精度に予測することができた。
(3)敵対的生成ネットワークを用いた弱教師あり学習法と教師なし学習法による肝臓腫瘍の検出とセグメンテーション法を開発した。
(4)TIP, TMI, JBHI, MICCAI, AAAI, ICCVなどのトップ学術誌及び国際学会で成果を発表してきた。
(5)General Chairとして、2021年6月に国際学会 InMed2021を主催し、国際連携ネットワークを構築した。

Current Status of Research Progress

2: Research has progressed on the whole more than it was originally planned.

Reason

計画通りに以下の成果が得られた。
(1)フェーズ内アテンション機構とフェーズ間アテンション機構を提案し、マルチフェーズCT画像における肝臓腫瘍を高精度にセグメンテーションすることができた。
(2)フェーズアテンション深層学習モデルを提案し、マルチフェーズCT画像を用いた肝臓がんの術後再発を高精度に予測することができた。
(3)敵対的生成ネットワークを用いた弱教師あり学習法と教師なし学習法による肝臓腫瘍の検出とセグメンテーション法を開発した。
(4)TIP, TMI, JBHI, MICCAI, AAAI, ICCVなどのトップ学術誌及び国際学会で成果を発表してきた。

Strategy for Future Research Activity

今後は以下の研究項目を中心に研究を進めていく。
(1)フェーズアテンションとチャンネルアテンション、空間アテンションを融合した複合アテンション機構を新たに開発し、高精度な肝臓がんの支援診断システムを開発する。
(2)少ないデータでも学習できる、新たな弱教師学習と半教師学習のフレームワークを開発する。
(3)マルチフェーズCT画像だけではなく、マルチフェーズMR画像や顕微鏡画像などの融合による高精度な肝臓がんの支援診断システムを開発する。

Research Products

• [Int'l Joint Research] 浙江大学計算機学院/浙江大学医学部附属病院(中国)
  • Country Name: CHINA
  • Counterpart Institution: 浙江大学計算機学院/浙江大学医学部附属病院
• [Journal Article] 深層学習を用いた肝臓がんの診断支援 (2023)
  • Author(s): 陳延偉
  • Journal Title: 計測と制御 Volume: 第62巻 Pages: 91-94
  • DOI: 10.11499/sicejl.62.91
• [Journal Article] MTL-ABS3Net: Atlas-Based Semi-Supervised Organ Segmentation Network With Multi-Task Learning for Medical Images (2022)
  • Author(s): Huang Huimin、Chen Qingqing、Lin Lanfen、Cai Ming、Zhang Qiaowei、Iwamoto Yutaro、Han Xianhua、Furukawa Akira、Kanasaki Shuzo、Chen Yen-Wei、Tong Ruofeng、Hu Hongjie
  • Journal Title: IEEE Journal of Biomedical and Health Informatics Volume: 26 Pages: 3988～3998
  • DOI: 10.1109/JBHI.2022.3153406
  • Peer Reviewed / Int'l Joint Research
• [Journal Article] DeepRecS: From RECIST Diameters to Precise Liver Tumor Segmentation (2022)
  • Author(s): Yue Zhang, Chengtao Peng, Liying Peng, Yingying Xu, Lanfen Lin, Ruo-feng Tong, Zhiyi Peng, Xionngwei Mao, Hongjie Hu, Yen-Wei Chen, Jingsong Li
  • Journal Title: IEEE Journal of Biomedical and Health Informatics Volume: Vol.26, Pages: 614-625
  • DOI: 10.1109/JBHI.2021.3091900
  • Peer Reviewed / Int'l Joint Research
• [Journal Article] Phase Attention Model for Prediction of Early Recurrence of Hepatocellular Carcinoma With Multi-Phase CT Images and Clinical Data (2022)
  • Author(s): Wang Weibin、Wang Fang、Chen Qingqing、Ouyang Shuyi、Iwamoto Yutaro、Han Xianhua、Lin Lanfen、Hu Hongjie、Tong Ruofeng、Chen Yen-Wei
  • Journal Title: Frontiers in Radiology Volume: 2 Pages: -
  • DOI: 10.3389/fradi.2022.856460
  • Peer Reviewed / Int'l Joint Research
• [Journal Article] Characterisation of focal liver lesions in multi-phase CT images using textural and pathological descriptors (2022)
  • Author(s): Moslehi Saeed、Foruzan Amir Hossein、Chen Yen-Wei、Hu Hongjie
  • Journal Title: Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization Volume: - Pages: 1～9
  • DOI: 10.1080/21681163.2022.2156390
  • Peer Reviewed / Int'l Joint Research
• [Journal Article] PA-ResSeg: A Phase Attention Residual Network for Liver Tumor Segmentation from Multi-phase CT Images (2021)
  • Author(s): Yingying Xu, Ming Cai, Lanfen Lin, Yue Zhang, Hongjie Hu, Zhiyi Peng, Qiaowei Zhang, Qingqing Chen, Xiongwei Mao, Yutaro Iwamoto, Xian-Hua Han, Yen-Wei Chen, Ruofeng Tong
  • Journal Title: Medical Physics Volume: Vol.48, Pages: 3752-3766
  • DOI: 10.1002/mp.14922
  • Peer Reviewed / Int'l Joint Research
• [Journal Article] VolumeNet: A Lightweight Parallel Network for Super-Resolution of Medical Volumetric Data (2021)
  • Author(s): Yinhao Li, Yutaro Iwamoto, Lanfen Lin, Rui Xu, Ruofen Tong and Yen-Wei Chen
  • Journal Title: IEEE Trans. Image Processing Volume: Vol. 30 Pages: 4840-4854
  • DOI: 10.1109/TIP.2021.3076285
  • Peer Reviewed / Int'l Joint Research
• [Journal Article] Medical Image Segmentation with Deep Atlas Prior (2021)
  • Author(s): Huiminng Huang, Han Zheng, Lanfen Lin, Ming Cai, Hongjie Hu, Qiaowei Zhang, Qingqing Chen, Yutaro Iwamoto, Xianhua Han, Yen-Wei Chen and Ruofeng Tong
  • Journal Title: IEEE Trans. Medical Imaging Volume: Vol.40, Pages: 3519-3530
  • DOI: 10.1109/TMI.2021.3089661
  • Peer Reviewed / Int'l Joint Research
• [Journal Article] Mutual Information-Based Graph Co-Attention Networks for Multimodal Prior-Guided Magnetic Resonance Imaging Segmentation (2021)
  • Author(s): Shaocong Mo, Ming Cai, Lanfen Lin, Ruofeng Tong, Qingqing Chen, Fang Wang, Hongjie Hu, Yutaro Iwamoto, Xian-Hua Han, and Yen-Wei Chen
  • Journal Title: IEEE Trans. Circuits and Systems for Video Technology Volume: Vol.32, Pages: 2512-2526
  • DOI: 10.1109/TCSVT.2021.3112551
  • Peer Reviewed / Int'l Joint Research
• [Presentation] AIと医用画像解析 (2022)
  • Author(s): 陳延偉
  • Organizer: 第22回日本術中画像情報学会（特別講演）
  • Invited
• [Presentation] ClassFormer: Exploring Class-aware Dependency with Transformer for Medical Image Segmentation (2022)
  • Author(s): Huimin Huang, Shiao Xie, Lanfen Lin, Ruofeng Tong, Yen-Wei Chen, Hong Wang, Yuexiang Li, Yawen Huang, Yefeng Zheng
  • Organizer: 2022 Association for the Advancement of Artificial Intelligence(AAAI 2022)
  • Int'l Joint Research
• [Presentation] ScaleFormer: Revisiting the Transformer-based Backbones from a Scale-wise Perspective for Medical Image Segmentation (2022)
  • Author(s): Huimin Huang, Shiao Xie, Lanfen Lin, Yutaro Iwamoto, Xian-Hua Han, Yen-Wei Chen and Ruofeng Tong
  • Organizer: International Joint Conference on Artificial Intelligence (IJCAI2022)
  • Int'l Joint Research
• [Presentation] An Accurate Unsupervised Liver Lesion Detection Method Using Pseudo-lesions (2022)
  • Author(s): He Li, Yutaro Iwamoto, Xianhua Han, Lanfen Lin, Hongjie Hu and Yen-Wei Chen
  • Organizer: International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2022)
  • Int'l Joint Research
• [Presentation] Unsupervised Domain Adaptation Using Adversarial Learning and Maximum Square Loss for Liver Tumors Detection in Multi-phase CT Images (2022)
  • Author(s): Rahul Kumar JAIN, Takahiro SATO, Taro WATASUE, Tomohiro NAKAGAWA, Yutaro IWAMOTO, Xianhua HAN, Lanfen LIN, Hongjie HU Xiang RUAN and Yen-Wei CHEN
  • Organizer: 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (IEEE EMBC 2022)
  • Int'l Joint Research
• [Presentation] Synthesizing Contrast-Enhanced Computed Tomography Images with an Improved Conditional Generative Adversarial Network (2022)
  • Author(s): Yulin Yang, Caie Xu, Qingqing Chen, Yutaro Iwamoto, Xian-Hua Han, Lanfen Lin, Hongjie Hu, Ruofeng Tong, and Yen-Wei Chen
  • Organizer: 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (IEEE EMBC 2022)
  • Int'l Joint Research
• [Presentation] Residual Multilayer Perceptrons for Genotype-Guided Recurrence Prediction of Non-Small Cell Lung Cancer (2022)
  • Author(s): Yang Ai, Panyanat Aonpong, Weibin Wang, Yinhao Li, Yutaro Iwamoto, Xianhua Han, Yen- Wei Chen
  • Organizer: 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (IEEE EMBC 2022)
  • Int'l Joint Research
• [Presentation] Dual Discriminator-Based Unsupervised Domain Adaptation Using Adversarial Learning for Liver Segmentation on Multiphase CT Images (2022)
  • Author(s): Swathi Ananda, Yutaro Iwamoto, Xianhua HAN, Lanfen Lin, Hongjie Hu and Yen-Wei Chen
  • Organizer: 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (IEEE EMBC 2022)
  • Int'l Joint Research
• [Presentation] Mixed Transformer U-Net for Medical Image Segmentation (2022)
  • Author(s): Hongyi Wang, Shiao Xie, Lanfen Lin, Yutaro Iwamoto, Xian-Hua Han, Yen-Wei Chen, Ruofeng Tong
  • Organizer: 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (IEEE ICASSP2022)
  • Int'l Joint Research
• [Presentation] Deep Learning-based Risk Prediction Model for Recurrence-free Survival in Patients with Hepatocellular Carcinoma Using Multi-phase CT Images (2022)
  • Author(s): Weibin Wang, Fang Wang, YunjunYang, YinhaoLi, Jing Liu, Xianhua Han, Lanfen Lin, Ruofeng Tong, Hongjie Hu, Yen-Wei Chen
  • Organizer: IEEE Global Conference on Consumer Electronics (GCCE2022)
  • Int'l Joint Research
• [Presentation] Improved Mask R-CNN with Deformable Convolutions for Accurate Liver Lesion Detection in Multiphase CT Images (2022)
  • Author(s): Chanyu LEE, Yutaro IWAMOTO, Lanfen LIN, Hongjie HU, and Yen-Wei CHEN
  • Organizer: International Conference on Innovation in Medicine and Healthcare (InMed-2022)
  • Int'l Joint Research
• [Presentation] Weakly-supervised and Semi-supervised Deep Learning in Medical Imaging (2021)
  • Author(s): Yen-Wei Chen
  • Organizer: 2021 4th International Conference on Digital Medicine and Image Processing (DMIP2021)
  • Int'l Joint Research / Invited
• [Presentation] Deep Learning for Computer-aided Diagnosis of COVID-19 (2021)
  • Author(s): Yen-Wei Chen
  • Organizer: The 28th International Display Workshops (IDW2021)
  • Int'l Joint Research / Invited
• [Presentation] A Tensor Sparse Representation-Based CBMIR System for Computer-Aided Diagnosis of Focal Liver Lesions and its Pilot Trial (2021)
  • Author(s): Jian Wang, Xian-Hua Han, Lanfen Lin, Hongjie Hu, Yen-Wei Chen
  • Organizer: ACM International Conference on Multimedia Retrieval 2021 (ICMR 2021)
  • Int'l Joint Research
• [Presentation] M-DFNet: Multi-phase Discriminative Feature Network for Retrieval of Focal Liver Lesions (2021)
  • Author(s): Yingying Xu, Jing Liu, Lanfen Lin, Hongjie Hu, Ruofeng Tong, Jingsong Li, Yen-Wei Chen
  • Organizer: ACM International Conference on Multimedia Retrieval 2021 (ICMR 2021)
  • Int'l Joint Research
• [Presentation] Automatic Detection and Segmentation of Liver Tumors in Multi-Phase CT Images by Phase Attention Mask R-CNN (2021)
  • Author(s): Ryo Hasegawa, Yutaro IWAMOTO, Xianhua HAN, Lanfen LIN, Hongjie HU, Xiujun CAI, and Yen-Wei CHEN
  • Organizer: 9th IEEE International Conference on Consumer Electronics (IEEE ICCE2021)
  • Int'l Joint Research
• [Presentation] Graph-BAS3Net: Boundary-Aware Semi-Supervised Segmentation Network with Bilateral Graph Convolution (2021)
  • Author(s): Huiming Huang, Lanfen Lin, Yue Zheng, Yingying Xu, Jing Zheng, Xiongwei Mao, Xiaohan Qian, Zhiyi Peng, Jianying Zhou, Yen-Wei Chen, Ruofeng Tong
  • Organizer: International Conference on Computer Vision (ICCV2021)
  • Int'l Joint Research
• [Presentation] Patch-Free 3D Medical Image Segmentation Driven by Super-Resolution Technique and Self-Supervised Guidance (2021)
  • Author(s): Hongyi Wang, Lanfen Lin, Hongjie Hu, Qingqing Chen, Yinhao Li, Yutaro Iwamoto, Xian-Hua Han, Yen-Wei Chen, and Ruofeng Tong
  • Organizer: International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2021)
  • Int'l Joint Research
• [Presentation] 3D Graph-S2Net: Shape-Aware Self-Ensembling Network for Semi-Supervised Segmentation with Bilateral Graph Convolution (2021)
  • Author(s): Huimin Huang, Nan Zhou, Lanfen Lin, Hongjie Hu, Yutaro Iwamoto, Han, Yen-Wei Chen, Ruofeng Tong
  • Organizer: International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2021)
  • Int'l Joint Research
• [Presentation] Multi-phase Liver Tumor Segmentation with Spatial Aggregation and Uncertain Region Inpainting (2021)
  • Author(s): Yue Zhang, Chengtao Peng, Liying Peng, Huimin Huang, Ruofeng Tong, Lanfen Lin, Jingsong Li, Yen-Wei Chen, Qingqing Chen, Hongjie Hu, and Zhiyi Peng
  • Organizer: International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2021)
  • Int'l Joint Research
• [Book] Innovation in Medicine and Healthcare (2022)
  • Author(s): Yen-Wei Chen, Satoshi Tanaka, Robert I. Howlett and Lakhmi C. Jain
  • Total Pages: 314
  • Publisher: Springer
  • ISBN: 978-981-19-3440-7
• [Book] Handbook of Artificial Intelligence in Healthcare Vol 2: Practicalities and Prospects (2022)
  • Author(s): Chee-Peng Lim, Yen-Wei Chen, Ashlesha Vaidya, Charu Mahorkar and Lahmi C. Jain
  • Total Pages: 422
  • Publisher: Springer
  • ISBN: 978-3-030-83619-1
• [Book] Advances in Machine Learning for Big Data Analysis (2022)
  • Author(s): Satchidananda Dehuri, Yen-Wei Chen
  • Total Pages: 239
  • Publisher: Springer
  • ISBN: 978-981-16-8930-7