Endoscopic Image Mapping System for Fetus Surgery

• Project/Area Number: 25350561
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Research Field: Medical systems
• Research Institution: The University of Tokyo
• Principal Investigator: Kobayashi Etsuko 東京大学, 工学(系)研究科(研究院), 准教授 (20345268)
• Co-Investigator(Kenkyū-buntansha): CHIBA Toshio 日本大学, 総合科学研究所, 教授 (20171944)
• Co-Investigator(Renkei-kenkyūsha): SAKUMA Ichiro 東京大学, 大学院工学系研究科, 教授 (50178597)
• Project Period (FY): 2013-04-01 – 2016-03-31
• Project Status: Completed (Fiscal Year 2015)
• Budget Amount: ¥4,940,000 (Direct Cost: ¥3,800,000、Indirect Cost: ¥1,140,000); Fiscal Year 2015: ¥1,560,000 (Direct Cost: ¥1,200,000、Indirect Cost: ¥360,000); Fiscal Year 2014: ¥1,690,000 (Direct Cost: ¥1,300,000、Indirect Cost: ¥390,000); Fiscal Year 2013: ¥1,690,000 (Direct Cost: ¥1,300,000、Indirect Cost: ¥390,000)
• Keywords: 低侵襲治療 / コンピュータ外科 / 内視鏡画像処理 / 低侵襲治療システム / 内視鏡画像マッピング

Outline of Final Research Achievements

We have developed a mapping system of endoscopic views on a 3D placenta model constructed from ultrasound images for Fetus surgery. The system does not require external trackers. The relative position between the endoscope and the 3D model was measured by 3D ultrasound and moving distance of endoscope during surgery was estimated by feature matching of endoscopic image. Using these relative position information and moving information endoscopic views were overlaid onto the surface of the model. This addresses operational challenges including the limited field-of-view and the lack of 3D perspective associated with minimally invasive procedure. Experimental results show that The R.M.S. error of the image mapping of 20 frames was 1.1 mm. The feasibility of the framework is also demonstrated through. We have also developed contrast enhance system between vasculature and placenta by using narrow band images.

Research Products

• [Journal Article] Self-contained image mapping of placental vasculature in 3D ultrasound-guided fetoscopy (2015)
  • Author(s): Liangjing Yang, Junchen Wang, Takehiro Ando, Akihiro Kubota, Hiromasa Yamashita, Ichiro Sakuma, Toshio Chiba, and Etsuko Kobayashi
  • Journal Title: Surgical Endoscopy Volume: - Issue: 9 Pages: 1-14
  • DOI: 10.1007/s00464-015-4690-z
  • Peer Reviewed / Acknowledgement Compliant
• [Journal Article] Towards scene adaptive image correspondence for placental vasculature mosaic in computer assisted fetoscopic procedures (2015)
  • Author(s): L. Yang, J. Wang, T. Ando, A. Kubota, H. Yamashita, I. Sakuma, T. Chiba, and E. Kobayashi
  • Journal Title: The International Journal of Medical Robotics and Computer Assisted Surgery Volume: - Issue: 3 Pages: 375-386
  • DOI: 10.1002/rcs.1700
  • Peer Reviewed / Acknowledgement Compliant
• [Journal Article] Vision-based endoscope tracking for 3D ultrasound image-guided surgical navigation (2015)
  • Author(s): L. Yang, J. Wang, T. Ando, H. Yamashita, I. Sakuma, T. Chiba, E. Kobayashi.
  • Journal Title: Computerized Medical Imaging and Graphics Volume: 40 Pages: 205-216
  • DOI: 10.1016/j.compmedimag.2014.09.003
  • Peer Reviewed
• [Journal Article] Image mapping of untracked free-hand endoscopic views to an ultrasound image-constructed 3D placenta model (2014)
  • Author(s): L. Yang, J. Wang, E. Kobayashi, T. Ando, H. Yamashita, I. Sakuma, T. Chiba
  • Journal Title: the International Journal of Medical Robotics and Computer Assisted Surgery Volume: online Issue: 2 Pages: 223-234
  • DOI: 10.1002/rcs.1592
  • Peer Reviewed / Open Access / Acknowledgement Compliant
• [Journal Article] Image mapping of untracked free-hand endoscopic views to an ultrasound image-constructed 3D placenta model (2014)
  • Author(s): L. Yang, J. Wang, E. Kobayashi, T. Ando, H. Yamashita, I. Sakuma, T. Chiba
  • Journal Title: International Journal of Medical Robotic and Computer Assisted Surgery Volume: 未定
  • Peer Reviewed
• [Presentation] 制限された視野を拡張する内視鏡下手術支援ロボットシステム (2016)
  • Author(s): 小林英津子
  • Organizer: 医工学治療学会第32回学術大会
  • Place of Presentation: 甲府富士屋ホテル（山梨県・甲府市）
  • Year and Date: 2016-03-19
  • Invited
• [Presentation] 双胎間輸血症候群手術支援のための狭帯域光画像を使った血管強調と画像マッピング (2014)
  • Author(s): 窪田 章宏，楊 量景，王 君臣，安藤 岳洋，山下 紘正，千葉 敏雄，佐久間 一郎，小林 英津子
  • Organizer: 日本生体医工学会関東支部若手研究者発表会2014
  • Place of Presentation: 東京農工大学（東京都小金井市）
  • Year and Date: 2014-11-22
• [Presentation] 狭帯域画像を使った双胎間輸血症候群手術のための画像マッピング (2014)
  • Author(s): 窪田 章宏，楊 量景，王 君臣，安藤 岳洋，山下 紘正，千葉 敏雄，佐久間 一郎，小林 英津子
  • Organizer: 第23回日本コンピュータ外科学会
  • Place of Presentation: 大阪大学（大阪府吹田市）
  • Year and Date: 2014-11-08 – 2014-11-09
• [Presentation] Workflow-centric design of 3D navigational cues for photocoagulation treatment in twin-to-twin transfusion syndrome (2014)
  • Author(s): L. Yang, J. Wang, T. Ando, A. Kubota, H. Yamashita, T. Chiba, I. Sakuma,E. Kobayashi
  • Organizer: LIFE 2014
  • Place of Presentation: ルスツリゾート（北海道虻田郡留寿都村）
  • Year and Date: 2014-09-24 – 2014-09-26
• [Presentation] Contrast enhancement between vasculature and placenta using narrow band images for TTTS surgery (2014)
  • Author(s): A. Kubota, L. Yang, J. Wang, T. Ando, I. Sakuma, E. Kobayashi,H. Yamashita, T. Chiba
  • Organizer: Computer Assisted Radiology and Surgery2014
  • Place of Presentation: 福岡国際会議場（福岡県福岡市）
  • Year and Date: 2014-06-25 – 2014-06-28
• [Presentation] Ultrasound Image-Guided Mapping of Endoscopic Views on a 3D Placenta Model: A Tracker-Less Approach (2013)
  • Author(s): L. Yang, J. Wang, E. Kobayashi, H. Liao, I. Sakuma, H. Yamashita, T. Chiba
  • Organizer: ” in 6th International Workshop on Medical Imaging and Augmented Reality
  • Place of Presentation: 名古屋大学、名古屋
• [Presentation] Ultrasound Image-based Endoscope Localization for Minimally Invasive Fetoscopic Surgery (2013)
  • Author(s): L. Yang, J. Wang, E. Kobayashi, H. Liao, H. Yamashita, I. Sakuma, T. Chiba
  • Organizer: 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society
  • Place of Presentation: 大阪国際会議場、大阪
• [Presentation] An intraoperative framework for mapping of untracked endoscopic vision to 3D ultrasound placenta model (2013)
  • Author(s): L. Yang, J. Wang, E. Kobayashi, H. Liao, H. Yamashita, I. Sakuma, T. Chiba
  • Organizer: 27th International Congress and Exhibition of Computer Assisted Radiology and Surgery
  • Place of Presentation: Heidelberg Convention Center, Heidelberg, Germany,
• [Presentation] Self-Registration of Ultrasound Imaging Device to Navigation System using Surgical Instrument Kinematics in Minimally Invasive Procedure (2013)
  • Author(s): L. Yang, J. Wang, E. Kobayashi, H. Liao, I. Sakuma, H. Yamashita, T. Chiba
  • Organizer: The 9th Asian Conference in Computer Aided Surgery
  • Place of Presentation: 早稲田大学、東京