Abdominal Laparoscopic Surgery Support System Using Projection-based Augmented Reality

2022 Fiscal Year Final Research Report

• Project/Area Number: 21K21304
• Research Category: Grant-in-Aid for Research Activity Start-up
• Allocation Type: Multi-year Fund
• Review Section: 1002:Human informatics, applied informatics and related fields
• Research Institution: University of Tsukuba
• Principal Investigator: Xie Chun 筑波大学, 計算科学研究センター, 研究員 (00913287)
• Project Period (FY): 2021-08-30 – 2023-03-31
• Keywords: 拡張現実 / 手術支援 / 画像処理 / 三次元センシング

Outline of Final Research Achievements

This research has developed a system that tracks laparoscopic scopes and surgical instruments, estimates the angles and depths of instruments inserted into the abdomen that are not visible from the laparoscope's imaging area, and displays them on the body surface with SAR(spatial augmented reality). By using three-dimensional shapes and projector positions, we can correct projection distortions and provide surgeons with geometrically consistent images. We have also proposed a method for constructing three-dimensional models of bones using widely used and relatively inexpensive C-Arm fluoroscopy equipment. Furthermore, we have proposed a surgical support system that records surgical procedures using a multi-view (bullet-time) filming system and switches to the optimal viewpoint as needed.

Free Research Field

拡張現実

Academic Significance and Societal Importance of the Research Achievements

本研究が構築したシステムはSARによる腹腔鏡手術支援技術やアプリケーションの開発、評価の基盤となり、医療メディア分野において新たな知見の創出を導く。また、腹腔鏡手術のみならず、他の内視鏡カメラを経由して術野を観察する手術への応用も期待される。また、提案した多視点映像システムは若手医師の育成効率の向上に有用である。さらに、C-Arm透視装置を使用した３次元復元方法は、手術中に骨の３次元モデルを生成し、術前計画との比較により迅速の手術評価が可能となり、医療効率の向上に貢献した。