Development of computer vision and assessment system using deep learning for laparoscopic rectal surgery

2021 Fiscal Year Final Research Report

• Project/Area Number: 18K16378
• Research Category: Grant-in-Aid for Early-Career Scientists
• Allocation Type: Multi-year Fund
• Review Section: Basic Section 55020:Digestive surgery-related
• Research Institution: National Cancer Center Japan
• Principal Investigator: Takeshita Nobuyoshi 国立研究開発法人国立がん研究センター, 東病院, 室長 (40645610)
• Project Period (FY): 2018-04-01 – 2022-03-31
• Keywords: 直腸癌 / 内視鏡外科 / 深層学習

Outline of Final Research Achievements

In recent years, new innovative surgical techniques have begun to be introduced in the field of rectal cancer surgery which enable further function preservation and recognition of detailed anatomical structures. On the other hand, there are many issues to be solved in standardizing increasingly difficult surgical techniques, such as the transfer of skills and knowledge, training, and evaluation. In this study, we digitized and extracted the procedures performed in a video image of a specular rectal cancer surgery, and constructed an image recognition and analysis system for surgical instruments, anatomical structures, and surgical processes using deep learning, especially a convolutional neural network. We constructed a system that recognizes and analyzes images of anatomical structures and surgical processes using convolutional neural networks. We were also able to develop an objective surgical evaluation system that incorporates tacit knowledge of skilled operators.

Free Research Field

内視鏡外科

Academic Significance and Societal Importance of the Research Achievements

本研究を通じて、畳み込みニューラルネットワーク（CNN）を用いた鏡視下直腸切除術における手術工程の自動認識の深層学習モデル、術中解剖構造検出としての前立腺認識モデル、さらには動画を用いた手術技能評価用の深層学習モデルの構築を達成した。これらは外科医の手術中の判断や技術の巧拙という暗黙知を定量化・客観化することに繋がり、手術支援による安全性向上や、手術教育への活用、最終的には外科手術の自動化をもたらすための要素技術となりうる。