Removing the Burden of Data Labeling: Automatic Surgical Video Understanding with Unsupervised Learning

2021 Fiscal Year Final Research Report

• Project/Area Number: 20K23343
• Research Category: Grant-in-Aid for Research Activity Start-up
• Allocation Type: Multi-year Fund
• Review Section: 1002:Human informatics, applied informatics and related fields
• Research Institution: Osaka University
• Principal Investigator: LI LIANGZHI 大阪大学, データビリティフロンティア機構, 特任助教(常勤) (10875545)
• Project Period (FY): 2020-09-11 – 2022-03-31
• Keywords: Few-shot Learning / Semantic Segmentation / Video Understanding / Medical Images / Computer Vision / Surgical Analysis / Deep Learning

Outline of Final Research Achievements

As most of the existing automatic surgical video analysis models require a large number of manually labeled data for training, this project aims to design a learning method to perform spatial and temporal segmentations with smaller requirements of humans’ input. During this project, I mainly studied the following sub-topics towards the goal.
1.Surgical images/frames semantic segmentation in a weakly-supervised way. I developed a new training strategy for video semantic segmentation models to utilized unlabeled data to improve their segmentation performance.
2.Surgical videos temporal analysis using no labels. I developed a retrieval-based method to automatically predict surgical duration.

Free Research Field

Computer Vision

Academic Significance and Societal Importance of the Research Achievements

The establishment of techniques capable of understanding what, where, and when is happening with no requirements on human efforts will ease the task of surgery indexing, clinical training, etc.