• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Basic Study on Real-time Recognition of Surgical Procedures by Surgery-assisting Robot Using Deep Learning

Research Project

Project/Area Number 18K11493
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeMulti-year Fund
Section一般
Review Section Basic Section 61050:Intelligent robotics-related
Research InstitutionTokyo Denki University

Principal Investigator

Miyawaki Fujio  東京電機大学, 理工学部, 教授 (50174222)

Co-Investigator(Kenkyū-buntansha) 日高 章理  東京電機大学, 理工学部, 准教授 (70553519)
Project Period (FY) 2018-04-01 – 2022-03-31
Project Status Completed (Fiscal Year 2021)
Budget Amount *help
¥4,290,000 (Direct Cost: ¥3,300,000、Indirect Cost: ¥990,000)
Fiscal Year 2020: ¥1,040,000 (Direct Cost: ¥800,000、Indirect Cost: ¥240,000)
Fiscal Year 2019: ¥1,040,000 (Direct Cost: ¥800,000、Indirect Cost: ¥240,000)
Fiscal Year 2018: ¥2,210,000 (Direct Cost: ¥1,700,000、Indirect Cost: ¥510,000)
Keywords外科手術工程リアルタイム認識 / 深層学習 / 深層ニューラルネットワーク / 執刀医動作特徴 / 術野内視鏡動画 / 模擬的腹腔鏡下胆嚢摘出術 / 器械出し看護師ロボット / 手術支援 / 手術支援ロボット / 模擬胆嚢摘出術 / 手術視覚特徴 / 術者動作特徴 / 術者動作動画 / 深層ニューラルネットワーク学習 / 状態遷移認識 / 外科手術工程 / 手術手順モデル
Outline of Final Research Achievements

The entire process of surgery can be shown as a sequence of plural surgical procedures such as dissection, hemostasis, ligation, division and others. No simple sensor, however, can detect any transition from one procedure to another. Then, we investigated whether each transition could be identified by deep learning. A main achievement is as follows. By using only operating surgeons’ motions as input data, the identification rates were 76.7% for dissection, 88.5% for ligation, 52.5% for division and 59.9% for hemostasis (the class average rate was 69.4%). These results were obtained from many one-second videos which were made by dividing original long videos at an interval of one second, thereby suggesting that a surgery-assisting robot may detect each transition of surgical procedures after a delay of one second.

Academic Significance and Societal Importance of the Research Achievements

本研究は外科手術を対象としているが、最終的には一般的な出来事にも拡張させることを目標としている。現実の出来事は時々刻々連続的に変化し切れ目が無いが,これを人間は理解・記憶し易いように意図的に区切っている。
例えば、外科手術でも執刀医の同じような連続的動作が繰り返し観察されるため,馴染みのない者は外科手術の工程が理解できない。しかし,連続的動作の何処から何処までは‘剥離’という目的で行われた連続的動作と分かれば理解・記憶できるようになる。この人間の意図的な区切りが深層学習によって同定できれば、人間の物事に対する見方をロボットにも学ばせることが可能となる。これが本研究の意義である。

Report

(5 results)
  • 2021 Annual Research Report   Final Research Report ( PDF )
  • 2020 Research-status Report
  • 2019 Research-status Report
  • 2018 Research-status Report
  • Research Products

    (14 results)

All 2022 2021 2020 2019 Other

All Int'l Joint Research (3 results) Journal Article (4 results) (of which Int'l Joint Research: 1 results,  Peer Reviewed: 3 results) Presentation (6 results) (of which Int'l Joint Research: 2 results) Remarks (1 results)

  • [Int'l Joint Research] タリン工科大学(エストニア)

    • Related Report
      2021 Annual Research Report
  • [Int'l Joint Research] Tallinn University of Technology(エストニア)

    • Related Report
      2020 Research-status Report
  • [Int'l Joint Research] タリン工科大学(エストニア)

    • Related Report
      2019 Research-status Report
  • [Journal Article] Improving Accuracy and Real-Time Performance of Recognition Methods for Surgical Procedure Recognition2022

    • Author(s)
      Nozomu Suzuki, Fujio Miyawaki and Akinori Hidaka
    • Journal Title

      Proceedings of the 28th International Workshop on Frontiers of Computer Vision (IW-FCV2022)

      Volume: ー Pages: 572-586

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Convolutional Neural Network based on Temporal Pose Features for Surgical Procedure Recognition2021

    • Author(s)
      Shota Kishi, Nozomu Suzuki, Shota Tsuyuki, Takio Kurita, Fujio Miyawaki, Akinori Hidaka
    • Journal Title

      Proceedings of the 52nd ISCIE International Symposium on Stochastic Systems Theory and Its Applications

      Volume: - Pages: 60-64

    • NAID

      130008150678

    • Related Report
      2020 Research-status Report
    • Peer Reviewed
  • [Journal Article] 深層学習による外科手術工程のリアルタイム認識に関する基礎的研究2020

    • Author(s)
      宮脇 富士夫
    • Journal Title

      Medical Science Digest

      Volume: 46(5) Pages: 45-47

    • Related Report
      2019 Research-status Report
  • [Journal Article] Integration of 'Central' and 'Peripheral' Nervous Systems in Scrub Nurse Robot and Comparison Between 'Voluntary' and 'Reflex' Movements2019

    • Author(s)
      Fujio Miyawaki, Miho Suzuki, Aivo Anier, Juri Vain
    • Journal Title

      14th IFAC Symposium on Analysis Design and Evaluation of Human Machine Systems

      Volume: - Pages: 229-234

    • Related Report
      2019 Research-status Report
    • Peer Reviewed / Int'l Joint Research
  • [Presentation] 深層学習による外科手術中の出血点発見に関する基礎的研究2021

    • Author(s)
      1)石黑達也,日高章理,宮脇富士夫
    • Organizer
      第30回ライフサポート学会フロンティア講演会
    • Related Report
      2020 Research-status Report
  • [Presentation] Convolutional Neural Network based on Temporal Pose Features for Surgical Procedure Recognition2020

    • Author(s)
      Shota Kishi, Nozomu Suzuki, Shota Tsuyuki, Takio Kurita, Fujio Miyawaki, Akinori Hidaka
    • Organizer
      The 52nd ISCIE International Symposium on Stochastic Systems Theory and Its Applications
    • Related Report
      2020 Research-status Report
    • Int'l Joint Research
  • [Presentation] 深層学習による外科手術の状況認識に関する基礎的検討2020

    • Author(s)
      宮崎峻伎,日高章理,宮脇富士夫
    • Organizer
      第29回ライフサポート学会フロンティア講演会
    • Related Report
      2019 Research-status Report
  • [Presentation] Integration of 'Central' and 'Peripheral' Nervous Systems in Scrub Nurse Robot and Comparison Between 'Voluntary' and 'Reflex' Movements2019

    • Author(s)
      Fujio Miyawaki, Miho Suzuki, Aivo Anier, Juri Vain
    • Organizer
      IFAC HMS 2019
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] 深層学習による動作認識手法を用いた手術状況理解技術の開発2019

    • Author(s)
      榎莉奈,齋藤啓太,堀部祐介,宮脇富士夫,日高章理
    • Organizer
      第51回測自動制御学会北海道支部学術講演会
    • Related Report
      2018 Research-status Report
  • [Presentation] 器械出しの自動化のためのCNN を用いた手術状況の識別2019

    • Author(s)
      齋藤啓太,榎莉奈,堀部祐介,宮脇富士夫,日高章理
    • Organizer
      第51回測自動制御学会北海道支部学術講演会
    • Related Report
      2018 Research-status Report
  • [Remarks] 東京電機大学 理工学部 臨床医用科学研究室

    • URL

      http://www2.miyawaki-lab.com

    • Related Report
      2020 Research-status Report

URL: 

Published: 2018-04-23   Modified: 2023-01-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi