Dynamics analysis of musculoskeletal and organ systems during exercise by body-mounted robotic monitoring device

• Project/Area Number: 19K21941
• Research Category: Grant-in-Aid for Challenging Research (Exploratory)
• Allocation Type: Multi-year Fund
• Review Section: Medium-sized Section 20:Mechanical dynamics, robotics, and related fields
• Research Institution: The University of Electro-Communications
• Principal Investigator: Koizumi Norihiro 電気通信大学, 大学院情報理工学研究科, 准教授 (10396765)
• Co-Investigator(Kenkyū-buntansha): 岡田 英孝 電気通信大学, 大学院情報理工学研究科, 教授 (20303018) ; 東郷 俊太 電気通信大学, 大学院情報理工学研究科, 准教授 (30751523)
• Project Period (FY): 2019-06-28 – 2022-03-31
• Project Status: Completed (Fiscal Year 2021)
• Budget Amount: ¥6,370,000 (Direct Cost: ¥4,900,000、Indirect Cost: ¥1,470,000); Fiscal Year 2020: ¥2,210,000 (Direct Cost: ¥1,700,000、Indirect Cost: ¥510,000); Fiscal Year 2019: ¥4,160,000 (Direct Cost: ¥3,200,000、Indirect Cost: ¥960,000)
• Keywords: 医療技能のデジタル化 / 医デジ化 / Me-DigIT / Medical Digitalization / 超音波ロボット / Robotic Ultrasound / 体内モーションキャプチャ / 体外モーションキャプチャ / ロボティック超音波診断・治療 / 体動補償 / 医療・バイオ用ロボットビジョン技術 / モーション・キャプチャ / 診断画像適正度評価 / 医療のデジタル化 / モーションキャプチャ / 体内臓器系モーションキャプチャ / 運動モニタリング / Robotic ultrasound

Outline of Research at the Start

本研究では超音波技術，人工知能技術，ならびにロボット技術を基盤として非侵襲的な身体装着型のモニタリング装置を新規に開発，これを用いて把持・歩行運動中の体内臓器の挙動を世界で初めて観察可能とすることで人体の骨格および内臓・筋肉の統合的な動力学的解析・理解を促進することを目的とする。
近年，光学あるいは慣性センサを用いたモーションキャプチャの技術により運動中の筋・骨格系の動力学の理解・解析が可能になりつつある。本研究では上記に加えて運動する体内臓器の挙動をも統合的に観察可能とするツールおよびこれに基づく人体の骨格および内臓・筋肉の統合的な動力学的解析手法についての研究を行なう。

Outline of Final Research Achievements

Extracting / tracking and following / monitoring technology for internal organs that move with deformation while interacting with each other by breathing, technology for linking / integrating information from external and internal motion capture, organs are displaced / deformed / rotated according to breathing It is possible to monitor the dynamics of organs by synthesizing time-series ultrasound images even if the organs that move in the body are partially deleted on the ultrasound image due to the case of swelling or shielding such as ribs. There have been remarkable achievements in the development of the algorithm technology, the diagnostic image optimization algorithm technology by the above-mentioned robotic avoidance operation of shielding, and the algorithm technology for evaluating the appropriateness of the acquired image from the contact state of the probe and the contour of the organ.

Academic Significance and Societal Importance of the Research Achievements

本研究により、これまで不可能であった人体の筋骨格系および体内臓器系の統一・統合的な動力学的解析・理解を可能とする、全く新しいロボット運動学の分野を開拓することが可能となり、人体の鏡像としてのロボット学の基盤になりうるものと期待している。

Research Products

• [Journal Article] A novel complementation method of an acoustic shadow region utilizing a convolutional neural network for ultrasound-guided therapy (2022)
  • Author(s): Momoko Matsuyama, Norihiro Koizumi, Akihide Otsuka, Kento Kobayashi, Shiho Yagasaki, Yusuke Watanabe, Jiayi Zhou, Yu Nishiyama, Naoki Matsumoto, Hiroyuki Tsukihara, Kazushi Numata
  • Journal Title: International Journal of Computer Assisted Radiology and Surgery (IJCARS) Volume: 17 Issue: 1 Pages: 107-119
  • DOI: 10.1007/s11548-021-02525-8
  • Peer Reviewed
• [Journal Article] Evaluation of Ultrasonic Fibrosis Diagnostic System Using Convolutional Network for Ordinal Regression (2021)
  • Author(s): Ryosuke Saito, Norihiro Koizumi, Yu Nishiyama, Tsubasa Imaizumi, Kenta Kusahara, Shiho Yagasaki, Naoki Matsumoto, Ryota Masuzaki, Toshimi Takahashi, Masahiro Ogawa
  • Journal Title: International Journal of Computer Assisted Radiology and Surgery (IJCARS) Volume: 16 Issue: 11 Pages: 1969-1975
  • DOI: 10.1007/s11548-021-02491-1
  • Peer Reviewed
• [Journal Article] Parametric identification of prostate shape using the superellipses and its correlation with pathology (2021)
  • Author(s): Yusuke Onodera, Norihiro Koizumi, Yuka Shigenari, Riki Igarashi
  • Journal Title: International Journal of Computer Assisted Radiology and Surgery (IJCARS) Volume: 16 Issue: S1 Pages: 1-119
  • DOI: 10.1007/s11548-021-02375-4
  • Peer Reviewed
• [Journal Article] Correlation between alterations in blood flow of malignant lymphomas after induction chemotherapies and clinical outcomes: a pilot study utilising contrast-enhanced ultrasonography for early interim evaluation of lymphoma treatment (2021)
  • Author(s): Mariko Kumagawa, Naoki Matsumoto, Katsuhiko Miura, Masahiro Ogawa, Hiromichi Takahashi, Yoshihiro Hatta, Ryosuke Kondo, Norihiro Koizumi, Masami Takei, Mitsuhiko Moriyama
  • Journal Title: Clinical Radiology Volume: 2021 Issue: 7 Pages: 1-9
  • DOI: 10.1016/j.crad.2021.02.007
  • Peer Reviewed
• [Journal Article] The value of hepatobiliary phase in EOB-MRI in predicting hypervascularization outcome of non-hypervascular hypointense lesions in high-risk patients for hepatocellular carcinoma (2021)
  • Author(s): Feiqian Wang, Kazushi Numata, Makoto Chuma, Hiromi Nihonmatsu, Satoshi Moriya, Akito Nozaki, Katsuaki Ogushi, Hiroyuki Fukuda, Masahiro Okada, Litao Ruan, Wen Luo, Norihiro Koizumi, Masayuki Nakano, Masako Otani, Yoshiaki Inayama, Shin Maeda
  • Journal Title: Abdom Radiology Volume: 2021 Issue: 6 Pages: 1-13
  • DOI: 10.1007/s00261-020-02881-0
  • Peer Reviewed / Open Access
• [Journal Article] Estimating 3-dimensional liver motion using deep learning and 2-dimensional ultrasound images (2020)
  • Author(s): Shiho Yagasaki, Norihiro Koizumi, Yu Nishiyama, Ryosuke Kondo, Tsubasa Imaizumi, Naoki Matsumoto, Masahiro Ogawa, Kazushi Numata
  • Journal Title: International Journal of Computer Assisted Radiology and Surgery Volume: 5 Issue: 12 Pages: 1989-1995
  • DOI: 10.1007/s11548-020-02265-1
  • Peer Reviewed
• [Journal Article] CNNを用いた高速な超音波画像上の臓器検出手法の比較・検証 (2020)
  • Author(s): 五十嵐 立樹, 冨田 恭平, 西山 悠, 小泉 憲裕
  • Journal Title: 計測自動制御学会論文集 Volume: 56 Issue: 12 Pages: 560-569
  • DOI: 10.9746/sicetr.56.560
  • NAID: 130007958053
  • Peer Reviewed
• [Journal Article] Sagittal alignment in an MR-TRUS fusion biopsy using only the prostate contour in the axial image (2020)
  • Author(s): Riki Igarashi, Norihiro Koizumi, Yu Nishiyama, Kyohei Tomita, Yuka Shigenari, Sunao Shoji
  • Journal Title: ROBOMECH Journal Volume: 7 Issue: 1 Pages: 1-7
  • DOI: 10.1186/s40648-020-0155-9
  • Peer Reviewed / Open Access
• [Journal Article] Use of intra-procedural fusion imaging combining contrast-enhanced ultrasound using a perflubutane-based contrast agent and auto sweep three-dimensional ultrasound for guiding radiofrequency ablation and evaluating its efficacy in patients with hepatocellular carcinomaInternational (2020)
  • Author(s): Katsuyuki Sanga, Kazushi Numata, Hiromi Nihonmatsu, Katsuaki Ogushi, Hiroyuki Fukuda, Makoto Chuma, Hiroshi Hashimoto, Norihiro Koizumi, Shin Maeda
  • Journal Title: Journal of Hyperthermia (IJH) Volume: 37 Issue: 1 Pages: 202-211
  • DOI: 10.1080/02656736.2020.1729422
  • Peer Reviewed / Open Access
• [Journal Article] Usefulness of computer-aided diagnosis system in evaluating severity of benign prostatic hyperplasia, using a super-ellipse model to characterize changes in prostate contours (2020)
  • Author(s): Sunao Shoji, Yuka Shigenari, Izumi Hanada, Tatsuya Otaki, Takahiro Ogawa, Masayoshi Kawakami, Hakushi Kim, Masanori Hasegawa,Yoshiaki Kawamura, Norihiro Koizumi, Akira Miyajima
  • Journal Title: The Journal of Urology Volume: 203 Issue: Supplement 4 Pages: 605-606
  • DOI: 10.1097/ju.0000000000000891.07
  • Peer Reviewed / Open Access
• [Journal Article] Method of Medical DigITalization (Me-DigIT) and Its Effects and Social Impacts (2019)
  • Author(s): 小泉憲裕, 西山 悠, 江浦史生,今泉飛翔, 大塚研秀, 佐々木雄大, 重成佑香, 五十嵐立樹, 草原健太, 小林賢人, 月原弘之, 松本直樹, 小川眞広
  • Journal Title: Journal of the Japan Society for Precision Engineering Volume: 85 Issue: 9 Pages: 749-752
  • DOI: 10.2493/jjspe.85.749
  • NAID: 130007703006
  • ISSN: 0912-0289, 1882-675X
  • Year and Date: 2019-09-05
  • Open Access
• [Journal Article] Automatic Diagnosis by Compact Portable Ultrasound Robot : State Estimation of Internal Organs with Steady-State Kalman Filter (2019)
  • Author(s): Yudai Sasaki, Fumio Eura, Kento Kobayashi, Ryosuke Kondo, Kyohei Tomita, Yu Nishiyama, Hiroyuki Tsukihara, Naoki Matsumoto, and Norihiro Koizumi
  • Journal Title: Proc. of 2019 IEEE Healthcare Innovations and Point of Care Technologies Conference (HI-POCT) Volume: 2019 Pages: 29-32
  • DOI: 10.1109/hi-poct45284.2019.8962758
  • Peer Reviewed / Open Access
• [Presentation] A robotic ultrasound diagnostic system for tracking kidney motion and adjusting contact posture using an organ tracking robot (2022)
  • Author(s): J. Zhou, K. Kobayashi, R. Igarashi, Y. Watanabe, T. Fujibayashi, M. Yamada, M. Matsuyama, H. Tsukihara, K. Yoshinaka, N. Matsumoto, M. Ogawa, H. Miyazaki, K. Numata, H. Nagaoka, T. Iwai, H. Iijima, Y. Nishiyama, N. Koizumi
  • Organizer: CARS 2021 Computer Assisted Radiology and Surgery
  • Int'l Joint Research
• [Presentation] Ultrasound Fibrosis Diagnostic System Utilizing U-Net and CORALNet (2022)
  • Author(s): R. Saito, N. Koizumi, Y. Nishiyama, T. Imaizumi, K. Kusahara, S. Yagasaki, N. Matsumoto, R. Masuzaki, T. Takahashi, M. Ogawa
  • Organizer: CARS 2021 Computer Assisted Radiology and Surgery
  • Int'l Joint Research
• [Presentation] A novel complementation method of an acoustic shadow region utilizing a convolutional neural network for ultrasound-guided therapy (2022)
  • Author(s): M. Matsuyama, N. Koizumi, A. Otsuka, K. Kobayashi, S. Yagasaki, Y. Watanabe, J. Zhou, Y. Nishiyama, N. Matsumoto, H. Tsukihara, K. Numata
  • Organizer: CARS 2021 Computer Assisted Radiology and Surgery
  • Int'l Joint Research
• [Presentation] Parametric identification of prostate shape using the superellipses and its correlation with pathology (2022)
  • Author(s): Y. Onodera, N. Koizumi, Y. Shigenari, R. Igarashi, Y. Nishiyama, S. Shoji
  • Organizer: CARS 2021 Computer Assisted Radiology and Surgery
  • Int'l Joint Research
• [Presentation] Estimating 3-dimensional liver motion using deep learning and 2-dimensional ultrasound images (2020)
  • Author(s): S. Yagasaki, N. Koizumi, Y. Nishiyama, R. Kondo, T. Imaizumi, N. Matsumoto, M. Ogawa, K. Numata
  • Organizer: 34th International Congress and Exhibition on computer assisted radiology and surgery (CARS 2020)
  • Int'l Joint Research
• [Presentation] Contact state adjustment method to enhance organ motion compensation performance for a bed-type ultrasound diagnostic and therapeutic robot (2020)
  • Author(s): K. Kobayashi, N. Koizumi, Y. Sasaki, T. Kobayashi, Y. Watanabe, J. Zhou, A. Otsuka, Y. Nishiyama, H.Tsukihara, N. Matsumoto, H. Miyazaki, K. Numata, H. Nagaoka, T. Iwai, H. Iijima
  • Organizer: 34th International Congress and Exhibition on computer assisted radiology and surgery (CARS 2020)
  • Int'l Joint Research
• [Presentation] Classification of hepatic hemangiomas and blood vessels from ultrasonography by deep learning (2020)
  • Author(s): K. Kusahara, N. Koizumi, Y. Nishiyama, T. Imaizumi, R. Saito, S. Yagasaki, N. Matsumoto, M. Ogawa
  • Organizer: 34th International Congress and Exhibition on computer assisted radiology and surgery (CARS 2020)
  • Int'l Joint Research
• [Presentation] Development of Ultrasound Robot for Automatic Acquisition of Ultrasound Images (2020)
  • Author(s): Jiayi Zhou, Kento Kobayashi, Yusuke Watanabe, Takumi Fujibayashi, Momoko Matsuyama, Miyu Yamada, Hiroyuki Tsukihara, Yu Nishiyama, Norihiro Koizumi
  • Organizer: 16th Annual Asian Conference on Computer Aided Surgery (ACCAS2020)
  • Int'l Joint Research
• [Presentation] Study on automatic aquisition of diagnostic images by ultrasound diagnostic robot (2020)
  • Author(s): Yusuke Watanabe, Norihiro Koizumi, Yudai Sasaki, Kento Kobayashi, Takahiro Kobayashi, Zhou Jiayi, Yu Nishiyama, Hiroyuki Tsukihara, Kazushi Numata, Hideyuki Iijima, Toshiyuki Iwai, Hidetoshi Nagaoka
  • Organizer: 16th Annual Asian Conference on Computer Aided Surgery (ACCAS2020)
  • Int'l Joint Research
• [Presentation] Development of Ultrasound Robot for Automatic Acquisition of Ultrasound Images (2020)
  • Author(s): Ryosuke Saito, Norihiro Koizumi, Yu Nishiyama, Tsubasa Imaizumi, Kenta Kusahara, Shiho Yagasaki, Masahiro Ogawa, Naoki Matsumoto, Development of Ultrasound Robot for Automatic Acquisition of Ultrasound Images
  • Organizer: 16th Annual Asian Conference on Computer Aided Surgery (ACCAS2020)
  • Int'l Joint Research
• [Presentation] A contact state adjustment method to enhance organ motion compensation performance for a bed-type ultrasound diagnostic and therapeutic robot (2020)
  • Author(s): Kento Kobayashi, Yusuke Watanabe, Jiayi Zhou, Takumi Fujibayashi, Momoko Matsuyama, Miyu Yamada, Hiroyuki Tsukihara Hideyuki Iijima, Norihiro Koizumi
  • Organizer: 16th Annual Asian Conference on Computer Aided Surgery (ACCAS2020)
  • Int'l Joint Research
• [Presentation] Deep Learning Based 3-Dimensional Liver Motion Estimation Using 2-Dimensional Ultrasound Images (2019)
  • Author(s): Tsubasa Imaizumi, Ryosuke Kondo, Kenta Kusahara, Yu Nishiyama, Hiroyuki Tsukihara, Naoki Matsumoto, Kazushi Numata, Norihiro Koizumi
  • Organizer: 2019 IEEE International Conference on Cyborg and Bionic Systems (CBS 2019)
  • Int'l Joint Research
• [Presentation] Lesion Tracking Method Using CNN in Non-Invasive Ultrasound Diagnostic Treatment Integrated System (2019)
  • Author(s): Riki Igarashi, Kyohei Tomita, Yu Nishiyama, Yuka Shigenari, Norihiro Koizumi
  • Organizer: 2019 IEEE International Conference on Cyborg and Bionic Systems (CBS 2019)
  • Int'l Joint Research
• [Presentation] Development of Bed-Type Ultrasound Diagnosis and Therapeutic Robot (2019)
  • Author(s): Kento Kobayashi, Yudai Sasaki, Fumio Eura, Ryosuke Kondo, Kyohei Tomita, Takahiro Kobayashi, Yusuke Watanabe, Akihide Otsuka, Hiroyuki Tsukihara, Naoki Matsumoto, Kazushi Numata, Hidetoshi Nagaoka, Toshiyuki Iwai, Hideyuki Iijima, Yu Nishiyama, Norihiro Koizumi
  • Organizer: 2019 IEEE International Conference on Cyborg and Bionic Systems (CBS 2019)
  • Int'l Joint Research
• [Presentation] Matching axial images of magnetic resonance imaging and transrectal ultrasound based on deep learning (2019)
  • Author(s): Riki Igarashi, Norihiro Koizumi, Yu Nishiyama, Kyohei Tomita, Yuka Shigenari, Sunao Shoji
  • Organizer: 33rd International Congress and Exhibition on computer assisted radiology and surgery (CARS 2019)
  • Int'l Joint Research
• [Presentation] Segmentation of liver parenchyma in ultrasound images for automatic diagnosis of liver cirrhosis (2019)
  • Author(s): Tsubasa Imaizumi, Norihiro Koizumi, Yu Nishiyama, Naoki Matsumoto, Masahiro Ogawa
  • Organizer: 33rd International Congress and Exhibition on computer assisted radiology and surgery (CARS 2019)
  • Int'l Joint Research
• [Remarks] 小泉 憲裕の研究ブログ
  • URL: http://blog2009nkoizumi.japanprize.jp/
• [Remarks] 電気通信大学 小泉憲裕研究室
  • URL: http://www.medigit.mi.uec.ac.jp/
• [Remarks] 電気通信大学 小泉憲裕研究室
  • URL: http://www.medigit.mi.uec.ac.jp/
• [Remarks] 電気通信大学 小泉憲裕研究室
  • URL: http://www.medigit.mi.uec.ac.jp
• [Remarks] 小泉憲裕の研究ブログ
  • URL: http://blog2009nkoizumi.japanprize.jp
• [Patent(Industrial Property Rights)] 画像評価装置及びプログラム (2022)
  • Inventor(s): 小泉憲裕, 西山 悠, 小林賢人, Zhoh Jiayi, 他
  • Industrial Property Rights Holder: 小泉憲裕, 西山 悠, 小林賢人, Zhoh Jiayi, 他
  • Industrial Property Rights Type: 特許
  • Filing Date: 2022
• [Patent(Industrial Property Rights)] 超音波治療診断方法、プログラムおよび患部追従評価方法 (2021)
  • Inventor(s): 小泉憲裕, 大塚研秀, 西山 悠, 月原弘之, ほか３名
  • Industrial Property Rights Holder: 小泉憲裕, 大塚研秀, 西山 悠, 月原弘之, ほか３名
  • Industrial Property Rights Type: 特許
  • Filing Date: 2021