Non-contact Vital and Emotional Sensing by Camera and Its Medical Application

• Project/Area Number: 19K12750
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Review Section: Basic Section 90110:Biomedical engineering-related
• Research Institution: Chiba University
• Principal Investigator: Tsumura Norimichi 千葉大学, 大学院工学研究院, 准教授 (00272344)
• Co-Investigator(Kenkyū-buntansha): 金井 智恵子 和洋女子大学, 人文学部, 准教授 (00611089) ; 小川 恵子 金沢大学, 附属病院, 特任准教授 (30514575) ; 土居 裕和 国士舘大学, 理工学部, 准教授 (40437827)
• Project Period (FY): 2019-04-01 – 2022-03-31
• Project Status: Completed (Fiscal Year 2021)
• Budget Amount: ¥4,290,000 (Direct Cost: ¥3,300,000、Indirect Cost: ¥990,000); Fiscal Year 2021: ¥910,000 (Direct Cost: ¥700,000、Indirect Cost: ¥210,000); Fiscal Year 2020: ¥1,820,000 (Direct Cost: ¥1,400,000、Indirect Cost: ¥420,000); Fiscal Year 2019: ¥1,560,000 (Direct Cost: ¥1,200,000、Indirect Cost: ¥360,000)
• Keywords: バイタル / 非接触 / 血圧 / 連続計測 / 脈波 / ウイナー推定 / 情動 / 画像処理 / 情動計測 / ASD / 早期発見 / 生体光学 / バイタルモニタリング / ASD早期診断

Outline of Research at the Start

顔の動画像から心拍数をはじめとしたバイタル・情動情報を抽出する非接触型バイタル・
情動センシング技術の発展が目覚ましい．研究代表者らは, 高額な機器を必要とする従来型の技術に代わり, 安価なRGBカメラでの非接触型バイタル・情動センシング技術を開発した．本研究は, この技術的基盤を更に発展・高度化させ, 小児医療応用に最適化された, 安価な非接触型バイタル・情動センシングの実現を目的とする．この目的を達成するため, 体動が激しい乳幼児を対象に, 低照度下でも安定してバイタル・情動センシングを行うための, 新たな画像処理・信号解析技術を開発する.

Outline of Final Research Achievements

In recent years, there have been studies that have attempted to estimate blood pressure using a camera, but there remains much room for improvement. The critical issues are the need to capture images of the face and palms at two locations and the lack of universality for individual differences. In this study, we introduced spatial pulse wave information, deep learning, and eliminated the need for window processing. To achieve these goals, we constructed a Residual-CNN-based deep learning architecture that introduces the spatial description of facial pulse wave and Attention, and demonstrated the effectiveness of the proposed method based on actual experiments.

Academic Significance and Societal Importance of the Research Achievements

顔の動画像から心拍数をはじめとした血圧などのバイタル・情動情報を抽出する非接触型バイタル・情動センシング技術の発展が目覚ましい．我々は, 高額な機器を必要とする従来型の技術に代わり, 安価なRGBカメラでの非接触型バイタル・情動センシング技術を開発した．今後は, この技術的基盤を更に発展・高度化させ, 小児医療応用に最適化された, 安価な非接触型バイタル・情動センシングの実現などを行う．小児医療応用以外にも様々な医療分野において適用可能であり，特に社会的意義は高い．

Research Products

• [Journal Article] Development of telemedicine tools with an emphasis on visual observation (2022)
  • Author(s): Masato Tahakashi, Reimei Koike, Kazuki Nagasawa, Manabe Yasuhiro, Hirofumi, Hirana,Mitsuyuki Takemura, Tetsuya Hongara,Izumi Kimoto, Keiko Ogawa-Ochiai, Norimichi Tsumura
  • Journal Title: Artif Life Robotics 27 Volume: 27 Issue: 1 Pages: 3847-3847
  • DOI: 10.1007/s10015-022-00731-4
  • Peer Reviewed / Open Access
• [Journal Article] Automatic classification of adult males with and without autism spectrum disorder by non-contac measurement of autonomic nervous system activation (2021)
  • Author(s): Hirokazu Doi, Norimichi Tsumura, Chieko Kanai, Kenta Masui, Ryota Mitsuhashi,Takumi Nagasawa
  • Journal Title: Frontiers in Psychiatry Volume: 17 Pages: 1-10
  • DOI: 10.3389/fpsyt.2021.625978
  • Peer Reviewed / Open Access
• [Journal Article] Lack of implicit visual perspective taking in adult males with autism spectrum disorders (2020)
  • Author(s): Doi, H., Kanai, C., Tsumura, N., Shinohara, K., Kato, N
  • Journal Title: Research in Developmental Disabilities Volume: 99 Pages: 103593-103593
  • DOI: 10.1016/j.ridd.2020.103593
  • Peer Reviewed
• [Journal Article] [Paper] Measurement of advertisement effect based on multimodal emotional responses considering personality (2020)
  • Author(s): 21.Kenta Masui, Genki Okada, Norimichi Tsumura
  • Journal Title: ITE Transactions on Media Technology and Applications Volume: 8 Issue: 1 Pages: 49-59
  • DOI: 10.3169/mta.8.49
  • NAID: 130007779389
  • ISSN: 2186-7364
  • Peer Reviewed
• [Journal Article] Non-contact method of blood pressure estimation using only facial video (2020)
  • Author(s): Ryo Takahashi, Keiko Ogawa-Ochiai, Norimichi Tsumura
  • Journal Title: Artificial Life and Robotics Volume: 25 Issue: 3 Pages: 343-350
  • DOI: 10.1007/s10015-020-00622-6
  • Peer Reviewed
• [Journal Article] Stress levels estimation from facial video based on non-contact measurement of pulse wave (2020)
  • Author(s): .Kaito Iuchi, Ryota Mitsuhashi, Takashi Goto, Akira Matsubara, Takahiro Hirayama, Hideki Hashizume, and Norimichi Tsumura
  • Journal Title: Artificial Life and Robotics Volume: 25 Issue: 3 Pages: 335-342
  • DOI: 10.1007/s10015-020-00624-4
  • Peer Reviewed
• [Journal Article] Development of a camera-based remote diagnostic system focused on color reproduction using color charts (2020)
  • Author(s): Masato Takahashi, Ryo Takahashi, Yasuhiro Morihara, Isseki Kin, Keiko Ogawa-Ochiai, Norimichi Tsumura
  • Journal Title: Artif Life Robotics Volume: 25 Issue: 3 Pages: 370-376
  • DOI: 10.1007/s10015-020-00627-1
  • Peer Reviewed / Open Access