Action estimate of person in need of nursing care for which AI input is time series variation of sound source image generated by microphones at bed sides

2023 Fiscal Year Final Research Report

• Project/Area Number: 20K12748
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Review Section: Basic Section 90150:Medical assistive technology-related
• Research Institution: Fukuyama University
• Principal Investigator: Nakajima Hajime 福山大学, 工学部, 教授 (80542068)
• Project Period (FY): 2020-04-01 – 2024-03-31
• Keywords: 介護 / ベッド内行動推定 / 深層学習 / 音源像 / マルチモーダル / LSTM / メルスペクトログラム

Outline of Final Research Achievements

In this study, with the goal of understanding the behavior of care recipients and inpatients in bed, we used the sound source distribution image of behavioral sounds as input to deep learning, and attempted to estimate in-bed behavior based on the time-series changes of this sound source image. We built a behavioral sound collection system with four microphones placed on the head and footboard to acquire behavioral sounds in bed, and used the acquired data as input data for deep learning to generate a sound source image sequence and a log mel-spectrogram was generated. The sound source image was generated by representing the position inside the bed and its outer edge using a mesh, and by correlating the flight time and making corrections based on the signal intensity ratio. As a result of acquiring behavioral sound data and performing deep learning, we were able to confirm that it was possible to improve the accuracy of estimating the location of getting out of bed and scratching.

Free Research Field

計測工学

Academic Significance and Societal Importance of the Research Achievements

病院での入院患者や高齢の要介護者はベッドを中心とした生活を送っているが，せん妄や認知による離床徘徊といったトラブルが発生しており、ベッド内の行動が、特殊なセンサ群や画像モニタ等を用いず、対象者の発する音で推定できれば、介護、看護品質の向上に貢献できる。また、2次元の音源像の時間系列による行動の推定は、擦過音のような発生位置により行動の意味を持つ情報に対して有効な認識ツールとなり得る。