Development of innovative QoE analysis / evaluation system for HD video using brain function and physiological measurement.

2020 Fiscal Year Final Research Report

• Project/Area Number: 17K00154
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Research Field: Multimedia database
• Research Institution: University of Toyama
• Principal Investigator: Horita Yuukou 富山大学, 学術研究部都市デザイン学系, 教授 (80209303)
• Project Period (FY): 2017-04-01 – 2021-03-31
• Keywords: 画質評価 / 生体情報 / NIRS / QoE / 脈波 / 人の嗜好 / HRV / CNN

Outline of Final Research Achievements

To develop the QoE analysis / evaluation system for HD video using brain function and physiological measurement, it is very important to find out the biometric features related to human preference. So, we extracted the information on pulse wave from the Hb changes signal of NIRS. By using the FFT to the Hb signals, we found out the 2-nd peak of power spectrum that is implying the frequency information of the pulse wave. The frequency deviation of 2-nd peak may have some information about the change of brain activity, it is associated with the human preference for viewing the significant image content.
Then, in order to estimate human preference, we created a subject's heart rate variability spectrogram (HRVS) for still images using a trained convolutional neural network (CNN) and Support vector machine (SVM). Using these models, we showed the images to the subjects and extracted the measured HRVS features, and confirmed whether the subjects' preferences could be discriminated using SVM.

Free Research Field

サービス情報学

Academic Significance and Societal Importance of the Research Achievements

人の五感に関連する技術分野は感性工学を筆頭として、開発システムの限界を感じることが多かった。しかし、脳機能・生体計測情報を利用することで、ユーザ1人ひとりに最適化された製品やサービスを提供することが可能となる。ユーザの価値観や嗜好が無意識に推定できる脳血流量、心拍変動や脈波などの脳機能・生体計測情報は有効な手段となり、人とコンピュータとを密接に関連づけるBCI 技術への革新的な発展が期待される。