• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Understanding and control of mind through gaze

Research Project

Project/Area Number 18H03312
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeSingle-year Grants
Section一般
Review Section Basic Section 61050:Intelligent robotics-related
Research InstitutionHiroshima City University

Principal Investigator

Mitsugami Ikuhisa  広島市立大学, 情報科学研究科, 教授 (00467458)

Project Period (FY) 2018-04-01 – 2022-03-31
Project Status Completed (Fiscal Year 2022)
Budget Amount *help
¥17,030,000 (Direct Cost: ¥13,100,000、Indirect Cost: ¥3,930,000)
Fiscal Year 2021: ¥2,860,000 (Direct Cost: ¥2,200,000、Indirect Cost: ¥660,000)
Fiscal Year 2020: ¥5,330,000 (Direct Cost: ¥4,100,000、Indirect Cost: ¥1,230,000)
Fiscal Year 2019: ¥3,510,000 (Direct Cost: ¥2,700,000、Indirect Cost: ¥810,000)
Fiscal Year 2018: ¥5,330,000 (Direct Cost: ¥4,100,000、Indirect Cost: ¥1,230,000)
Keywords視線推定 / 心的状態推定 / 眼球・頭部協調運動 / VR / 注視推定 / e-learning / 視覚探索課題 / ドローン / 人物行動解析 / 注視 / 人物映像処理 / 理解度推定 / 健康状態推定 / 姿勢推定
Outline of Final Research Achievements

This research project aims to estimate individuals' mental states through their gazing behavior. To achieve this, we developed technologies that measure and estimate gaze, along with ones for estimating mental states using this data. This project involves the following four topics; 1) modeling the eye-head coordination movement, an unconscious reflex of people, to estimate the direction of gaze through machine learning, 2) visual search behavior analysis under a VR environment to estimate mental health, and 3) analysis of the students' gazing behavior to estimate their level of understanding during lectures. We also revealed that the eye-head coordination model is not only useful for gaze estimation but also for mental state estimation.

Academic Significance and Societal Importance of the Research Achievements

本研究は,心的状態推定の問題を,大量データと機械学習手法によって人の内部での減少に立ち入らずに取り扱うのではなく,心的影響を強く受けるが外部からの観測が可能な注視行動に着眼し,認知科学・心理学的の知見を踏まえて解決を図った点で独自性があり,その成果には一定の学術的価値があると考えている.本研究では,成果概要で述べたとおりさまざまなトピックを取り扱ったが,特に眼球・頭部協調運動のモデル化は,利用範囲が広いことを踏まえ,定量的で信頼性の高い性能評価を行った上で,この技術を容易に利活用できるようPythonライブラリとして整理しており,その学術的・社会的な意義は大きい.

Report

(5 results)
  • 2022 Final Research Report ( PDF )
  • 2021 Annual Research Report
  • 2020 Annual Research Report
  • 2019 Annual Research Report
  • 2018 Annual Research Report
  • Research Products

    (63 results)

All 2022 2021 2020 2019 2018

All Journal Article (14 results) (of which Int'l Joint Research: 2 results,  Peer Reviewed: 14 results,  Open Access: 11 results) Presentation (49 results) (of which Int'l Joint Research: 15 results,  Invited: 6 results)

  • [Journal Article] The Effect of Mood State on Visual Search Times under VR Environment2021

    • Author(s)
      坂井 美友、世波 里菜、村上 純一、満上 育久、山岸 典子
    • Journal Title

      Transactions of the Virtual Reality Society of Japan

      Volume: 26 Issue: 1 Pages: 52-61

    • DOI

      10.18974/tvrsj.26.1_52

    • NAID

      130008014109

    • ISSN
      1344-011X, 2423-9593
    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Part-to-full shape matching of different human subjects2021

    • Author(s)
      Rakprayoon Panjawee、Ruchanurucks Miti、Thainimit Somying、Mitsugami Ikuhisa
    • Journal Title

      Heliyon

      Volume: 7 Issue: 10 Pages: e08214-e08214

    • DOI

      10.1016/j.heliyon.2021.e08214

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Detecting Lower MMSE Scores in Older Adults Using Cross-Trial Features From a Dual-Task With Gait and Arithmetic2021

    • Author(s)
      Wu Shuqiong、Matsuura Taku、Okura Fumio、Makihara Yasushi、Zhou Chengju、Aoki Kota、Mitsugami Ikuhisa、Yagi Yasushi
    • Journal Title

      IEEE Access

      Volume: 9 Pages: 150268-150282

    • DOI

      10.1109/access.2021.3126067

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] 3D Pose Estimation Using Multiple Asynchronous Cameras2020

    • Author(s)
      Morimoto Takashi、Mitsugami Ikuhisa
    • Journal Title

      Smart Innovation, Systems and Technologies

      Volume: 204 Pages: 39-50

    • DOI

      10.1007/978-981-15-8944-7_3

    • ISBN
      9789811589430, 9789811589447
    • Related Report
      2020 Annual Research Report
    • Peer Reviewed
  • [Journal Article] A Video-Based Gait Disturbance Assessment Tool for Diagnosing Idiopathic Normal Pressure Hydrocephalus2020

    • Author(s)
      R. Liao, Y. Makihara, D. Muramatsu, I. Mitsugami, Y. Yagi, K. Yoshiyama, H. Kazui, M. Takeda
    • Journal Title

      IEEJ Transactions on Electrical and Electronic Engineering

      Volume: 15 Issue: 3 Pages: 433-441

    • DOI

      10.1002/tee.23072

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Analysis of gait changes caused by fixing left knee joint focusing on balance and kinematics2020

    • Author(s)
      山添大丈, 満上育久, 小川拓也, 八木康史
    • Journal Title

      Journal of Nursing Science and Engineering

      Volume: 7 Issue: 0 Pages: 33-42

    • DOI

      10.24462/jnse.7.0_33

    • NAID

      130007803844

    • ISSN
      2188-4323, 2432-6283
    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Statistical Analysis of Dual-task Gait Characteristics for Cognitive Score Estimation2019

    • Author(s)
      Taku Matsuura, Kazuhiro Sakashita, Andrey Grushnikov, Fumio Okura, Ikuhisa Mitsugami, Yasushi Yagi
    • Journal Title

      Scientific Reports

      Volume: 9 Issue: 1 Pages: 1-12

    • DOI

      10.1038/s41598-019-56485-w

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Analysis of head and chest movements during walking that correspond to gaze directions2019

    • Author(s)
      Hirotake Yamazoe, Ikuhisa Mitsugami, Tsukasa Okada, Yasushi Yagi
    • Journal Title

      Experimental Brain Research

      Volume: 237 Issue: 11 Pages: 3047-3058

    • DOI

      10.1007/s00221-019-05650-8

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Early Detection of Lower MMSE Scores in Elderly Based on Dual-Task Gait2019

    • Author(s)
      Aoki Kota、Ngo Trung Thanh、Mitsugami Ikuhisa、Okura Fumio、Niwa Masataka、Makihara Yasushi、Yagi Yasushi、Kazui Hiroaki
    • Journal Title

      IEEE Access

      Volume: 7 Pages: 40085-40096

    • DOI

      10.1109/access.2019.2906908

    • Related Report
      2018 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Depth Error Correction for Projector-Camera Based Consumer Depth Camera2018

    • Author(s)
      Hirotake Yamazoe, Hitoshi Habe, Ikuhisa Mitsugami, Yasushi Yagi
    • Journal Title

      Computational Visual Media

      Volume: 4 Issue: 2 Pages: 103-111

    • DOI

      10.1007/s41095-017-0103-7

    • Related Report
      2018 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] [Paper] Automatic Collection of Dual-task Human Behavior for Analysis of Cognitive Function2018

    • Author(s)
      Fumio Okura, Ikuhisa Mitsugami, Masataka Niwa, Kota Aoki, Chengju Zhou, Yasushi Yagi
    • Journal Title

      ITE Transactions on Media Technology and Applications

      Volume: 6 Issue: 2 Pages: 138-150

    • DOI

      10.3169/mta.6.138

    • NAID

      130006603032

    • ISSN
      2186-7364
    • Related Report
      2018 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Directional Characteristics Evaluation of Appearance-Based Gait Recognition2018

    • Author(s)
      Yui Shigeki, Fumio Okura, Ikuhisa Mitsugami, Kenichi Hayashi, Yasushi Yagi
    • Journal Title

      IPSJ Transactions on Computer Vision and Applications

      Volume: 10 Issue: 1

    • DOI

      10.1186/s41074-018-0046-7

    • Related Report
      2018 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] [Papers] Growth Assessment of School-age Children using Dualtask Observation2018

    • Author(s)
      Zhou Chengju、Mitsugami Ikuhisa、Okura Fumio、Aoki Kota、Yagi Yasushi
    • Journal Title

      ITE Transactions on Media Technology and Applications

      Volume: 6 Issue: 4 Pages: 286-296

    • DOI

      10.3169/mta.6.286

    • NAID

      130007493366

    • ISSN
      2186-7364
    • Related Report
      2018 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Estimating 3D human shape under clothing from a single RGB image2018

    • Author(s)
      Yui Shigeki, Fumio Okura, Ikuhisa Mitsugami, Yasushi Yagi
    • Journal Title

      IPSJ Transactions on Computer Vision and Applications

      Volume: 10 Issue: 1

    • DOI

      10.1186/s41074-018-0052-9

    • Related Report
      2018 Annual Research Report
    • Peer Reviewed / Open Access
  • [Presentation] Geometrically and Temporally Consistent Visual Annotation for Smart Glasses2022

    • Author(s)
      Kanade Sumino, Naoya Wakita, Ikuhisa Mitsugami
    • Organizer
      The 14th Asia-Pacific Workshop on Mixed and Augmented Reality
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Expression-Controllable Facial Video Generation for Impression Quantification2022

    • Author(s)
      Kanade Sumino, Ikuhisa Mitsugami, Ryusuke Sagawa
    • Organizer
      2022 IEEE 11th Global Conference on Consumer Electronics
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Fly at My Side: 歩行者の速度変化に即応する自動追従ドローン2022

    • Author(s)
      山下浩豊, 満上育久
    • Organizer
      画像の認識・理解シンポジウム
    • Related Report
      2021 Annual Research Report
  • [Presentation] 対面時の印象評価に向けた多様な顔表情映像の生成2022

    • Author(s)
      住野奏, 満上育久, 佐川立昌
    • Organizer
      画像の認識・理解シンポジウム
    • Related Report
      2021 Annual Research Report
  • [Presentation] 全身姿勢の変化に基づいた歩行者の未来位置予測2022

    • Author(s)
      平井孝明, 山下浩豊, 満上育久
    • Organizer
      画像の認識・理解シンポジウム
    • Related Report
      2021 Annual Research Report
  • [Presentation] 対面時の印象評価に向けた多様な顔表情映像の生成2022

    • Author(s)
      住野奏, 満上育久, 佐川立昌
    • Organizer
      情報処理学会CVIM研究会
    • Related Report
      2021 Annual Research Report
  • [Presentation] 全身姿勢の変化に基づいた歩行者の未来位置予測2022

    • Author(s)
      平井孝明, 山下浩豊, 満上育久
    • Organizer
      情報処理学会CVIM研究会
    • Related Report
      2021 Annual Research Report
  • [Presentation] Autonomous Human-Following Drone for Monitoring a Pedestrian from Constant Distance and Direction2021

    • Author(s)
      Hiroto Yamashita, Takashi Morimoto, Ikuhisa Mitsugami
    • Organizer
      2021 IEEE 10th Global Conference on Consumer Electronics
    • Related Report
      2021 Annual Research Report 2020 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 複数台ドローンカメラを用いたマーカーレスモーションキャプチャ2021

    • Author(s)
      平井孝明, 山下浩豊, 森本隆志, 満上育久
    • Organizer
      第23回IEEE広島支部学生シンポジウム
    • Related Report
      2021 Annual Research Report 2020 Annual Research Report
  • [Presentation] 眼球・頭部協調関係の学習による頭部運動からの視線確率分布推定2021

    • Author(s)
      村本修斗, 山下浩豊, 満上育久
    • Organizer
      第23回IEEE広島支部学生シンポジウム
    • Related Report
      2021 Annual Research Report 2020 Annual Research Report
  • [Presentation] 眼球・頭部協調運動モデリングに基づく頭部運動からの注視分布推定2021

    • Author(s)
      村上純一, 満上育久
    • Organizer
      画像の認識・理解シンポジウム
    • Related Report
      2021 Annual Research Report
  • [Presentation] Gaze Analysis based on Computer Vision and Virtual Reality Technologies2021

    • Author(s)
      Ikuhisa Mitsugami
    • Organizer
      東北大学 電気通信研究所 令和元年度共同プロジェクト研究発表会
    • Related Report
      2020 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] 心を推し量る人物行動解析2021

    • Author(s)
      満上育久
    • Organizer
      立命館大学稲盛経営哲学研究センターワークショップ
    • Related Report
      2020 Annual Research Report
    • Invited
  • [Presentation] 二人称映像を用いたアイトラッカー2021

    • Author(s)
      渡邊康平, 満上育久, 佐川立昌
    • Organizer
      情報処理学会CVIM研究会
    • Related Report
      2020 Annual Research Report
  • [Presentation] 非同期・非固定カメラ群を用いたマーカーレスモーションキャプチャ2021

    • Author(s)
      山下浩豊, 森本隆志, 満上育久
    • Organizer
      画像の認識・理解シンポジウム
    • Related Report
      2020 Annual Research Report
  • [Presentation] スマートグラスを用いた人物認識ウェアラブルシステム2021

    • Author(s)
      住野奏, 脇田直也, 満上育久
    • Organizer
      画像の認識・理解シンポジウム
    • Related Report
      2020 Annual Research Report
  • [Presentation] VR講義動画視聴時の注視行動に基づく理解度推定2021

    • Author(s)
      胡間舞子, 黒田明日香, 満上育久
    • Organizer
      画像の認識・理解シンポジウム
    • Related Report
      2020 Annual Research Report
  • [Presentation] VR視覚探索時の頭部運動と心的状態の関係の調査2021

    • Author(s)
      清武優衣, 世波里菜, 山岸典子, 満上育久
    • Organizer
      画像の認識・理解シンポジウム
    • Related Report
      2020 Annual Research Report
  • [Presentation] サーマルカメラを用いた鼻呼吸数測定2021

    • Author(s)
      才野翔伍, 満上育久
    • Organizer
      第39回日本ロボット学会学術講演会
    • Related Report
      2020 Annual Research Report
  • [Presentation] 3D Pose Estimation Using Multiple Asynchronous Cameras2020

    • Author(s)
      Takashi Morimoto, Ikuhisa Mitsugami
    • Organizer
      International Conference on Activity and Behavior Computing
    • Related Report
      2020 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Drone Control for Monitoring a Walking Person from Constant Distance2020

    • Author(s)
      Hiroto Yamashita, Takashi Morimoto, Ikuhisa Mitsugami
    • Organizer
      International Conference on Activity and Behavior Computing
    • Related Report
      2020 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 広域移動する人物の映像解析のための自動追従ドローン2020

    • Author(s)
      山下浩豊, 満上育久
    • Organizer
      情報処理学会CVIM研究会
    • Related Report
      2020 Annual Research Report
  • [Presentation] 3次元姿勢推定に基づく人物自動追従ドローン2020

    • Author(s)
      山下浩豊, 満上育久
    • Organizer
      画像の認識・理解シンポジウム
    • Related Report
      2020 Annual Research Report
  • [Presentation] Gaze from Head: Gaze Estimation without Observing Eye2019

    • Author(s)
      Jun'ichi Murakami, Ikuhisa Mitsugami
    • Organizer
      The 5th Asian Conference on Pattern Recognition
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Dataset of Eye-Head Coordination under VR Experiences2019

    • Author(s)
      Jun'ichi Murakami, Ikuhisa Mitsugami
    • Organizer
      The International Workshop on Electronics, Information and Communication
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Integration of Body Poses Captured by Asynchronous Multiple Kinects2019

    • Author(s)
      Takashi Morimoto, Ikuhisa Mitsugami
    • Organizer
      The International Workshop on Electronics, Information and Communication
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Gaze Distribution related to Degree of Understanding under VR-based E-learning System2019

    • Author(s)
      Asuka Kuroda, Ikuhisa Mitsugami
    • Organizer
      The International Workshop on Electronics, Information and Communication
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] VR-based Eye and Head Motion Collection for Modeling Their Coordination2019

    • Author(s)
      Jun'ichi Murakami, Ikuhisa Mitsugami
    • Organizer
      2019 IEEE 8th Global Conference on Consumer Electronics
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Motion Capture System by Spatio-Temporal Integration of Multiple Kinects2019

    • Author(s)
      Takashi Morimoto, Ikuhisa Mitsugami
    • Organizer
      2019 IEEE 8th Global Conference on Consumer Electronics
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] The effect of mood state on visual search times under VR environment2019

    • Author(s)
      Ikuhisa Mitsugami, Miyu Sakai, Rina Yonami, Jun'ichi Murakami, Noriko Yamagishi
    • Organizer
      The 15th Asia-Pacific Conference on Vision
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Gaze from Head: Gaze Estimation without Observing Eyes2019

    • Author(s)
      Ikuhisa Mitsugami
    • Organizer
      APSCIT 2019 Annual Meeting
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] コンピュータビジョン・VRを活用した注視行動分析2019

    • Author(s)
      満上育久
    • Organizer
      電子情報通信学会ヒューマン情報処理研究会
    • Related Report
      2019 Annual Research Report
    • Invited
  • [Presentation] 心を推し量るための人物映像解析技術2019

    • Author(s)
      満上育久
    • Organizer
      広島市立大学産学連携研究発表会
    • Related Report
      2019 Annual Research Report
    • Invited
  • [Presentation] デュアルタスク歩行解析に基づく高齢者の認知機能スコア推定2019

    • Author(s)
      松浦拓, 大倉史生, 槇原靖, 周成菊, 青木工太, 満上育久, 八木康史
    • Organizer
      電子情報通信学会ヒューマンコミュニケーション基礎研究会
    • Related Report
      2019 Annual Research Report
  • [Presentation] 歩行者の全身を撮影し続けるためのドローン制御2019

    • Author(s)
      山下浩豊, 満上育久
    • Organizer
      第21回IEEE広島支部学生シンポジウム
    • Related Report
      2019 Annual Research Report
  • [Presentation] KinectとOpenPoseの統合による姿勢推定性能向上のための関節点の対応関係調査2019

    • Author(s)
      森本隆志, 満上育久
    • Organizer
      第18回情報科学技術フォーラム
    • Related Report
      2019 Annual Research Report
  • [Presentation] VR講義動画視聴における理解度と注視分布の関係の分析2019

    • Author(s)
      黒田明日香, 満上育久
    • Organizer
      第18回情報科学技術フォーラム
    • Related Report
      2019 Annual Research Report
  • [Presentation] 視線追跡型VRゴーグルを用いた眼球・頭部協調運動データ収集システム2019

    • Author(s)
      村上純一, 満上育久
    • Organizer
      画像の認識・理解シンポジウム
    • Related Report
      2019 Annual Research Report
  • [Presentation] 視線追跡型VRゴーグルを用いた眼球・頭部協調運動データ収集システム2019

    • Author(s)
      村上純一, 満上育久
    • Organizer
      情報処理学会CVIM研究会
    • Related Report
      2019 Annual Research Report
  • [Presentation] 複数Kinectの姿勢データの時空間統合によるモーションキャプチャシステム2019

    • Author(s)
      森本隆志, 満上育久
    • Organizer
      情報処理学会CVIM研究会
    • Related Report
      2019 Annual Research Report
  • [Presentation] VR環境を用いた受講生の理解度と注視運動の関係性の調査2019

    • Author(s)
      黒田明日香, 満上育久
    • Organizer
      情報処理学会CVIM研究会
    • Related Report
      2019 Annual Research Report
  • [Presentation] 多重時間幅オプティカルフローによる顔の各部位の微小動作検出2019

    • Author(s)
      田川聖一, 越後富夫, 満上育久
    • Organizer
      第63回システム制御情報学会研究発表講演会
    • Related Report
      2019 Annual Research Report
  • [Presentation] VR環境下での視覚探索課題とその分析2019

    • Author(s)
      坂井美友, 山岸典子, 満上育久
    • Organizer
      日本視覚学会2019年冬季大会
    • Related Report
      2018 Annual Research Report
  • [Presentation] Gaze from Head: Gaze Estimation without Observing Eyes2018

    • Author(s)
      Ikuhisa Mitsugami
    • Organizer
      脳情報通信融合研究センター
    • Related Report
      2018 Annual Research Report
    • Invited
  • [Presentation] 早期認知障害検出に向けた高齢者デュアルタスクデータセットの統計分析2018

    • Author(s)
      松浦 拓, 阪下 和弘, 満上 育久, 大倉 史生, 八木 康史
    • Organizer
      情報処理学会CVIM研究会
    • Related Report
      2018 Annual Research Report
  • [Presentation] 低解像度俯瞰映像からの注視推定2018

    • Author(s)
      沖中大和, 満上育久, 西野恒, 八木康史
    • Organizer
      第21回画像の認識・理解シンポジウム
    • Related Report
      2018 Annual Research Report
  • [Presentation] Estimating 3D human shape under clothing from a single RGB image2018

    • Author(s)
      Yui Shigeki, Fumio Okura, Ikuhisa Mitsugami, Yagi Yasushi
    • Organizer
      第21回画像の認識・理解シンポジウム
    • Related Report
      2018 Annual Research Report
  • [Presentation] 単一RGB画像からの着衣変化に頑健な体型推定および個人認証への応用2018

    • Author(s)
      繁木 結衣, 大倉 史生, モハメド ハサン, 満上 育久, 八木 康史
    • Organizer
      電子情報通信学会BioX研究会
    • Related Report
      2018 Annual Research Report
  • [Presentation] Gaze and Body Capture System under VR Experiences2018

    • Author(s)
      Jun'ichi Murakami, Takashi Morimoto, Ikuhisa Mitsugami
    • Organizer
      The 24th ACM Symposium on Virtual Reality Software and Technology
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research

URL: 

Published: 2018-04-23   Modified: 2024-01-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi