• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Sensing and Actuation for Accelerating Knowledge I/O by Humans

Research Project

Project/Area Number 20H04213
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeSingle-year Grants
Section一般
Review Section Basic Section 61010:Perceptual information processing-related
Research InstitutionOsaka Metropolitan University (2022)
Osaka Prefecture University (2020-2021)

Principal Investigator

Kise Koichi  大阪公立大学, 大学院情報学研究科, 教授 (80224939)

Co-Investigator(Kenkyū-buntansha) 石丸 翔也  大阪公立大学, 研究推進機構, 客員研究員 (10788730)
岩田 基  大阪公立大学, 大学院情報学研究科, 准教授 (70316008)
Project Period (FY) 2020-04-01 – 2023-03-31
Project Status Completed (Fiscal Year 2022)
Budget Amount *help
¥17,810,000 (Direct Cost: ¥13,700,000、Indirect Cost: ¥4,110,000)
Fiscal Year 2022: ¥5,330,000 (Direct Cost: ¥4,100,000、Indirect Cost: ¥1,230,000)
Fiscal Year 2021: ¥5,980,000 (Direct Cost: ¥4,600,000、Indirect Cost: ¥1,380,000)
Fiscal Year 2020: ¥6,500,000 (Direct Cost: ¥5,000,000、Indirect Cost: ¥1,500,000)
Keywords知能増強 / Learning Augmentation / Eye tracking / EOG / Accelometer / self-supervised learning / contrastive learning / vocabulometer / 学習増強 / センシング / アクチュエーション / 知識状態 / 心的状態 / モチベーション / 睡眠 / 学習支援 / 人工知能 / 視線解析 / 音読解析 / 自己教師あり学習 / Transformer / 読書行動解析 / 未知語推定 / 学習センシング / 学習アクチュエーション / 行動認識 / 英単語学習 / 手書き文字 / 確信度推定 / ナッジ戦略 / 英文多読
Outline of Research at the Start

学習増強(Learning Augmentation;LA) とは,人の学習に対する知能増強を意味するものであり,機械が人間を助けてより良い学習を実現するという人間中心の研究分野である.本研究では,「人に無理なく知識のI/O を高速化すること」をLA の理念とし,これを実現する新しいモデルを提案し,有効性を大規模実験によって実証することを目的とする.本研究で提案するLA のモデルは,LA の3 軸: (1) 知識の入力と出力,(2) 知識状態と心的状態,(3) 知識状態と心的状態のセンシングとアクチュエーション,を統合的に扱うものである.

Outline of Final Research Achievements

In this study, we focused on Learning Augmentation as a field of human-centered AI, and developed a system and evaluated experiments to realize it. The results of this study are (1) a proposal for a new vocabulary learning method called Vocabulometer, (2) a proposal for a nudge strategy to facilitate learning using Wordometer, which we have already developed, (3) a proposal for a method to facilitate learning by estimating the learner's confidence using eye trackers and arm movements (4) a deep learning method called Self-Supervised Learning that can accurately estimate reading behavior recognition and confidence even when labeled data is scarce.

Academic Significance and Societal Importance of the Research Achievements

本研究では、人間の学習をAIが助けるという枠組にのっとって、より人間が使いやすく,学習が容易であり,かつ離脱しにくいシステムを目指して研究を行ってきた.研究成果の学術的意義については,以下が挙げられる.(1) In-the-wildの環境で動作するツール群を構築し,有効性を検証した.(2)時系列データに対するself-supervised learningとその関連手法を提案した.一方で社会的意義としては,以下が挙げられる.(1) 作成した学習システムが,実際に学習者のパフォーマンスを改善することを示した.(2) 学習のモチベーションの維持,向上に有効となるナッジ戦略を4種類提案した.

Report

(4 results)
  • 2022 Annual Research Report   Final Research Report ( PDF )
  • 2021 Annual Research Report
  • 2020 Annual Research Report
  • Research Products

    (39 results)

All 2023 2022 2021 2020 Other

All Int'l Joint Research (2 results) Journal Article (11 results) (of which Int'l Joint Research: 3 results,  Peer Reviewed: 10 results,  Open Access: 4 results) Presentation (26 results) (of which Int'l Joint Research: 14 results,  Invited: 3 results)

  • [Int'l Joint Research] University of Melbourne(オーストラリア)

    • Related Report
      2021 Annual Research Report
  • [Int'l Joint Research] ドイツ人工知能研究センター(ドイツ)

    • Related Report
      2020 Annual Research Report
  • [Journal Article] Exploring Sensor Modalities to Capture User Behaviors for Reading Detection2022

    • Author(s)
      ISLAM Md. Rabiul、VARGO Andrew W.、IWATA Motoi、IWAMURA Masakazu、KISE Koichi
    • Journal Title

      IEICE Transactions on Information and Systems

      Volume: E105.D Issue: 9 Pages: 1629-1633

    • DOI

      10.1587/transinf.2020ZDL0003

    • ISSN
      0916-8532, 1745-1361
    • Year and Date
      2022-09-01
    • Related Report
      2022 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Obtaining Labels for In-the-Wild Studies:Using Visual Cues and Recall2021

    • Author(s)
      Andrew Vargo, Shoya Ishimaru, Md. Rabiul Islam, Benjamin Tag, Koichi Kise
    • Journal Title

      IEEE Pervasive

      Volume: 21 Issue: 1 Pages: 9-17

    • DOI

      10.1109/mprv.2021.3129500

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Self-supervised Learning for Reading Activity Classification2021

    • Author(s)
      Md. Rabiul Islam, Shuji Sakamoto, Yoshihiro Yamada, Andrew W. Vargo, Motoi Iwata, Masakazu Iwamura, and Koichi Kise DOI: 10.1145/3478088
    • Journal Title

      Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (ACM IMWUT journal)

      Volume: 5 Issue: 3 Pages: 1-22

    • DOI

      10.1145/3478088

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Confidence Estimation via Wrist Movement2021

    • Author(s)
      Seiya Tanaka, Andrew W. Vargo, Motoi Iwata, Koichi Kise
    • Journal Title

      Proc. of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing

      Volume: 1 Pages: 76-77

    • DOI

      10.1145/3460418.3479286

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Quality Assessment of Crowdwork via Eye Gaze:Towards Adaptive Personalized Crowdsourcing2021

    • Author(s)
      Md. Rabiul Islam, Shun Nawa, Andrew W. Vargo,Masaki Matsubara, Atsuyuki Morishima, Koichi Kise
    • Journal Title

      Proceedings of 18th IFIP TC13 International Conference on Human-Computer Interaction (INTRERACT2021)

      Volume: LNISA 12933 Pages: 104-113

    • DOI

      10.1007/978-3-030-85616-8_8

    • ISBN
      9783030856151, 9783030856168
    • Related Report
      2021 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Confidence-Aware Learning Assistant2021

    • Author(s)
      Shoya Ishimaru, Takanori Maruichi, Andreas Dengel, Koichi Kise
    • Journal Title

      arXiv:2102.07312 [cs.HC]

      Volume: 1 Pages: 1-9

    • Related Report
      2020 Annual Research Report
    • Int'l Joint Research
  • [Journal Article] Focusing on the face or getting distracted by social signals? The effect of distracting gestures on attentional focus in natural interaction2020

    • Author(s)
      Kajopoulos Jasmin、Cheng Gordon、Kise Koichi、Mueller Hermann J.、Wykowska Agnieszka
    • Journal Title

      Psychological Research

      Volume: 85 Issue: 2 Pages: 491-502

    • DOI

      10.1007/s00426-020-01383-4

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Mobile vocabulometer: A Context-based Learning Mobile Application to Enhance English Vocabulary Acquisition2020

    • Author(s)
      Yamaguchi Kohei、Iwata Motoi、Vargo Andrew、Kise Koichi
    • Journal Title

      Proceedings of the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing

      Volume: 1 Pages: 156-159

    • DOI

      10.1145/3410530.3414406

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Handwriting behavior as a self-confidence discriminator2020

    • Author(s)
      Maruichi Takanori、Uragami Taichi、Vargo Andrew、Kise Koichi
    • Journal Title

      Proceedings of the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing

      Volume: 1 Pages: 78-81

    • DOI

      10.1145/3410530.3414383

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Self-confidence prediction based on analysis of handwriting behavior using log-normal distributions of features2020

    • Author(s)
      Takanori Maruichi, Koichi Kise
    • Journal Title

      Proceedings of IAPR Workshop on Document Analysis Systems

      Volume: 1 Pages: 1-4

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Preliminary Experiments Toward Personalized Nudging Strategies Extensive Reading of English2020

    • Author(s)
      Sadafumi Tonomoto, Motoi Iwata, Koichi Kise
    • Journal Title

      Proc. of COBI2020

      Volume: 1 Pages: 1-8

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Open Access
  • [Presentation] 英単語学習を長期的に継続するための適応型学習システムの開発2023

    • Author(s)
      岩下 雄一郎, 岩田 基, Andrew W. Vargo, 黄瀬 浩一
    • Organizer
      2023年電子情報通信学会総合大会
    • Related Report
      2022 Annual Research Report
  • [Presentation] バーチャルペット飼育型単語学習システムとその評価2023

    • Author(s)
      浦上 太一, 黄瀬 浩一
    • Organizer
      情報処理学会第202回ヒューマンコンピュータインタラクション研究発表会
    • Related Report
      2022 Annual Research Report
  • [Presentation] バーチャルペット飼育型単語学習システムの提案2023

    • Author(s)
      浦上 太一, 黄瀬 浩一
    • Organizer
      情報処理学会第85回全国大会
    • Related Report
      2022 Annual Research Report
  • [Presentation] text-to-imageモデルによる生成画像を用いた英単語学習の提案と画像で表現可能な英単語の調査2023

    • Author(s)
      中島 悠太,岩田 基,黄瀬 浩一
    • Organizer
      情報処理学会研究報告教育学習支援情報システム(CLE)
    • Related Report
      2022 Annual Research Report
  • [Presentation] Towards Guided-Breath as an Intervention Within Computer-Assisted Learning2022

    • Author(s)
      Kanta Yamaoka, Andrew W. Vargo, Motoi Iwata, Koichi Kise
    • Organizer
      2022 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2022 International Symposium on Wearable Computers (UbiComp/ISWC '22 Adjunct)
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Sleep Quality Tracking and Alignment with a Simple In-the-Wild Task2022

    • Author(s)
      Andrew W. Vargo, Masayuki Fujimoto, Koichi Kise
    • Organizer
      2022 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2022 International Symposium on Wearable Computers (UbiComp/ISWC '22 Adjunct)
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Just the Right Amount of Exercise: Aerobic Exercise and Memory2022

    • Author(s)
      Andrew W. Vargo, Kentaro Ohara, Takumi Nakai, Koichi Kise
    • Organizer
      2022 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2022 International Symposium on Wearable Computers (UbiComp/ISWC '22 Adjunct)
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Evaluating Contrastive Learning for Fine-grained Reading Detection2022

    • Author(s)
      Md. Rabiul Islam, Andrew W. Vargo, Motoi Iwata, Masakazu Iwamura, and Koichi Kise
    • Organizer
      12th International Congress on Advanced Applied Informatics (IIAI-AAI)
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Helping Mobile Learners Know Unknown Words through their Reading Behavior2022

    • Author(s)
      Riku Higashimura, Andrew W. Vargo, Motoi Iwata, Koichi Kise
    • Organizer
      CHI Conference on Human Factors in Computing Systems
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] モバイル学習者の読書行動に基づく未知単語推定2022

    • Author(s)
      東村 理功,Andrew W. Vargo,岩田 基,黄瀬 浩一
    • Organizer
      第21回情報科学技術フォーラム(FIT2022)
    • Related Report
      2022 Annual Research Report
  • [Presentation] スマートフォンを用いた読書活動データからの未知単語推定2022

    • Author(s)
      東村 理功,岩田 基,黄瀬 浩一
    • Organizer
      2022年 電子情報通信学会
    • Related Report
      2021 Annual Research Report
  • [Presentation] 腕の動きとWebカメラを用いた英語問題の確信度推定2022

    • Author(s)
      田中 勢也, Andrew W. Vargo, 黄瀬 浩一
    • Organizer
      2022年電子情報通信学会総合大会
    • Related Report
      2021 Annual Research Report
  • [Presentation] 英語空所補充問題の自動生成2022

    • Author(s)
      森瀧 瑞希, 湯浅 成章, Andrew W. Vargo, 黄瀬 浩一
    • Organizer
      2022年電子情報通信学会総合大会
    • Related Report
      2021 Annual Research Report
  • [Presentation] 個人に適した英語多肢選択問題の自動生成方法の提案2022

    • Author(s)
      湯浅 成章, Andrew Vargo, 黄瀬 浩一
    • Organizer
      情報処理学会 教育学習支援情報システム研究会(CLE)
    • Related Report
      2021 Annual Research Report
  • [Presentation] マルチメディアVocabulometerによる動画の字幕を利用した英単語学習2022

    • Author(s)
      山口 航平, 岩田 基, Andrew Vargo, 黄瀬 浩一
    • Organizer
      情報処理学会 教育学習支援情報システム研究会(CLE)
    • Related Report
      2021 Annual Research Report
  • [Presentation] 習慣添付型マイクロラーニングの提案2022

    • Author(s)
      中井 拓海, Andrew Vargo, 黄瀬 浩一
    • Organizer
      情報処理学会 教育学習支援情報システム研究会(CLE)
    • Related Report
      2021 Annual Research Report
  • [Presentation] Reading of "Reading" for "Actuating": Augmenting Human Learning by "Experiential Supplements"2022

    • Author(s)
      Koichi Kise
    • Organizer
      Workshop on Words/Machines #4
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] Confidence Estimation via Wrist Movement2021

    • Author(s)
      Seiya Tanaka, Andrew W. Vargo, Motoi Iwata, Koichi Kise
    • Organizer
      2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Quality Assessment of Crowdwork via Eye Gaze:Towards Adaptive Personalized Crowdsourcing2021

    • Author(s)
      Md. Rabiul Islam, Shun Nawa, Andrew W. Vargo,Masaki Matsubara, Atsuyuki Morishima, Koichi Kise
    • Organizer
      18th IFIP TC13 International Conference on Human-Computer Interaction (INTRERACT2021)
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Reading of "Reading" for "Actuating": Augmenting Human Learning by "Experiential Supplements"2021

    • Author(s)
      Koichi Kise
    • Organizer
      The 3rd International Conference on Activity and Behavior Computing
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] Beyond Camera-Based Document Analysis and Recognition: Towards Camera-Based Human Document Interactions2021

    • Author(s)
      Koichi Kise
    • Organizer
      International Workshop on Document Analysis Systems (2021-9)
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] Mobile vocabulometer: A Context-based Learning Mobile Application to Enhance English Vocabulary Acquisition2020

    • Author(s)
      Yamaguchi Kohei、Iwata Motoi、Vargo Andrew、Kise Koichi
    • Organizer
      2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing
    • Related Report
      2020 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Handwriting behavior as a self-confidence discriminator2020

    • Author(s)
      Maruichi Takanori、Uragami Taichi、Vargo Andrew、Kise Koichi
    • Organizer
      2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing
    • Related Report
      2020 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Self-confidence prediction based on analysis of handwriting behavior using log-normal distributions of features2020

    • Author(s)
      Takanori Maruichi, Koichi Kise
    • Organizer
      IAPR Workshop on Document Analysis Systems
    • Related Report
      2020 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Preliminary Experiments Toward Personalized Nudging Strategies Extensive Reading of English2020

    • Author(s)
      Sadafumi Tonomoto, Motoi Iwata, Koichi Kise
    • Organizer
      COBI2020
    • Related Report
      2020 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Reading Activity Classification Using Self-supervised Deep Learning2020

    • Author(s)
      Md. Rabiul Islam, Shuji Sakamoto, Yoshihiro Yamada, Andrew Vargo, Motoi Iwata, Masakazu Iwamura, Koichi Kise
    • Organizer
      情報処理学会ヒューマンコンピュータインタラクション研究会
    • Related Report
      2020 Annual Research Report

URL: 

Published: 2020-04-28   Modified: 2024-01-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi