• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Development of a system to support playing musical instruments using augmented reality technology

Research Project

Project/Area Number 18K11573
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeMulti-year Fund
Section一般
Review Section Basic Section 62030:Learning support system-related
Research InstitutionThe University of Tokushima

Principal Investigator

SHISHIBORI Masami  徳島大学, 大学院社会産業理工学研究部(理工学域), 教授 (50274262)

Co-Investigator(Kenkyū-buntansha) 光原 弘幸  徳島大学, 大学院社会産業理工学研究部(理工学域), 准教授 (90363134)
大野 将樹  徳島大学, 大学院社会産業理工学研究部(理工学域), 講師 (90433739)
Project Period (FY) 2018-04-01 – 2021-03-31
Project Status Completed (Fiscal Year 2020)
Budget Amount *help
¥4,290,000 (Direct Cost: ¥3,300,000、Indirect Cost: ¥990,000)
Fiscal Year 2020: ¥1,040,000 (Direct Cost: ¥800,000、Indirect Cost: ¥240,000)
Fiscal Year 2019: ¥1,560,000 (Direct Cost: ¥1,200,000、Indirect Cost: ¥360,000)
Fiscal Year 2018: ¥1,690,000 (Direct Cost: ¥1,300,000、Indirect Cost: ¥390,000)
Keywords学習支援 / 拡張現実 / 回帰問題 / ピアノ演奏 / 教育工学 / 音楽情報処理 / AR技術 / 楽器演奏 / 教育支援システム
Outline of Final Research Achievements

The purpose of this study is to develop a support system for playing musical instruments that can efficiently help students learn how to play instruments correctly, especially fingering, without the professional advisor. Fingering refers to the use of fingers when playing a musical instrument, and is an important performance skill for smooth playing. Conventional systems support performance by displaying a fingering model as a teacher on a monitor screen. The purpose of this study is to develop an instrument performance support system that enables learners to learn intuitively by wearing an augmented reality (AR) device and displaying a fingering model that serves as a teacher in the hand of the learner playing, while simultaneously viewing the fingering model that serves as a teacher and the learner's own fingering.

Academic Significance and Societal Importance of the Research Achievements

従来の楽器演奏システムは,カメラで撮影した自身の運指をモニター画面に映し,それを見ながら運指学習をするミラーライクのものであった.本研究の提案するAR技術を用いて自身の手元に運指モデルを投影するシステムは,国内外でも本研究が初めてである.また,教育工学の視点から見ると,近年,教育現場にもAR技術が取り入れられている.例えば,英語教育においては,仮想的なキャラクターが英会話をするものや, 理科教育においては,複雑な物理現象を3Dモデルで仮想的に再現するシステムが開発されている.しかしながら,音楽教育の分野にAR技術を取り入れる事例はほとんどなく,本研究の成果が音楽教育現場にもたらす影響は大きい.

Report

(4 results)
  • 2020 Annual Research Report   Final Research Report ( PDF )
  • 2019 Research-status Report
  • 2018 Research-status Report
  • Research Products

    (16 results)

All 2020 2019 2018

All Journal Article (5 results) (of which Peer Reviewed: 5 results,  Open Access: 4 results) Presentation (11 results) (of which Int'l Joint Research: 9 results)

  • [Journal Article] Prototype Digital Signage System for Learning Material Delivery Focusing on Student Friendship2020

    • Author(s)
      Mitsuhara Hiroyuki、Shishibori Masami
    • Journal Title

      International Journal of Emerging Technologies in Learning (iJET)

      Volume: 15 Issue: 22 Pages: 161-161

    • DOI

      10.3991/ijet.v15i22.14171

    • NAID

      120007127452

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Comparative Experiments on Simulated Tornado Experience via Virtual Reality and Augmented Reality2020

    • Author(s)
      Hiroyuki Mitsuhara, Masami Shishibori
    • Journal Title

      The Journal of Information and Systems in Education

      Volume: 19 Issue: 1 Pages: 21-31

    • DOI

      10.12937/ejsise.19.21

    • NAID

      130007868011

    • ISSN
      1348-236X, 2186-3679
    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] The Development of a Hand-Washing Support System Using Image Processing Techniques2020

    • Author(s)
      Katsumi Nagata、Masaki Oono、Masami Shishibori
    • Journal Title

      International Journal of Advanced Intelligence

      Volume: 11 Pages: 1-13

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Comparative Experiments on Mouse-based and Typing-based Copy-and-Paste Methods2018

    • Author(s)
      Hiroyuki Mitsuhara, Masami Shishibori and Akihiro Kashihara
    • Journal Title

      The Journal of Information and Systems in Education

      Volume: 17 Issue: 1 Pages: 17-26

    • DOI

      10.12937/ejsise.17.17

    • NAID

      130006742886

    • ISSN
      1348-236X, 2186-3679
    • Related Report
      2018 Research-status Report
    • Peer Reviewed / Open Access
  • [Journal Article] Investigative Report Writing Support System for Effective Knowledge Construction from the Web2018

    • Author(s)
      Hiroyuki Mitsuhara, Masami Shishibori and Akihiro Kashihara
    • Journal Title

      IEICE Transactions on Information and Systems

      Volume: E101.D Issue: 4 Pages: 874-883

    • DOI

      10.1587/transinf.2016IIP0023

    • NAID

      130006602380

    • ISSN
      0916-8532, 1745-1361
    • Related Report
      2018 Research-status Report
    • Peer Reviewed / Open Access
  • [Presentation] Academic Support for All Students Based on Reasonable Accommodations in Emergency Situations Using AI Chatbots2020

    • Author(s)
      Noboru Ueno, Hiroyuki Mitsuhara and Masami Shishibori
    • Organizer
      Proc. of the 28th International Conference on Computers in Education (ICCE 2020)
    • Related Report
      2020 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Construction of Reasonable Accommodation by Chatbot Using SNS and Operation of an Accessibility Center2020

    • Author(s)
      Noboru Ueno, Hiroyuki Mitsuhara and Masami Shishibori
    • Organizer
      Proc. of the 28th International Conference on Computers in Education (ICCE2020)
    • Related Report
      2020 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Make it Fun: The Application of Gamification in Earthquake Education for Foreigners2020

    • Author(s)
      Liu Meiqin, Hiroyuki Mitsuhara and Masami Shishibori
    • Organizer
      Proc. of the 28th International Conference on Computers in Education (ICCE2020)
    • Related Report
      2020 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 深層学習を用いた尿細胞診断の自動化手法2020

    • Author(s)
      鷲田武晃,尾矢剛志,常山幸一,大野将樹,獅々堀正幹
    • Organizer
      第19回情報科学技術フォーラム(FIT2020)
    • Related Report
      2020 Annual Research Report
  • [Presentation] 卓球動画における深層学習を用いたスローモーションシーン検出手法2020

    • Author(s)
      迫田峻,大野将樹,獅々堀正幹
    • Organizer
      第19回情報科学技術フォーラム(FIT2020)
    • Related Report
      2020 Annual Research Report
  • [Presentation] The Development of a Hand-Washing Education System2019

    • Author(s)
      Katsumi Nagata , Masaki Oono and Masami Shishibori
    • Organizer
      Proc. of the 27th International Conference on Computers in Education (ICCE2019)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] Exploring the Application of ICT base Disaster Education System for Foreigners in Japan2019

    • Author(s)
      Liu Meiqin, Hiroyuki Mitsuhara and Masami Shishibori
    • Organizer
      Proc. of the 27th International Conference on Computers in Education (ICCE2019)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] Evacuation Training Using Scenario-based Augmented Reality Game2019

    • Author(s)
      Hiroyuki Mitsuhara and Masami Shishibori
    • Organizer
      Proc. of International Conference of Virtual and Augmented Reality in Education 2019 (VARE2019)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] Failure-enhanced evacuation training using a VR-based disaster simulator: A comparative experiment with simulated evacuees2019

    • Author(s)
      Hiroyuki Mitsuhara, Chie Tanimura, Junko Nemoto and Masami Shishibori
    • Organizer
      Proc. of 23rd International Conference on Knowledge-Based and Intelligent Information & Engineering Systems
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] Why Dont You Evacuate Speedily? Augmented Reality-based Evacuee Visualisation in ICT-based Evacuation Drill2018

    • Author(s)
      Hiroyuki Mitsuhara, Chie Tanimura, Junko Nemoto and Masami Shishibori
    • Organizer
      Proc.of 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering
    • Related Report
      2018 Research-status Report
    • Int'l Joint Research
  • [Presentation] Evacuation Drill Using Augmented Reality and a Handheld Head-Mounted Display2018

    • Author(s)
      Hiroyuki Mitsuhara, Hiroki IRIE and Masami Shishibori
    • Organizer
      Workshop Proceedings of 26th International Conference on Computers in Education
    • Related Report
      2018 Research-status Report
    • Int'l Joint Research

URL: 

Published: 2018-04-23   Modified: 2022-01-27  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi