• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Multimodal Interaction Environment Based on High-Speed Projection

Research Project

Project/Area Number 16H02853
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeSingle-year Grants
Section一般
Research Field Human interface and interaction
Research InstitutionTohoku University

Principal Investigator

Kagami Shingo  東北大学, 情報科学研究科, 准教授 (90361542)

Co-Investigator(Kenkyū-buntansha) 嵯峨 智  熊本大学, 大学院先端科学研究部(工), 准教授 (10451535)
Project Period (FY) 2016-04-01 – 2019-03-31
Project Status Completed (Fiscal Year 2019)
Budget Amount *help
¥16,250,000 (Direct Cost: ¥12,500,000、Indirect Cost: ¥3,750,000)
Fiscal Year 2018: ¥3,900,000 (Direct Cost: ¥3,000,000、Indirect Cost: ¥900,000)
Fiscal Year 2017: ¥5,980,000 (Direct Cost: ¥4,600,000、Indirect Cost: ¥1,380,000)
Fiscal Year 2016: ¥6,370,000 (Direct Cost: ¥4,900,000、Indirect Cost: ¥1,470,000)
Keywordsユーザインタフェース / マルチモーダルインタフェース / 画像,文章,音声等認識 / バーチャルリアリティ / ディスプレイ / ユーザインターフェース / マルチモーダルインターフェース / 画像、文章、音声等認識
Outline of Final Research Achievements

This study investigated projection-based augmented reality environments that make use of high-speed optical projection to produce multimodal sensation and to enable multimodal detection. Namely, we developed an interactive projection mapping system for a fast moving surface, optical touch detection systems for remote surfaces, projection-based tactile augmentation, and a thermal-radiation-based haptic display utilizing a scanned laser beam.

Academic Significance and Societal Importance of the Research Achievements

光学投影技術に基づく映像提示,触覚提示,タッチ操作検出のための新しい技術の開発に成功した.映像提示技術については,映像表示用の投影光が画像追跡に干渉するという積年の課題を解決し,事前調整やマーカ配置の不要な高精度・高速プロジェクションマッピングを実現した.さらに,レーザー光投影に基づく遠隔タッチ操作検出や空間触覚提示といった新技術の可能性を切り拓いた.

Report

(4 results)
  • 2019 Final Research Report ( PDF )
  • 2018 Annual Research Report
  • 2017 Annual Research Report
  • 2016 Annual Research Report
  • Research Products

    (49 results)

All 2020 2019 2018 2017 2016

All Journal Article (16 results) (of which Peer Reviewed: 15 results,  Open Access: 1 results,  Acknowledgement Compliant: 1 results) Presentation (32 results) (of which Int'l Joint Research: 6 results,  Invited: 2 results) Patent(Industrial Property Rights) (1 results)

  • [Journal Article] Interactive Stickies: Low-Latency Projection Mapping for Dynamic Interaction with Projected Images on a Movable Surface2020

    • Author(s)
      Shingo Kagami, Koichi Hashimoto
    • Journal Title

      ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques (SIGGRAPH 2020 Emerging Technologies)

      Volume: -

    • Related Report
      2018 Annual Research Report
    • Peer Reviewed
  • [Journal Article] DMD プロジェクタを用いた低遅延な運動追従投影手法2020

    • Author(s)
      大城 和可菜, 鏡 慎吾, 橋本 浩一
    • Journal Title

      日本バーチャルリアリティ学会論文誌

      Volume: 25

    • NAID

      130007866522

    • Related Report
      2018 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Perception of Motion-Adaptive Color Images Displayed by a High-Speed DMD Projector2019

    • Author(s)
      Oshiro Wakana、Kagami Shingo、Hashimoto Koichi
    • Journal Title

      Proc. 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR 2019 Workshop on Perception-driven Graphics and Displays)

      Volume: - Pages: 1790-1793

    • DOI

      10.1109/vr.2019.8797850

    • Related Report
      2018 Annual Research Report 2017 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Puppeteered rain2019

    • Author(s)
      Kagami Shingo、Higuchi Kotone、Hashimoto Koichi
    • Journal Title

      Proc. ACM SIGGRAPH Conference on Computer Graphics and Interactive Techniques (SIGGARPH 2019 Posters)

      Volume: - Pages: 48-48

    • DOI

      10.1145/3306214.3338603

    • Related Report
      2018 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Animated Stickies: Fast Video Projection Mapping onto a Markerless Plane through a Direct Closed-Loop Alignment2019

    • Author(s)
      Kagami Shingo、Hashimoto Koichi
    • Journal Title

      IEEE Transactions on Visualization and Computer Graphics

      Volume: 25 Issue: 11 Pages: 3094-3104

    • DOI

      10.1109/tvcg.2019.2932248

    • Related Report
      2018 Annual Research Report
    • Peer Reviewed
  • [Journal Article] A full-color single-chip-DLP projector with an embedded 2400-fps homography warping engine2018

    • Author(s)
      Kagami Shingo、Hashimoto Koichi
    • Journal Title

      ACM SIGGRAPH 2018 Emerging Technologies, SIGGRAPH 2018

      Volume: - Pages: 1-2

    • DOI

      10.1145/3214907.3214927

    • Related Report
      2018 Annual Research Report 2017 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Thermal-Radiation-Based Haptic Display - Laser-Emission-Based Radiation System -2018

    • Author(s)
      Satoshi Saga
    • Journal Title

      Lecture Notes in Electrical Engineering (AsiaHaptics 2018)

      Volume: 535 Pages: 196-197

    • DOI

      10.1007/978-981-13-3194-7_43

    • ISBN
      9789811331930, 9789811331947
    • Related Report
      2018 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Children's Social Behavior Analysis System Using BLE and Accelerometer2017

    • Author(s)
      Shuta Nakamae, Shumpei Kataoka, Can Tang, Simona Vasilache, Satoshi Saga, Buntarou Shizuki, Shin Takahashi
    • Journal Title

      International Conference on Collaboration Technologies

      Volume: - Pages: 153-167

    • DOI

      10.1007/978-3-319-63088-5_14

    • ISBN
      9783319630878, 9783319630885
    • Related Report
      2017 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Synchronization-free single-shot detection of imperceptible AR markers in a high-speed video display2016

    • Author(s)
      Tsutomu Kusanagi, Shingo Kagami and Koichi Hashimoto
    • Journal Title

      3rd International Workshop on Image Sensors and Imaging Systems

      Volume: - Pages: 229-234

    • DOI

      10.1109/ismar-adjunct.2017.75

    • NAID

      130007742445

    • Related Report
      2017 Annual Research Report 2016 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Alignment of a flexible sheet object with position-based and image-based visual servoing2016

    • Author(s)
      Shingo Kagami, Kotaro Omi, Koichi Hashimoto
    • Journal Title

      Advanced Robotics

      Volume: 30 Issue: 15 Pages: 965-978

    • DOI

      10.1080/01691864.2016.1183518

    • Related Report
      2016 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Display tracking using blended images with unknown mixing ratio as a template2016

    • Author(s)
      Akifumi Goto, Shingo Kagami, Koichi Hashimoto
    • Journal Title

      Proc. 9th ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia, Technical Briefs

      Volume: - Pages: 1-3

    • DOI

      10.1145/2818466.2818485

    • Related Report
      2016 Annual Research Report
    • Peer Reviewed
  • [Journal Article] High-frame-rate region-based visual tracking on CPU: an implementation perspective2016

    • Author(s)
      Shingo Kagami, Koichi Hashimoto
    • Journal Title

      Proc. 2016 IEEE/SICE International Symposium on System Integration

      Volume: - Pages: 562-567

    • DOI

      10.1109/sii.2016.7844058

    • Related Report
      2016 Annual Research Report
    • Peer Reviewed / Acknowledgement Compliant
  • [Journal Article] Estimation of a large relative rotation between two images of a fast spinning marker-less golf ball2016

    • Author(s)
      Yuta Imai, Shingo Kagami, Koichi Hashimoto
    • Journal Title

      Proc. 2016 IEEE/SICE International Symposium on System Integration

      Volume: - Pages: 556-561

    • DOI

      10.1109/sii.2016.7844057

    • Related Report
      2016 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Fast projection mapping onto a moving planar surface2016

    • Author(s)
      Shingo Kagami
    • Journal Title

      SPIE Newsroom

      Volume: -

    • DOI

      10.1117/2.1201603.006423

    • Related Report
      2016 Annual Research Report
  • [Journal Article] Thermal-Radiation-Based Haptic Display - Calibration and Shape display -2016

    • Author(s)
      Satoshi Saga
    • Journal Title

      Proc. AsiaHaptics 2016

      Volume: - Pages: 67-70

    • DOI

      10.1007/978-981-10-4157-0_11

    • ISBN
      9789811041563, 9789811041570
    • Related Report
      2016 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Calibration Method of Thermal-Radiation-Based Haptic Display2016

    • Author(s)
      Satoshi Saga
    • Journal Title

      Proc. EuroHaptics 2016

      Volume: - Pages: 470-478

    • DOI

      10.1007/978-3-319-42324-1_46

    • NAID

      130007224602

    • ISBN
      9783319423234, 9783319423241
    • Related Report
      2016 Annual Research Report
    • Peer Reviewed
  • [Presentation] 運動追従プロジェクションマッピングのためのDMDプロジェクタの色表示技術2020

    • Author(s)
      鏡 慎吾, 大城 和可菜, 宮口 綾乃, 橋本 浩一
    • Organizer
      第40回レーザー学会学術講演会年次大会
    • Related Report
      2018 Annual Research Report
  • [Presentation] 運動物体三次元計測のための高速縞プロジェクタのシステム実装2020

    • Author(s)
      長沼 朋哉, 鏡 慎吾, 橋本 浩一
    • Organizer
      第40回レーザー学会学術講演会年次大会
    • Related Report
      2018 Annual Research Report
  • [Presentation] Vision-based Guitar Chord Recognition Using Convolutional Neural Network2019

    • Author(s)
      Yaoyi Bian, Shingo Kagami, Koichi Hashimoto
    • Organizer
      第13回新画像システム・情報フォトニクス研究討論会
    • Related Report
      2018 Annual Research Report
  • [Presentation] 運動物体三次元計測のための高速縞プロジェクタの光学系2019

    • Author(s)
      長沼 朋哉,鏡 慎吾,橋本 浩一
    • Organizer
      第13回新画像システム・情報フォトニクス研究討論会
    • Related Report
      2018 Annual Research Report
  • [Presentation] レーザを光源とする縞パターン高速投影プロジェクタを用いた動物体三次元計測システム2019

    • Author(s)
      長沼 朋哉,鏡 慎吾,橋本 浩一
    • Organizer
      第25回画像センシングシンポジウム
    • Related Report
      2018 Annual Research Report
  • [Presentation] DMDプロジェクタにおける運動追従多値画像表示のためのバイナリフレーム更新手法の比較検討2019

    • Author(s)
      大城 和可菜, 鏡 慎吾, 橋本 浩一
    • Organizer
      第24回バーチャルリアリティ学会大会
    • Related Report
      2018 Annual Research Report
  • [Presentation] ハンドヘルド高速プロジェクタを利用した大画面ディスプレイとの低遅延なインタラクション2019

    • Author(s)
      吉川 太陽, 鏡 慎吾, 橋本 浩一
    • Organizer
      エンタテインメントコンピューティングシンポジウム2019
    • Related Report
      2018 Annual Research Report
  • [Presentation] Thermal-Radiation-Based Haptic Display - Laser-Emission-Based Radiation Control -2019

    • Author(s)
      Satoshi Saga
    • Organizer
      WorldHaptics 2019
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] A high-pattern-rate photostimulation system capable of low-latency geometrical control for bio-navigation studies2018

    • Author(s)
      Shingo Kagami, Koichi Hashimoto
    • Organizer
      International Symposium on System Science of Bio-Navigation
    • Related Report
      2018 Annual Research Report 2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 高速カメラを用いたタッチ検出のための振動解析2018

    • Author(s)
      成田 正太郎, 鏡 慎吾, 橋本 浩一
    • Organizer
      日本機械学会ロボティクス・メカトロニクス講演会
    • Related Report
      2018 Annual Research Report 2017 Annual Research Report
  • [Presentation] フルカラー高速ホモグラフィ変換プロジェクタ2018

    • Author(s)
      鏡 慎吾, 橋本 浩一
    • Organizer
      日本機械学会ロボティクス・メカトロニクス講演会
    • Related Report
      2018 Annual Research Report 2017 Annual Research Report
  • [Presentation] DMDを用いた三次元計測のための濃淡縞パターン高速生成2018

    • Author(s)
      長沼 朋哉, 鏡 慎吾, 橋本 浩一
    • Organizer
      画像センシングシンポジウム
    • Related Report
      2018 Annual Research Report 2017 Annual Research Report
  • [Presentation] 投影型バーチャルエアホッケーにおける打突感提示のための視覚的刺激の検討2018

    • Author(s)
      島森 洸, 鏡 慎吾, 橋本 浩一
    • Organizer
      エンタテインメントコンピューティングシンポジウム
    • Related Report
      2018 Annual Research Report 2017 Annual Research Report
  • [Presentation] 高速DMDプロジェクタを用いた運動追従多値画像の知覚の評価2018

    • Author(s)
      大城 和可菜, 鏡 慎吾, 橋本 浩一
    • Organizer
      日本バーチャルリアリティ学会大会
    • Related Report
      2018 Annual Research Report 2017 Annual Research Report
  • [Presentation] 機械学習による高速カメラ映像からのタッチイベント検出2018

    • Author(s)
      成田 正太郎, 鏡 慎吾, 橋本 浩一
    • Organizer
      計測自動制御学会システムインテグレーション部門講演会
    • Related Report
      2018 Annual Research Report 2017 Annual Research Report
  • [Presentation] 空間積分光学系を備える DMD プロジェクタを用いた3次元計測2018

    • Author(s)
      長沼 朋哉, 鏡 慎吾, 橋本 浩一
    • Organizer
      計測自動制御学会システムインテグレーション部門講演会
    • Related Report
      2018 Annual Research Report 2017 Annual Research Report
  • [Presentation] Thermal-Radiation-Based Haptic Display - Position and Radiation Intensity Calibration -2018

    • Author(s)
      Satoshi Saga
    • Organizer
      EuroHaptics 2018
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 動きをとらえる高フレームレート画像処理とその応用2017

    • Author(s)
      鏡 慎吾
    • Organizer
      第6回バイオメトリクスと認識・認証シンポジウム
    • Related Report
      2017 Annual Research Report 2016 Annual Research Report
    • Invited
  • [Presentation] バーチャルプロジェクション型インタラクションシステムの高速化2017

    • Author(s)
      乙部成史, 鏡慎吾, 橋本浩一
    • Organizer
      日本機械学会ロボティクス・メカトロニクス講演会
    • Related Report
      2017 Annual Research Report 2016 Annual Research Report
  • [Presentation] 高速三次元計測のための合成正弦波を用いた位相シフト法2017

    • Author(s)
      塩谷美帆, 鏡慎吾, 橋本浩一
    • Organizer
      画像センシングシンポジウム
    • Related Report
      2017 Annual Research Report 2016 Annual Research Report
  • [Presentation] weAtheR: AR技術を用いたアンビエントな天気情報の提示2017

    • Author(s)
      成田正太郎, 鏡慎吾, 橋本浩一
    • Organizer
      エンタテインメントコンピューティングシンポジウム
    • Related Report
      2017 Annual Research Report 2016 Annual Research Report
  • [Presentation] 高速プロジェクタによる投影コンテンツとの動的インタラクション2017

    • Author(s)
      鏡慎吾, 橋本浩一
    • Organizer
      エンタテインメントコンピューティングシンポジウム
    • Related Report
      2017 Annual Research Report 2016 Annual Research Report
  • [Presentation] 高速ディスプレイ映像に隠蔽されたマーカを用いるモバイルARシステム2017

    • Author(s)
      草彅 勉, 鏡 慎吾, 橋本 浩一
    • Organizer
      計測自動制御学会システムインテグレーション部門講演会
    • Related Report
      2017 Annual Research Report 2016 Annual Research Report
  • [Presentation] Thermal-Radiation-Based Haptic Display - Precise Calibration System and Shape Displaying Method -2017

    • Author(s)
      Satoshi Saga
    • Organizer
      IEEE World Haptics 2017
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Image and Graphics Processing for Fast Projection Mapping onto a Moving Surface2016

    • Author(s)
      Shingo Kagami
    • Organizer
      31th International Congress on High-Speed Imaging and Photonics
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 加法合成画像の追跡に基づく動く平面への追従投影2016

    • Author(s)
      後藤 皓史, 鏡 慎吾, 橋本 浩一
    • Organizer
      日本ロボット学会学術講演会
    • Related Report
      2016 Annual Research Report
  • [Presentation] 領域ベース平面追跡の高速計算2016

    • Author(s)
      鏡 慎吾, 橋本 浩一
    • Organizer
      計測自動制御学会システムインテグレーション部門講演会
    • Related Report
      2016 Annual Research Report
  • [Presentation] 高速ディスプレイ映像への埋め込みに適した不可視マーカの検討2016

    • Author(s)
      草彅 勉, 鏡 慎吾, 橋本 浩一
    • Organizer
      日本光学会年次学術講演会
    • Related Report
      2016 Annual Research Report
  • [Presentation] Synchronization-free single-shot detection of imperceptible AR markers in a high-speed video display2016

    • Author(s)
      Tsutomu Kusanagi, Shingo Kagami, Koichi Hashimoto
    • Organizer
      3rd International Workshop on Image Sensors and Imaging Systems
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 高フレームレート画像追跡とその応用2016

    • Author(s)
      鏡 慎吾
    • Organizer
      パーティクルフィルタ研究会
    • Related Report
      2016 Annual Research Report
    • Invited
  • [Presentation] ディスプレイ映像を基準としたモバイルARのためのマーカ隠蔽手法2016

    • Author(s)
      草彅 勉, 鏡 慎吾, 橋本 浩一
    • Organizer
      日本機械学会ロボティクス・メカトロニクス講演会
    • Related Report
      2016 Annual Research Report
  • [Presentation] 未知混合比の加法合成画像の追跡2016

    • Author(s)
      後藤 皓史, 鏡 慎吾, 橋本 浩一
    • Organizer
      日本機械学会ロボティクス・メカトロニクス講演会
    • Related Report
      2016 Annual Research Report
  • [Patent(Industrial Property Rights)] 投影システム、投影システム制御装置、投影方法及びプログラム2019

    • Inventor(s)
      鏡 慎吾
    • Industrial Property Rights Holder
      国立大学法人東北大学
    • Industrial Property Rights Type
      特許
    • Industrial Property Number
      2019-156876
    • Filing Date
      2019
    • Related Report
      2018 Annual Research Report

URL: 

Published: 2016-04-21   Modified: 2022-01-27  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi