• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Development of See-Through Mixed Reality Mobility System

Research Project

Project/Area Number 16H05864
Research Category

Grant-in-Aid for Young Scientists (A)

Allocation TypeSingle-year Grants
Research Field Perceptual information processing
Research InstitutionThe University of Tokyo

Principal Investigator

Oishi Takeshi  東京大学, 生産技術研究所, 准教授 (80569509)

Project Period (FY) 2016-04-01 – 2020-03-31
Project Status Completed (Fiscal Year 2019)
Budget Amount *help
¥24,050,000 (Direct Cost: ¥18,500,000、Indirect Cost: ¥5,550,000)
Fiscal Year 2019: ¥2,860,000 (Direct Cost: ¥2,200,000、Indirect Cost: ¥660,000)
Fiscal Year 2018: ¥3,510,000 (Direct Cost: ¥2,700,000、Indirect Cost: ¥810,000)
Fiscal Year 2017: ¥5,980,000 (Direct Cost: ¥4,600,000、Indirect Cost: ¥1,380,000)
Fiscal Year 2016: ¥11,700,000 (Direct Cost: ¥9,000,000、Indirect Cost: ¥2,700,000)
Keywords複合現実感 / 3次元ビジョン / 視覚情報処理 / 視覚メディア処理 / コンピュータビジョン
Outline of Final Research Achievements

In this research project, we have evaluated human visual perception in Mixed Reality (MR) and developed related technologies to realize a mobility MR system to enable a large number of people to experience MR at the same time. We have clarified the relationship between occlusion and depth perception and the visibility of virtual objects, and have developed methods to improve the visibility and to handle occlusion by using the characteristics. We have also developed robust and accurate pose estimation methods for mobility using 3D data and omni-directional images, and have realized a mobility MR system using the optical see-through (OST) HMD.

Academic Significance and Societal Importance of the Research Achievements

MRにおける既存問題の解決だけでなく、仮想物体の視認性を評価することによって人間の視覚特性を明らかにするとともに、それを利用した技術を開発しており、複数の研究分野に貢献している。また3次元ビジョン技術とMRモビリティシステムの開発により、これまでは個人や狭い範囲にとどまっていたMRの利用を拡大させ、今後のMRの可能性を広げる研究となっており社会的な意義も大きいと考えられる。

Report

(5 results)
  • 2019 Annual Research Report   Final Research Report ( PDF )
  • 2018 Annual Research Report
  • 2017 Annual Research Report
  • 2016 Annual Research Report
  • Research Products

    (36 results)

All 2019 2018 2017 2016 Other

All Int'l Joint Research (2 results) Journal Article (2 results) (of which Peer Reviewed: 2 results,  Open Access: 1 results) Presentation (32 results) (of which Int'l Joint Research: 20 results,  Invited: 6 results)

  • [Int'l Joint Research] Microsoft Research(米国)

    • Related Report
      2019 Annual Research Report
  • [Int'l Joint Research] Universite de Picardie Jules Verne(フランス)

    • Related Report
      2018 Annual Research Report
  • [Journal Article] Real-Time Dense Depth Estimation Using Semantically-Guided LIDAR Data Propagation and Motion Stereo2019

    • Author(s)
      Hirata Atsuki、Ishikawa Ryoichi、Roxas Menandro、Oishi Takeshi
    • Journal Title

      IEEE Robotics and Automation Letters

      Volume: 4 Issue: 4 Pages: 3806-3811

    • DOI

      10.1109/lra.2019.2927126

    • Related Report
      2019 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Real-time rendering of aerial perspective effect based on turbidity estimation2017

    • Author(s)
      Morales Carlos、Oishi Takeshi、Ikeuchi Katsushi
    • Journal Title

      IPSJ Transactions on Computer Vision and Applications

      Volume: 9 Issue: 1

    • DOI

      10.1186/s41074-016-0012-1

    • Related Report
      2016 Annual Research Report
    • Peer Reviewed / Open Access
  • [Presentation] Dynamic Calibration between a Mobile Robot and SLAM Device for Navigation2019

    • Author(s)
      R. Ishikawa, T. Oishi, K. Ikeuchi
    • Organizer
      The 28th IEEE International Conference on Robot and Human Interactive Communication
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Real-Time Dense Depth Estimation Using Semantically-Guided LIDAR Data Propagation and Motion Stereo2019

    • Author(s)
      A. Hirata, R. Ishikawa, M. Roxas and T. Oishi
    • Organizer
      The 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Accurate Visual Localization for Vehicles using Effective Feature Points in Dense 3D Ma2019

    • Author(s)
      Y. Ike, K. Fujimoto, M. Roxas, Y. Okamoto, T. Oishi
    • Organizer
      第22回 画像の認識・理解シンポジウム
    • Related Report
      2019 Annual Research Report
  • [Presentation] LIDAR Upsampling Based on Semantically-Guided Propagation2019

    • Author(s)
      A. Hirata, R. Ishikawa, M. Roxas, T. Oishi
    • Organizer
      第22回 画像の認識・理解シンポジウム
    • Related Report
      2019 Annual Research Report
  • [Presentation] A profiler-camera fusion scanning system with direct based motion and calibration parameter correction2019

    • Author(s)
      Ryoichi Ishikawa, Yoshihiro Sato, Takeshi Oishi, Katsushi Ikeuchi
    • Organizer
      第22回 画像の認識・理解シンポジウム
    • Related Report
      2019 Annual Research Report
  • [Presentation] Cyber-Archaeology: 3D Scanning and Visualization of Historical Sites through Mobility Systems2019

    • Author(s)
      Takeshi Oishi
    • Organizer
      Japan-Korea Workshop on Next Generation Robotics
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] Occlusion Handling using Semantic Segmentation and Visibility-Based Rendering for Mixed Reality2019

    • Author(s)
      M. Roxas, T. Hori, T. Fukiage, Y. Okamoto, T. Oishi
    • Organizer
      The 13th International Workshop on Robust Computer Vision
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Interpolation and Extrapolation of LiDAR Data using Camera Images2019

    • Author(s)
      A. Hirata, R. Ishikawa, M. Roxas, T. Oishi
    • Organizer
      The 13th International Workshop on Robust Computer Vision
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Lidar and Camera Calibration using Motions Estimated by Sensor Fusion Odometry2019

    • Author(s)
      R. Ishikawa, T. Oishi, K. Ikeuchi
    • Organizer
      The 13th International Workshop on Robust Computer Vision
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Occlusion Handling using Semantic Segmentation and Visibility-Based Rendering for Mixed Reality2018

    • Author(s)
      M. Roxas, T. Hori, T. Fukiage, Y. Okamoto, T. Oishi
    • Organizer
      24th ACM Symposium on Virtual Reality Software and Technology
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] LiDAR and Camera Calibration using Motion Estimated by Sensor Fusion Odometry2018

    • Author(s)
      R. Ishikawa, T. Oishi, K. Ikeuchi
    • Organizer
      IEEE/RSJ International Conference on Intelligent Robots and Systems
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 3D Vision for Mobility2018

    • Author(s)
      T. Oishi
    • Organizer
      2nd International MIS workshop on 3D Vision and Robotics
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] Occlusion Handling using Semantic Segmentation and Visibility-Based Rendering for Augmented Reality2018

    • Author(s)
      M. Roxas, T. Hori, T. Fukiage, Y. Okamoto, T. Oishi
    • Organizer
      第21回 画像の認識・理解シンポジウム
    • Related Report
      2018 Annual Research Report
  • [Presentation] センサフュージョンによるオドメトリを用いたLiDARとカメラのキャリブレーション2018

    • Author(s)
      石川涼一, 大石岳史, 池内克史
    • Organizer
      第21回 画像の認識・理解シンポジウム
    • Related Report
      2018 Annual Research Report
  • [Presentation] Real-Time Simultaneous 3D Reconstruction and Optical Flow Estimation2018

    • Author(s)
      M. Roxas, T. Oishi
    • Organizer
      IEEE Winter Conf. on Applications of Computer Vision
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Real-Time Simultaneous 3D Reconstruction and Optical Flow Estimation2018

    • Author(s)
      M. Roxas, T. Oishi
    • Organizer
      The 12th International Workshop on Robust Computer Vision
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Robust Visual Localization using Effective Feature Points in Dense 3D Map2018

    • Author(s)
      Y. Ike, R. Menandro, Y. Okamoto, T. Oishi
    • Organizer
      The 12th International Workshop on Robust Computer Vision
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Geometrically and Optically Robust Optical See-Through Mixed Reality System with Eye Tracking Techniques2018

    • Author(s)
      K. Hasegawa, Y. Okamoto, T. Oishi, T. Fukiage
    • Organizer
      The 12th International Workshop on Robust Computer Vision
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Accurate Position and Pose Estimation for Mobile Platforms using Dense 3D Model2017

    • Author(s)
      T. Oishi
    • Organizer
      Korea-Japan Workshop on Robotics and Information Technology for Better Quality of Life
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 視線追跡技術による幾何・光学的に安定な透過型複合現実感システムの開発2017

    • Author(s)
      長谷川健太, 吹上大樹, 岡本泰英, 大石岳史
    • Organizer
      第53回 複合現実感研究会
    • Related Report
      2017 Annual Research Report
  • [Presentation] 追跡に効果的な特徴量による頑健な位置合わせ2017

    • Author(s)
      池友太, 岡本泰英, 大石岳史
    • Organizer
      第53回 複合現実感研究会
    • Related Report
      2017 Annual Research Report
  • [Presentation] e-Heritageにおける「ホンモノ」と「ニセモノ」2017

    • Author(s)
      大石岳史
    • Organizer
      文化資源学フォーラム
    • Related Report
      2016 Annual Research Report
    • Invited
  • [Presentation] Outdoor Omnidirectional Video Completion via Depth Estimation by Motion Analysis2016

    • Author(s)
      C. Morales, S. Ono, Y. Okamoto, M. Roxas, T. Oishi and K. Ikeuchi
    • Organizer
      Proc. International Conference on Pattern Recognition
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Variational Optical Flow with 3D Smoothness Constraint for a Single Moving Camera2016

    • Author(s)
      M. Roxas, T. Hori, and T. Oishi
    • Organizer
      The 11th International Workshop on Robust Computer Vision
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Panoramic Video Completion from Depth Recovery by Pixel Motion Assessment2016

    • Author(s)
      C. Morales, S. Ono, Y. Okamoto, M. Roxas, T. Oishi and K. Ikeuchi
    • Organizer
      The 11th International Workshop on Robust Computer Vision
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Occlusion Handling by Semantic Segmentation and Transparency Blending for Outdoor Mixed Reality2016

    • Author(s)
      T. Hori, T. Fukiage, R. Menandro, Y. Okamato and T. Oishi
    • Organizer
      The 11th International Workshop on Robust Computer Vision
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Robust Motion Estimation for MR Mobility System with Multiple Panoramic Cameras2016

    • Author(s)
      Y. Okamoto, K. Fujimoto, T. Oishi and K. Ikeuchi
    • Organizer
      The 11th International Workshop on Robust Computer Vision
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 車両型MR のための複数全方位カメラを利用した頑健な位置合わせ2016

    • Author(s)
      藤本浩介, 岡本泰英, 大石岳史, 池内克史
    • Organizer
      第14回ITSシンポジウム
    • Related Report
      2016 Annual Research Report
  • [Presentation] MR環境下における遮蔽矛盾解消のための前景推定手法2016

    • Author(s)
      堀智貴, 吹上大樹, 岡本泰英, Roxas Menandro, 大石岳史
    • Organizer
      第50回 複合現実感研究会
    • Related Report
      2016 Annual Research Report
  • [Presentation] Laser Scanning Systems on Mobile Platforms2016

    • Author(s)
      Takeshi Oishi
    • Organizer
      2016 Korea-Japan Workshop on Robotics and Information Technology for Better Quality of Life
    • Related Report
      2016 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] 3D e-Heritage 大規模文化財の3次元デジタル化、解析、展示2016

    • Author(s)
      大石岳史
    • Organizer
      産業総合技術研究所コンピュータビジョン勉強会
    • Related Report
      2016 Annual Research Report
    • Invited
  • [Presentation] 3D e-Heritage 大規模文化財の3次元デジタル化、解析、展示2016

    • Author(s)
      大石岳史
    • Organizer
      日本ロボット学会 ロボット工学セミナー
    • Related Report
      2016 Annual Research Report
    • Invited

URL: 

Published: 2016-04-21   Modified: 2021-02-19  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi