Unifying multiple RGB and depth cameras for real-time large-scale dynamic 3D modeling with unmanned micro aerial vehicles
| Project/Area Number |
19K20297
|
|---|
| Research Category |
Grant-in-Aid for Early-Career Scientists
|
| Allocation Type | Multi-year Fund |
|---|
| Review Section |
Basic Section 61010:Perceptual information processing-related
|
|---|
| Research Institution | Kyushu University |
|---|
Principal Investigator |
Thomas Diego 九州大学, システム情報科学研究院, 助教 (10804651)
|
|---|
| Project Period (FY) |
2019-04-01 – 2021-03-31
|
| Project Status |
Completed (Fiscal Year 2020)
|
| Budget Amount *help |
¥4,160,000 (Direct Cost: ¥3,200,000、Indirect Cost: ¥960,000)
Fiscal Year 2020: ¥1,430,000 (Direct Cost: ¥1,100,000、Indirect Cost: ¥330,000)
Fiscal Year 2019: ¥2,730,000 (Direct Cost: ¥2,100,000、Indirect Cost: ¥630,000)
|
|---|
| Keywords | RGB-D SLAM / Aerial 3D capture / Dynamic scene / Aritificial intelligence / Deep neural network / 3D human body / Aerial drone / outdoor scene / 3D reconstruction / sensor fusion / RGB―D SLAM / Unmanned vehicle / dynamic modeling / real-time / unmanned vehicle |
|---|
| Outline of Research at the Start |
Various sensors from unmanned aerial vehicles will be unified to reconstruct large-scale outdoor dynamic 3D scenes in real-time.
(1)Compact RGB and depth cameras will be fused to incrementally reconstruct outdoor scenes in a coarse-to-fine strategy.
(2)NURBS surfaces will maintain the large-scale evolving 3D model while displacement mapping will generate detailed 3D geometry.
(3)Physical and semantic constraints in the scene will be used to enforce at all time a globally consistent 3D reconstruction with loops closed and minimal registration errors.
|
| Outline of Final Research Achievements |
The goal of this project was to develop a real-time system for 3D reconstruction of dynamic scenes with unmanned micro aerial vehicles.
We proposed a new method for robust and accurate fusion of depth images without increasing computational speed. We also proposed a method that can handle dynamic scenes like a moving person. This was achieved by jointly optimizing non rigid motion and geometry. To handle situations when depth information is not available, we proposed a solution for 3D shape estimation from a single RGB image. We focused on the case of the human body and proposed a new deep neural network to reconstruct detailed shapes of humans wearing loose clothes from single RGB images.
We proposed a new 3D scanning system equipped on a consumer-grade aerial drone that can capture live sequences of RGB-D data. our proposed system consists of a minicomputer powered by a portable battery and an RGB-D camera. We shared material and code and captured real world data with our system.
|
| Academic Significance and Societal Importance of the Research Achievements |
単一の画像から人体の緻密で詳細な3Dモデルを作成する最初の方法の1つを提案しました。 コンピュータビジョンのトップ国際会議で私たちの仕事を発表し、コードを一般に公開しました。 制御されていない環境での3Dシーン再構築の新しい可能性を開きます。
このシステムを消費者向けの空中ドローンに簡単に装備するためのソリューションと、それを制御するソフトウェアを紹介します。 実世界のデータをキャプチャし、システムを使用していくつかの最先端のRGB-DSLAM技術を評価しました。 空中3Dスキャンとマッピングの研究開発を後押しできるように、すべてのデータとコードをコミュニティで公開しました。
|
Report
(3 results)
Research Products
(11 results)
