2020 Fiscal Year Final Research Report
Unifying multiple RGB and depth cameras for real-time large-scale dynamic 3D modeling with unmanned micro aerial vehicles
| Project/Area Number |
19K20297
|
|---|
| Research Category |
Grant-in-Aid for Early-Career Scientists
|
| Allocation Type | Multi-year Fund |
|---|
| Review Section |
Basic Section 61010:Perceptual information processing-related
|
|---|
| Research Institution | Kyushu University |
|---|
Principal Investigator |
Thomas Diego 九州大学, システム情報科学研究院, 助教 (10804651)
|
|---|
| Project Period (FY) |
2019-04-01 – 2021-03-31
|
| Keywords | RGB-D SLAM / Aerial 3D capture / Dynamic scene / Aritificial intelligence / Deep neural network / 3D human body |
|---|
| Outline of Final Research Achievements |
The goal of this project was to develop a real-time system for 3D reconstruction of dynamic scenes with unmanned micro aerial vehicles.
We proposed a new method for robust and accurate fusion of depth images without increasing computational speed. We also proposed a method that can handle dynamic scenes like a moving person. This was achieved by jointly optimizing non rigid motion and geometry. To handle situations when depth information is not available, we proposed a solution for 3D shape estimation from a single RGB image. We focused on the case of the human body and proposed a new deep neural network to reconstruct detailed shapes of humans wearing loose clothes from single RGB images.
We proposed a new 3D scanning system equipped on a consumer-grade aerial drone that can capture live sequences of RGB-D data. our proposed system consists of a minicomputer powered by a portable battery and an RGB-D camera. We shared material and code and captured real world data with our system.
|
| Free Research Field |
Computer Vision
|
| Academic Significance and Societal Importance of the Research Achievements |
単一の画像から人体の緻密で詳細な3Dモデルを作成する最初の方法の1つを提案しました。 コンピュータビジョンのトップ国際会議で私たちの仕事を発表し、コードを一般に公開しました。 制御されていない環境での3Dシーン再構築の新しい可能性を開きます。
このシステムを消費者向けの空中ドローンに簡単に装備するためのソリューションと、それを制御するソフトウェアを紹介します。 実世界のデータをキャプチャし、システムを使用していくつかの最先端のRGB-DSLAM技術を評価しました。 空中3Dスキャンとマッピングの研究開発を後押しできるように、すべてのデータとコードをコミュニティで公開しました。
|
