360-Degree Camera based Fast Indoor Localization using Image Gradients

2020 Fiscal Year Research-status Report

• Project/Area Number: 20K22383
• Research Institution: The University of Tokyo
• Principal Investigator: パトハック サーサクマヘシ 東京大学, 大学院工学系研究科(工学部), 特任助教 (40815619)
• Project Period (FY): 2020-09-11 – 2023-03-31
• Keywords: 360 camera / localization / image gradients

Outline of Annual Research Achievements

In this year, rotation estimation was achieved using gradient information. Gradient information was plotted in a parametric space of 3D line projections on 360 degree images and the vanishing points were estimated directly.

Moreover, literature research was conducted and the problems of previous research were discovered, i.e. occlusion and line detection errors. A new algorithm for matching image edges and 3D lines was discussed, considering occlusion in the environment. Two steps will be performed i.e. edge-point to 3D line matching, and edge-point to distance calculation. Gradient information will be used to find which edge belongs to which line. Finally, the line map will be reprojected to the image and a distance metric will be minimized to estimate image position.

Current Status of Research Progress

1: Research has progressed more than it was originally planned.

Reason

In this year, the rotation estimation was already achieved using gradient information. The steps remaining are to convert this to orientation estimation, and to estimate position using a 3D line map. For the second step, discussions were done with collaborators and an algorithm has already been designed. This algorithm will work by matching 2D image edge points to a 3D line map using gradient information.

Strategy for Future Research Activity

In this year, the details of the distance metric to match 3D lines to 2D edges using gradient information will be finalized. A preliminary experiment suggests that the designed distance metric is valid. The 3D lines will be reprojected on the images and a novel distance metric will be minimized to make sure that the image lines and 3D edges match. Localization will be performed first in a simulated environment, followed by a real environment.

Causes of Carryover

In the previous year, most of the work was in literature review and formulating a design for the algorithm to match 3D lines to 2D image edges. This year, the algorithm will be evaluated via experiments performed in simulation environments and real environments. For this, computational hardware as well as cameras will be required.