Interpolation of vegetation remote sensing images and anomaly detection using deep learning image generation technology

2022 Fiscal Year Final Research Report

• Project/Area Number: 20K21345
• Research Category: Grant-in-Aid for Challenging Research (Exploratory)
• Allocation Type: Multi-year Fund
• Review Section: Medium-sized Section 41:Agricultural economics and rural sociology, agricultural engineering, and related fields
• Research Institution: The University of Tokyo
• Principal Investigator: Hosoi Fumiki 東京大学, 大学院農学生命科学研究科(農学部), 准教授 (80526468)
• Project Period (FY): 2020-07-30 – 2023-03-31
• Keywords: 深層学習 / 樹木 / ３次元点群画像 / ライダー

Outline of Final Research Achievements

There is a problem in remote sensing of vegetation where information is missing in the blind spots of the sensor, such as inside of vegetation and shadows, which is a big problem, especially for obtaining 3D vegetation information. To solve this problem, in this research, we attempted to interpolate the missing information using GAN (Generative Adversarial Net-work), a deep learning technique. We also introduced VAE (Variational Auto Encoder) for detection of vegetation diseases and pests. Although these methods are currently undergoing trial and error, we were able to develop methods that enables highly accurate separation by using deep learning and point cloud features to identify individual trees and separate each organ in the point cloud, which is necessary for the processing. We were also able to develop a method that can be applied to agriculture, such as harvest counting, using this method.

Free Research Field

リモートセンシング

Academic Significance and Societal Importance of the Research Achievements

植生リモートセンシングにおいて、植生内部や影などセンサーの死角部分の情報が欠落する問題や植生の病虫害検知に関しては、技術的改善が急務であったが、GANによる植生領域欠落部補間の検討及びVAE異常部識別器の開発がなされることで、その改善が大幅に進むこととなる。現在、この技術の開発は進行中であるが、その処理に必要な点群内での個々の樹木の識別や各器官の分離に関しては有効な技術を開発することができた。また、各器官の分離技術を転用し、ドローンから果実の検出を行う技術やトラクターで動的に収穫物をカウントする技術の開発など、農業分野への適用が可能な技術を新たに開発することができた。