Development of depth estimation technique with tilted optics using chromatic aberration and color filter aperture

• Project/Area Number: 18K11384
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Review Section: Basic Section 61010:Perceptual information processing-related
• Research Institution: Fukuyama University
• Principal Investigator: Ikeoka Hiroshi 福山大学, 工学部, 准教授 (20579966)
• Project Period (FY): 2018-04-01 – 2022-03-31
• Project Status: Completed (Fiscal Year 2021)
• Budget Amount: ¥3,640,000 (Direct Cost: ¥2,800,000、Indirect Cost: ¥840,000); Fiscal Year 2020: ¥780,000 (Direct Cost: ¥600,000、Indirect Cost: ¥180,000); Fiscal Year 2019: ¥1,560,000 (Direct Cost: ¥1,200,000、Indirect Cost: ¥360,000); Fiscal Year 2018: ¥1,300,000 (Direct Cost: ¥1,000,000、Indirect Cost: ¥300,000)
• Keywords: 距離推定 / アオリ光学系 / カラーフィルタ / 絞り / カラーフィルタ絞り / 車載 / アオリ光学 / 鮮鋭度 / ３次元位置推定 / 色収差 / ニューラルネットワーク / 深層学習

Outline of Final Research Achievements

In a depth estimation method using an tilted optics, it was found that the range of depth estimation in which each RGB channel excels differs greatly. Therefore, we proposed and investigated a method that can estimate the depth from the sharpness ratio between channels at each pixel by using demosaicing images of each channel with different focal plane positions and depth of field. In addition, we investigated a method that enables depth estimation with a single image sensor by incorporating an aperture composed of color filters into the optical system to set a large difference in the depth-of-field distribution between each channel. For example, we proposed a method to estimate the depth from the sharpness ratio between the RB average image and the G image, where the normal aperture is F5.6 and the G color filter aperture is F11.

Academic Significance and Societal Importance of the Research Achievements

焦点距離の異なる２枚以上画像から，鮮鋭度の差異より距離推定を行うDepth-from-Defocus(DFD)方式があるが，これにイメージセンサとレンズ主面を非平行に配置したアオリ光学系を導入することで， (1)低コスト，(2)実時間対応，(3)外乱に頑健，というDFD方式の特徴に加え(4)広範囲推定の４要件を備えたリアルタイム用途に適した距離推定システムを実現できる．本研究では，さらに光学系を工夫し，カラー（RGB）成分で異なる３枚アオリ画像を１枚のイメージセンサから取得し，距離を推定する方式を確立することで，装置のコンパクト化やさらなる低コスト化の実現を図る道筋をつけることに成功した．

Research Products

• [Presentation] Study on Adaptation of Auto Feeding System for Sillago Japonica to Actual Aquaculture Environment (2022)
  • Author(s): Yutaka Saragai, Takuya Sato, Haruki Kuroki,Hiroshi Ikeoka, Koichi Isawa
  • Organizer: International Conference on Image Processing and Robotics(ICIPRob)
  • Int'l Joint Research
• [Presentation] DEVELOPMENT OF REAL-TIME FISH POSITION RECOGNITION SYSTEM FOR AUTOMATIC FEEDING AQUACULTURE (2021)
  • Author(s): Takuya Sato, Haruki Kuroki, Hiroshi Ikeoka, Koichi Isawa
  • Organizer: Image Electronics and Visual Computing Workshop(IEVC)
  • Int'l Joint Research
• [Presentation] Depth Estimation with Tilted Optics by Multi-Aperture Using Color Filter (2020)
  • Author(s): Hiroshi Ikeoka, Takayuki Hamamoto
  • Organizer: International Workshop on Advanced Image Technology (IWAIT)
  • Int'l Joint Research
• [Presentation] Development of simulator for efficient aquaculture of Sillago japonica using reinforcement learning (2020)
  • Author(s): Haruki Kuroki, Hiroshi Ikeoka, Koichi Isawa
  • Organizer: International Conference on Image Processing and Robotics (ICIPRob)
  • Int'l Joint Research
• [Presentation] Accuracy improvement of depth estimation with tilted optics by optimizing neural network (2019)
  • Author(s): Hiroshi Ikeoka, Takayuki Hamamoto
  • Organizer: International Workshop on Advanced Image Technology (IWAIT)
  • Int'l Joint Research
• [Presentation] アオリ光学系における色収差を用いた距離推定 (2018)
  • Author(s): 西峯沙耶, 池岡宏, 浜本隆之
  • Organizer: 2018映像情報メディア学会年次大会