Development of 1-chip CNN real-time processor for semantic segmentation, distance estimation, and motion estimation

2020 Fiscal Year Final Research Report

• Project/Area Number: 18K11350
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Review Section: Basic Section 61010:Perceptual information processing-related
• Research Institution: Kanazawa University
• Principal Investigator: Miyama Masayuki 金沢大学, 電子情報通信学系, 准教授 (30324106)
• Project Period (FY): 2018-04-01 – 2021-03-31
• Keywords: CNN / 意味分割 / フロー推定 / SLAM / FPGA

Outline of Final Research Achievements

We devised a convolutional neural network (CNN) that simultaneously estimates the contour of an object and the distance (parallax) in order to recognize the surrounding environment from an image, and implemented it on an FPGA (Field Programmable Gate Array). Contour detection is transformed to a regression problem that estimates the distance from the object boundary instead of the conventional binary classification of contour and non-contour, enabling multitask learning with parallax estimation. Then, we devised a CNN with full weight sharing that performs two estimations at the same time. This was quantized to 3 bits, but the decrease in accuracy was slight. As a result of FPGA implementation, it operates at 250 MHz and has a throughput of 134 fps for an image of 480 x 320 pixels.

Free Research Field

VLSI画像処理

Academic Significance and Societal Importance of the Research Achievements

ステレオ画像を用いた一般物体輪郭検出と視差推定の同時学習とマルチタスクCNNはこれまでに報告されていない。これを3ビットまで量子化したCNNの回路設計やFPGA実装も行われていない。このCNN回路は畳み込みの重みを変更すれば画像を画素単位でカテゴリ分類する意味的分割も実行できる。開発したFPGAは解像度480×320画素の画像に対してスループット134 fpsで動作し、自律ロボットのSLAM（自己位置推定と地図作成の同時実行）や周囲環境認識に応用できる。