On a noise convolutional neural network

• Project/Area Number: 19H04078
• Research Category: Grant-in-Aid for Scientific Research (B)
• Allocation Type: Single-year Grants
• Section: 一般
• Review Section: Basic Section 60040:Computer system-related
• Research Institution: Tohoku University (2023); Tokyo Institute of Technology (2019-2022)
• Principal Investigator: 中原 啓貴 東北大学, 未踏スケールデータアナリティクスセンター, 教授 (20624414)
• Co-Investigator(Kenkyū-buntansha): 佐野 健太郎 国立研究開発法人理化学研究所, 計算科学研究センター, チームリーダー (00323048) ; 佐藤 真平 信州大学, 学術研究院工学系, 准教授 (80782763)
• Project Period (FY): 2019-04-01 – 2024-03-31
• Project Status: Completed (Fiscal Year 2023)
• Budget Amount: ¥17,160,000 (Direct Cost: ¥13,200,000、Indirect Cost: ¥3,960,000); Fiscal Year 2023: ¥1,690,000 (Direct Cost: ¥1,300,000、Indirect Cost: ¥390,000); Fiscal Year 2022: ¥1,690,000 (Direct Cost: ¥1,300,000、Indirect Cost: ¥390,000); Fiscal Year 2021: ¥1,690,000 (Direct Cost: ¥1,300,000、Indirect Cost: ¥390,000); Fiscal Year 2020: ¥5,330,000 (Direct Cost: ¥4,100,000、Indirect Cost: ¥1,230,000); Fiscal Year 2019: ¥6,760,000 (Direct Cost: ¥5,200,000、Indirect Cost: ¥1,560,000)
• Keywords: AI / Machine Learning / FPGA / ニューラルネットワーク / 雑音畳み込み / CNN / ノイズCNN / LSI / 深層学習 / 組込みシステム / 画像圧縮 / Deep Learning / 計算機システム / 高性能計算 / Noise Convolution

Outline of Research at the Start

学習済みCNNのパラメータの統計的解析に元づく雑音畳込みニューラルネットワーク(雑音CNN)を提案する。雑音CNNは畳込み演算の大部分を雑音の畳込みで代用してパラメータを最大で約90%削減できる。本研究では、雑音CNNの雑音畳み込み回路を開発し、組込み機器向けに専用チップを設計する。また、雑音CNNの学習時間を短縮するため、FPGAベースの高速学習クラスタを開発する。FPGAによるプロトタイプ実装でさらなる性能向上を明らかにする。

Outline of Annual Research Achievements

雑音CNN(Convolutional Neural Network)とはCNNの大量のパラメータを疑似乱数をベースとした雑音モデルで置き換えるCNNの一種であり、回路面積の削減により高速なCNN専用回路を実現できる。
本研究開発において、雑音CNNは既存CNNと等価であることを導出した。雑音CNN専用回路の設計を行い、FPGAプロトタイプ実装を行った。雑音生成回路と1×1サイズ畳込みを組み合わせた構成に適したノイズ生成回路を調査し、回路実装した。GPUと比較して提案手法の優位性を示した。雑音CNN回路に既存のパラメータ削減方法である低ビット化と枝刈り手法を適用して更に性能向上させた。応用事例としてクラス分類、物体検出、姿勢推定、深度推定に適用可能であることを確認した。また、再帰型ニューラルネットワークへの適用を検討しFPGA実装を行なった。アンサンブル手法を適用し、モデルを軽量化しつつ高速な専用回路実装方法を研究開発した。学習時に認識精度劣化を抑制するパラメータ調整方法を実験的に導き出した。
雑音畳み込みは既存畳み込みと等価なので、既存手法と組合せが可能であることを示した。また、雑音CNNの学習高速化のためのFPGAクラスタの環境構築を行い、雑音CNNの学習方式を検討した。
研究成果を国際会議、雑誌論文にまとめて出版した。また、一部の研究成果は英国インペリアルカレッジロンドン校との国際共同研究成果として発表した。

Research Progress Status

令和5年度が最終年度であるため、記入しない。

Strategy for Future Research Activity

令和5年度が最終年度であるため、記入しない。

Research Products

• [Int'l Joint Research] Imperial College London(英国)
• [Int'l Joint Research] Imperial College London(英国)
• [Int'l Joint Research] Imperial College London(英国)
• [Int'l Joint Research] Imperial College London(英国)
• [Journal Article] Remarn: A Reconfigurable Multi-threaded Multi-core Accelerator for Recurrent Neural Networks (2023)
  • Author(s): Zhiqiang Que, Hiroki Nakahara, Hongxiang Fan, He Li, Jiuxi Meng, Kuen Hung Tsoi, Xinyu Niu, Eriko Nurvitadhi, Wayne Luk:
  • Journal Title: ACM Trans. Reconfigurable Technol. Volume: 16 Pages: 1-26
  • Peer Reviewed / Int'l Joint Research
• [Journal Article] Multilayer Perceptron Training Accelerator Using Systolic Array (2022)
  • Author(s): Takeshi Senoo, Akira Jinguji, Ryosuke Kuramochi, Hiroki Nakahara
  • Journal Title: EICE Trans. Inf. Syst. Volume: 105-D(12) Pages: 2048-2056
  • Peer Reviewed
• [Journal Article] Recurrent Neural Networks With Column-Wise Matrix-Vector Multiplication on FPGAs (2022)
  • Author(s): Zhiqiang Que, Hiroki Nakahara, Eriko Nurvitadhi, Andrew Boutros, Hongxiang Fan, Chenglong Zeng, Jiuxi Meng, Kuen Hung Tsoi, Xinyu Niu, Wayne Luk
  • Journal Title: IEEE Trans. Very Large Scale Integr. Syst. Volume: 30(2) Pages: 227-237
  • Peer Reviewed / Int'l Joint Research
• [Journal Article] Optimizations of Ternary Generative Adversarial Networks (2022)
  • Author(s): Kennichi Nakamura, Hiroki Nakahara
  • Journal Title: IEEE Int. Symp. on Multi-valued Logic (ISMVL) Pages: 158-163
  • Peer Reviewed
• [Journal Article] Fast Interface with Ensemble Ternary Neural Network (2022)
  • Author(s): Ryota Kayanoma, Hiroki Nakahara
  • Journal Title: IEEE Int. Symp. on Multi-valued Logic (ISMVL) Pages: 182-187
  • Peer Reviewed
• [Journal Article] Energy-Efficient ECG Signals Outlier Detection Hardware Using a Sparse Robust Deep Autoencoder (2021)
  • Author(s): Naoto Soga, Shimpei Sato, Hiroki Nakahara
  • Journal Title: IEICE Trans. Inf. Syst Volume: 104-D(8) Pages: 1121-1129
  • NAID: 130008070358
  • Peer Reviewed
• [Journal Article] Weight Sparseness for a Feature-Map-Split-CNN Toward Low-Cost Embedded FPGAs (2021)
  • Author(s): Akira Jinguji, Shimpei Sato, Hiroki Nakahara
  • Journal Title: IEICE Trans. Inf. Syst. Volume: 104-D(12) Pages: 2040-2047
  • NAID: 130008123390
  • Peer Reviewed
• [Journal Article] A Low-Latency Inference of Randomly Wired Convolutional Neural Networks on an FPGA (2021)
  • Author(s): Ryosuke Kuramochi, Hiroki Nakahara
  • Journal Title: IEICE Trans. Inf. Syst. Volume: 104-D(12) Pages: 2068-2077
  • NAID: 130008123389
  • Peer Reviewed
• [Journal Article] FPGA-Based Inter-layer Pipelined Accelerators for Filter-Wise Weight-Balanced Sparse Fully Convolutional Networks with Overlapped Tiling (2021)
  • Author(s): Masayuki Shimoda, Youki Sada, Hiroki Nakahara
  • Journal Title: J. Signal Process. Syst. Volume: 93(5) Pages: 499-512
  • Peer Reviewed
• [Journal Article] SENTEI: Filter-Wise Pruning with Distillation towards Efficient Sparse Convolutional Neural Network Accelerators (2020)
  • Author(s): Masayuki Shimoda, Youki Sada, Ryosuke Kuramochi, Shimpei Sato, Hiroki Nakahara
  • Journal Title: IEICE Transactions on Information and Systems Volume: E103.D Issue: 12 Pages: 2463-2470
  • DOI: 10.1587/transinf.2020PAP0013
  • NAID: 130007948567
  • ISSN: 0916-8532, 1745-1361
  • Year and Date: 2020-12-01
  • Peer Reviewed
• [Journal Article] GUINNESS: A GUI Based Binarized Deep Neural Network Framework for Software Programmers (2019)
  • Author(s): Hiroki Nakahara, Haruyoshi Yonekawa, Tomoya Fujii, Masayuki Shimoda, Shimpei Sato
  • Journal Title: IEICE Trans. Inf. Syst. Volume: 102-D(5) Pages: 1003-1011
  • NAID: 130007641155
  • Peer Reviewed
• [Journal Article] Power Efficient Object Detector with an Event-Driven Camera for Moving Object Surveillance on an FPGA (2019)
  • Author(s): Masayuki Shimoda, Shimpei Sato, Hiroki Nakahara
  • Journal Title: IEICE Trans. Inf. Syst. Volume: 102-D(5) Pages: 1020-1028
  • NAID: 130007641234
  • Peer Reviewed
• [Presentation] A Light-Weight Vision Transformer Toward Near Memory Computation on an FPGA (2023)
  • Author(s): Takeshi Senoo, Ryota Kayanoma, Akira Jinguji, Hiroki Nakahara
  • Organizer: ARC
  • Int'l Joint Research
• [Presentation] A Consideration on Ternary Adversarial Generative Networks (2023)
  • Author(s): Kennichi Nakamura, Hiroki Nakahara
  • Organizer: ISMVL
  • Int'l Joint Research
• [Presentation] A Multilayer Perceptron Training Accelerator using Systolic Array (2021)
  • Author(s): Takeshi Senoo, Akira Jinguji, Ryosuke Kuramochi, Hiroki Nakahara
  • Organizer: IEEE APCCAS
  • Int'l Joint Research
• [Presentation] Edge Inference Engine for Deep & Random Sparse Neural Networks with 4-bit Cartesian-Product MAC Array and Pipelined Activation Aligner (2021)
  • Author(s): Kota Ando, Jaehoon Yu, Kazutoshi Hirose, Hiroki Nakahara, Kazushi Kawamura, Thiem Van Chu, Masato Motomura
  • Organizer: IEEE HCS
  • Int'l Joint Research
• [Presentation] A High-Throughput Detection Circuit based on 2q+1-Valued Deep Neural Networks (2021)
  • Author(s): Naoto Soga, Ryosuke Kuramochi, Hiroki Nakahara
  • Organizer: IEEE ISMVL
  • Int'l Joint Research
• [Presentation] High-Throughput Convolutional Neural Network on an FPGA by Customized JPEG Compression (2020)
  • Author(s): Hiroki Nakahara, Zhiqiang Que, Wayne Luk
  • Organizer: FCCM2020
  • Int'l Joint Research
• [Presentation] Optimizing Reconfigurable Recurrent Neural Networks (2020)
  • Author(s): Zhiqiang Que, Hiroki Nakahara, Eriko Nurvitadhi, Hongxiang Fan, Chenglong Zeng, Jiuxi Meng, Xinyu Niu, Wayne Luk
  • Organizer: FCCM2020
  • Int'l Joint Research
• [Presentation] An FPGA-Based Low-Latency Accelerator for Randomly Wired Neural Networks (2020)
  • Author(s): Ryosuke Kuramochi, Hiroki Nakahara
  • Organizer: FPL2020
  • Int'l Joint Research
• [Presentation] 2n+1-valued SSS-Net: Uniform Shift, Channel Sparseness, and Channel Shuffle (2020)
  • Author(s): Hiroki Nakahara
  • Organizer: ISMVL2020
  • Int'l Joint Research
• [Presentation] Fast Monocular Depth Estimation on an FPGA (2020)
  • Author(s): Youki Sada, Naoto Soga, Masayuki Shimoda, Akira Jinguji, Shimpei Sato, Hiroki Nakahara
  • Organizer: IPDPS Workshops 2020
  • Int'l Joint Research
• [Presentation] Filter-Wise Pruning Approach to FPGA Implementation of Fully Convolutional Network for Semantic Segmentation (2019)
  • Author(s): Masayuki Shimoda, Youki Sada, Hiroki Nakahara
  • Organizer: ARC2019
  • Int'l Joint Research
• [Presentation] FPGA-Based Training Accelerator Utilizing Sparseness of Convolutional Neural Network. (2019)
  • Author(s): Hiroki Nakahara, Youki Sada, Masayuki Shimoda, Kouki Sayama, Akira Jinguji, Shimpei Sato
  • Organizer: FPL2019
  • Int'l Joint Research
• [Presentation] A Dataflow Pipelining Architecture for Tile Segmentation with a Sparse MobileNet on an FPGA (2019)
  • Author(s): Youki Sada, Masayuki Shimoda, Akira Jinguji, Hiroki Nakahara
  • Organizer: FPT 2019
  • Int'l Joint Research
• [Presentation] Noise Convolutional Neural Networks and FPGA Implementation (2019)
  • Author(s): Atsuki Munakata, Hiroki Nakahara, Shimpei Sato
  • Organizer: ISMVL 2019
  • Int'l Joint Research
• [Presentation] Many Universal Convolution Cores for Ensemble Sparse Convolutional Neural Networks (2019)
  • Author(s): Ryosuke Kuramochi, Youki Sada, Masayuki Shimoda, Shimpei Sato
  • Organizer: MCSoC 2019
  • Int'l Joint Research
• [Presentation] Deep Learning Accelerator for an Intelligent Camera (2019)
  • Author(s): Hiroki Nakahara
  • Organizer: HEART 2019
  • Int'l Joint Research / Invited