Transportation analysis of deep neural networks

2021 Fiscal Year Final Research Report

• Project/Area Number: 18K18113
• Research Category: Grant-in-Aid for Early-Career Scientists
• Allocation Type: Multi-year Fund
• Review Section: Basic Section 61030:Intelligent informatics-related
• Research Institution: Institute of Physical and Chemical Research
• Principal Investigator: Sonoda Sho 国立研究開発法人理化学研究所, 革新知能統合研究センター, 研究員 (00801218)
• Project Period (FY): 2018-04-01 – 2022-03-31
• Keywords: ニューラルネット / ホワイトボックス化 / 積分表現理論 / リッジレット変換 / Neural ODE / カーネル求積 / 非コンパクト対称空間 / 群畳み込み

Outline of Final Research Achievements

Joint research with Dr. Isao Ishikawa (Ehime Univ.) and Dr. Masahiro Ikeda (RIKEN) has led to dramatic progress in integral representation theory. In particular, we have found a general method for deriving ridgelet transforms for various hidden layers, such as fully-connected layers on manifolds and group convolution layers on signal spaces, which has dramatically improved the applicability of integral representation theory. Moreover, the integral representation theory and transport theory have triggered many collaborative researches with researchers in related fields such as quantum machine learning, neuroscience, harmonic analysis, probabilistic numerical analysis, control theory, and differential equation theory. On the other hand, research on transport theory is still at the case-by-case stage, and I believe that a more fundamental theory needs to be developed in future.

Free Research Field

機械学習

Academic Significance and Societal Importance of the Research Achievements

一般に学習済ニューラルネット（NN）の情報処理様式を外部から読み解くことは難しい．NNが誤動作しないよう制御するため，ホワイトボックス化が求められる．積分表現と輸送解釈はいずれも，NNを線形空間という性質の良い空間で表現する方法論であり，ホワイトボックス化の有力候補である．積分表現の強みであるリッジレット変換は特定の全結合型NNに限って発見されていたが，本研究により現代的なNNに対して機械的に導出できるようになり，NNのホワイトボックス化に貢献した．