Intensifying deep learning theory and its application to structure analysis of deep neural network

2021 Fiscal Year Final Research Report

• Project/Area Number: 18H03201
• Research Category: Grant-in-Aid for Scientific Research (B)
• Allocation Type: Single-year Grants
• Section: 一般
• Review Section: Basic Section 60010:Theory of informatics-related
• Research Institution: The University of Tokyo
• Principal Investigator: Suzuki Taiji 東京大学, 大学院情報理工学系研究科, 准教授 (60551372)
• Project Period (FY): 2018-04-01 – 2022-03-31
• Keywords: 深層学習 / カーネル法 / 汎化誤差解析 / ノンパラメトリック統計 / モデル圧縮

Outline of Final Research Achievements

Deep learning currently plays a central role in machine learning and has shown high performance in many tasks. On the other hand, theoretical understanding of its principles has not progressed. Indeed, at the beginning of this research project, it was almost a black box. In order to change this situation, we have obtained the following research results on the principles of deep learning. (1) Compression based generalization error analysis of deep learning from the kernel method perspective, (2) Proposing a new method to obtain the optimal model structure based on statistical degrees of freedom and its application to model compression, (3) Proposal of new stochastic optimization methods, and (4) Theoretical proof of the superiority of deep learning over the kernel method and other classical methods. Through these studies, we have obtained many insights into the question of why deep learning is better than other methods.

Free Research Field

機械学習

Academic Significance and Societal Importance of the Research Achievements

深層学習は機械学習の社会実装が進む中，社会的重要な技術となっている．一方でその原理が解明されずに応用だけ進むことは，制御可能性や説明可能性という観点からも望ましくない．本研究では，種々の数学的道具を用いて深層学習の原理解明に貢献し，また理論の応用として最適なモデルの探索やモデル圧縮法を提案した．研究成果により研究開始時期と比べて非常に多くの理論的知見が得られた．これは，深層学習をホワイトボックス化するという意味で社会的意義が大きい成果である，