2023 Fiscal Year Final Research Report
Statistical Representation of Internal States of Depth Neural Networks and Exploration of New Learning Methods
| Project/Area Number |
18K11449
|
|---|
| Research Category |
Grant-in-Aid for Scientific Research (C)
|
| Allocation Type | Multi-year Fund |
|---|
| Section | 一般 |
|---|
| Review Section |
Basic Section 61030:Intelligent informatics-related
|
|---|
| Research Institution | Hosei University (2021-2023)
Tokyo University of Technology (2018-2020) |
|---|
Principal Investigator |
|
|---|
| Co-Investigator(Kenkyū-buntansha) |
持橋 大地 統計数理研究所, 数理・推論研究系, 准教授 (80418508)
吉仲 亮 東北大学, 情報科学研究科, 准教授 (80466424)
|
|---|
| Project Period (FY) |
2018-04-01 – 2024-03-31
|
| Keywords | 深層学習(ディープラーニング) |
|---|
| Outline of Final Research Achievements |
Tracing and extracting the internal representations of deep learning models is one of the approaches towards enhancing explainablity of AI. In this research, we analyzed deep learning models such as RNNs and Transformers, which were trained from various datasets, from the perspective of syntactic structures. Particularly, we used formal language models to explore what syntactic features can be learned and how these are represented within internal vectors. Additionally, to clarify underlying issues related to internal representations in advance, we employed adversarial datasets. Adversarial datasets contain syntactic errors but only minor differences. We verified whether deep learning models truly acquire the ability to discern syntactic correctness.
|
| Free Research Field |
機械学習
|
| Academic Significance and Societal Importance of the Research Achievements |
RNNやTransformer などの言語モデルがどの程度構文的な知識を獲得できるのか，また，獲得できるとすれば，それらがどのように埋め込まれるのか，言語モデルの理論に照らし合わせて追求することで，未だにブラックボックスである深層学習モデルの説明可能性に対して一定の方向性を示すことができたと考える．また，今後とも，形式言語クラスやそのアルゴリズム的学習の理論的な研究と，実際の産業で使われるような深層学習の分野との架け橋としての役割を果たしていきたい．
|
