| Project/Area Number |
18K11449
|
|---|
| Research Category |
Grant-in-Aid for Scientific Research (C)
|
| Allocation Type | Multi-year Fund |
|---|
| Section | 一般 |
|---|
| Review Section |
Basic Section 61030:Intelligent informatics-related
|
|---|
| Research Institution | Hosei University (2021-2023)
Tokyo University of Technology (2018-2020) |
|---|
Principal Investigator |
|
|---|
| Co-Investigator(Kenkyū-buntansha) |
持橋 大地 統計数理研究所, 数理・推論研究系, 准教授 (80418508)
吉仲 亮 東北大学, 情報科学研究科, 准教授 (80466424)
|
|---|
| Project Period (FY) |
2018-04-01 – 2024-03-31
|
| Project Status |
Completed (Fiscal Year 2023)
|
| Budget Amount *help |
¥4,550,000 (Direct Cost: ¥3,500,000、Indirect Cost: ¥1,050,000)
Fiscal Year 2020: ¥650,000 (Direct Cost: ¥500,000、Indirect Cost: ¥150,000)
Fiscal Year 2019: ¥1,950,000 (Direct Cost: ¥1,500,000、Indirect Cost: ¥450,000)
Fiscal Year 2018: ¥1,950,000 (Direct Cost: ¥1,500,000、Indirect Cost: ¥450,000)
|
|---|
| Keywords | 深層学習(ディープラーニング) / 形式言語理論 / 形式言語 / Transformer / RNN / 表現学習 / 自然言語処理 / 情報ボトルネック法 / 分布エンコーディング / 深層学習 / 内部表現 / LSTM / ディープラーニング / 統計的学習理論 / 時系列予測 |
|---|
| Outline of Final Research Achievements |
Tracing and extracting the internal representations of deep learning models is one of the approaches towards enhancing explainablity of AI. In this research, we analyzed deep learning models such as RNNs and Transformers, which were trained from various datasets, from the perspective of syntactic structures. Particularly, we used formal language models to explore what syntactic features can be learned and how these are represented within internal vectors. Additionally, to clarify underlying issues related to internal representations in advance, we employed adversarial datasets. Adversarial datasets contain syntactic errors but only minor differences. We verified whether deep learning models truly acquire the ability to discern syntactic correctness.
|
| Academic Significance and Societal Importance of the Research Achievements |
RNNやTransformer などの言語モデルがどの程度構文的な知識を獲得できるのか，また，獲得できるとすれば，それらがどのように埋め込まれるのか，言語モデルの理論に照らし合わせて追求することで，未だにブラックボックスである深層学習モデルの説明可能性に対して一定の方向性を示すことができたと考える．また，今後とも，形式言語クラスやそのアルゴリズム的学習の理論的な研究と，実際の産業で使われるような深層学習の分野との架け橋としての役割を果たしていきたい．
|
