| 研究課題/領域番号 |
21K17802
|
|---|
| 研究種目 |
若手研究
|
| 配分区分 | 基金 |
|---|
| 審査区分 |
小区分61030:知能情報学関連
|
|---|
| 研究機関 | 奈良先端科学技術大学院大学 (2023)
お茶の水女子大学 (2021-2022) |
|---|
研究代表者 |
KANASHIRO・PEREIRA LIS・WEIJI (KANASHIROPEREIRA LISWEIJI) 奈良先端科学技術大学院大学, 先端科学技術研究科, 特任助教 (50896579)
|
|---|
| 研究期間 (年度) |
2021-04-01 – 2025-03-31
|
| 研究課題ステータス |
交付 (2023年度)
|
| 配分額 *注記 |
4,290千円 (直接経費: 3,300千円、間接経費: 990千円)
2022年度: 1,170千円 (直接経費: 900千円、間接経費: 270千円)
2021年度: 3,120千円 (直接経費: 2,400千円、間接経費: 720千円)
|
|---|
| キーワード | adversarial training / nlp / NLP / machine learning / robustness / deep learning / language model |
|---|
| 研究開始時の研究の概要 |
In this research, we aim to improve the generalization and robustness of pre-trained language models on downstream NLP tasks by adopting adversarial training. Adversarial training has a great potential to improve model robustness and generalization, as shown by recent works. Moreover, adversarial training works as an online data augmentation method and can help improve model performance on low-resource scenarios. It can also help improve model performance without increasing the model size, which is helpful in scenarios where computational resources are limited.
|
| 研究実績の概要 |
We have so far accomplished most of the proposed research questions from the initial proposal during the execution of the project. We have shown that applying perturbations to other layers of the network improves current adversarial training methods for natural language processing (NLP). Besides applying perturbations at the embedding level, and exploring applying perturbations to other layers of the model or a combination of layers and performing a comparison of these variations. Similarity, we have shown that multi-task learning also improves current adversarial training methods for NLP. We have also applied our models to Japanese NLP tasks and achieved similar improvements, showing that our models are language-agnostic.
|
| 現在までの達成度 (区分) |
現在までの達成度 (区分)
2: おおむね順調に進展している
理由
As for the remaining research question: using prior knowledge that can guide the algorithm to generate better perturbations, we have made advancements by performing several experiments and we have a draft under submission at the moment.
|
| 今後の研究の推進方策 |
As for this last year, we plan to gather all results obtained into a major publication, such as a journal. In addition, with the rapid progress and releases of language models, such as ChatGPT, we plan to also apply our proposed models to such language models, and verify if they can further improve the performance of such models. Current research has shown that even models such as ChatGPT are susceptible to adversarial attacks and can have their performance degraded by them. From these results, we plan to prepare and submit another draft to a major conference.
|
