| Project/Area Number |
23K19981
|
|---|
| Research Category |
Grant-in-Aid for Research Activity Start-up
|
| Allocation Type | Multi-year Fund |
|---|
| Review Section |
1001:Information science, computer engineering, and related fields
|
|---|
| Research Institution | The Institute of Statistical Mathematics |
|---|
Principal Investigator |
李 静沛 統計数理研究所, 先端データサイエンス研究系, 准教授 (00984767)
|
|---|
| Project Period (FY) |
2023-08-31 – 2025-03-31
|
| Project Status |
Granted (Fiscal Year 2023)
|
| Budget Amount *help |
¥2,860,000 (Direct Cost: ¥2,200,000、Indirect Cost: ¥660,000)
Fiscal Year 2024: ¥1,430,000 (Direct Cost: ¥1,100,000、Indirect Cost: ¥330,000)
Fiscal Year 2023: ¥1,430,000 (Direct Cost: ¥1,100,000、Indirect Cost: ¥330,000)
|
|---|
| Keywords | Deep learning / Adaptive proximal method / nonlinear optimization / proximal methods / regularized optimization |
|---|
| Outline of Research at the Start |
This project will investigate condensing deep learning models via low-rank structures such that each layer of the neural network can be decomposed into two layers with much fewer neurons and thus fewer model parameters. We will leverage techniques from nonlinear optimization to train such models.
|
| Outline of Annual Research Achievements |
Our research has led to the following paper preprints:
- Huang, Zih-Syuan and Lee, Ching-pei. Regularized adaptive momentum dual averaging with an efficient inexact subproblem solver for training structured neural network, 2024. https://arxiv.org/abs/2403.14398.
- Chen, Yen-Ju, Huang, Nai-Chieh, Lee, Ching-pei, and Hsieh, Ping-Chun. Accelerated policy gradient: On the convergence rates of the Nesterov momentum for reinforcement learning. In Proceedings of the 41st International Conference on Machine Learning (ICML), 2024. https://openreview.net/forum?id=aeXRBnLoPP. Accepted.
We have also released a package based on the second paper above at the following URL: https://github.com/ismoptgroup/RAMDA
|
| Current Status of Research Progress |
Current Status of Research Progress
2: Research has progressed on the whole more than it was originally planned.
Reason
We have finished two research papers on related directions, and released a software package based on our research outcomes. In general our progress is smooth and in an expected pace.
|
| Strategy for Future Research Activity |
In the coming year, the principal investigator will make some travels to attend conferences to present our research outcomes to the research community. We will also continue working on the convergence guarantees for the developed methods as well as its combination with low-rank regularization.
|
