| Project/Area Number |
23K24856
|
|---|
| Project/Area Number (Other) |
22H03600 (2022-2023)
|
|---|
| Research Category |
Grant-in-Aid for Scientific Research (B)
|
| Allocation Type | Multi-year Fund (2024)
Single-year Grants (2022-2023) |
|---|
| Section | 一般 |
|---|
| Review Section |
Basic Section 60090:High performance computing-related
|
|---|
| Research Institution | Institute of Physical and Chemical Research |
|---|
Principal Investigator |
WAHIB MOHAMED 国立研究開発法人理化学研究所, 計算科学研究センター, チームリーダー (00650037)
|
|---|
| Co-Investigator(Kenkyū-buntansha) |
ドローズド アレクサンドロ 国立研究開発法人理化学研究所, 計算科学研究センター, 研究員 (90740126)
|
|---|
| Project Period (FY) |
2022-04-01 – 2026-03-31
|
| Project Status |
Granted (Fiscal Year 2024)
|
| Budget Amount *help |
¥17,290,000 (Direct Cost: ¥13,300,000、Indirect Cost: ¥3,990,000)
Fiscal Year 2025: ¥4,030,000 (Direct Cost: ¥3,100,000、Indirect Cost: ¥930,000)
Fiscal Year 2024: ¥4,290,000 (Direct Cost: ¥3,300,000、Indirect Cost: ¥990,000)
Fiscal Year 2023: ¥4,290,000 (Direct Cost: ¥3,300,000、Indirect Cost: ¥990,000)
Fiscal Year 2022: ¥4,680,000 (Direct Cost: ¥3,600,000、Indirect Cost: ¥1,080,000)
|
|---|
| Keywords | Code Generation / Neural Networks / Intelligent Programming / HPC / Compilers / Machine Learning / GPUs / Numerical Methods |
|---|
| Outline of Research at the Start |
Develop a machine learning framework for automated code generation and optimization, replacing manual porting. Utilize diverse datasets to train models for various hardware architectures. Validate, refine, and deploy the framework within a year for wide accessibility and usability.
|
| Outline of Annual Research Achievements |
In this fiscal year we developed an approach that automatically generated neural networks. Neural architecture search is an effective approach for automating the design of deep neural networks. Evolutionary computation (EC) is commonly used in Neural architecture search due to its global optimization capability. However, the evaluation phase of architecture candidates in EC-based NAS is compute-intensive, limiting its application for many real-world problems. To overcome this challenge, we proposed a novel progressive evaluation strategy for the evaluation phase in convolutional neural network architecture search, in which the number of training epochs of network individuals is progressively increased. Our proposed algorithm reduces the computational cost of the evaluation phase and promotes population diversity and fairness by preserving promising networks based on their distribution. We evaluated the proposed progressive evaluation and sub-population preservation of neural architecture search (PEPNAS) algorithm on the CIFAR10, CIFAR100, and ImageNet benchmark datasets, and compare it with 36 state-of-the-art algorithms, including manually designed networks, reinforcement learning (RL) algorithms, gradient-based algorithms, and other EC-based ones. The experimental results demonstrate that PEP-NAS effectively identifies networks with competitive accuracy while also markedly improving the efficiency of the search process.
|
| Current Status of Research Progress |
Current Status of Research Progress
2: Research has progressed on the whole more than it was originally planned.
Reason
The project is progressing as expected. We were capable of publishing several papers.
|
| Strategy for Future Research Activity |
Our plan for the next fiscal year is to incorporate our approach to auto-generate neural networks by proposing a progressive neural predictor that uses score-based sampling to improve the performance of the surrogate model with limited training data. Different from existing algorithms that rely on initial sample selection uses an online method to progressively select new samples of the surrogate model based on potential information from the previous search process. During the iterative process, the sampled scores are dynamically adjusted based on the prediction rankings in each round to keep track of good architectures, which gradually optimises the surrogate model. In this way, the processes of training the predictor and searching for architectures are jointly combined to improve the efficiency of sample utilization. In addition, the surrogate model with different degrees of training is assigned prediction confidence equal to the accuracy of the current stage.
|
