| 研究実績の概要 |
In this fiscal year we developed an approach that automatically generated neural networks. Neural architecture search is an effective approach for automating the design of deep neural networks. Evolutionary computation (EC) is commonly used in Neural architecture search due to its global optimization capability. However, the evaluation phase of architecture candidates in EC-based NAS is compute-intensive, limiting its application for many real-world problems. To overcome this challenge, we proposed a novel progressive evaluation strategy for the evaluation phase in convolutional neural network architecture search, in which the number of training epochs of network individuals is progressively increased. Our proposed algorithm reduces the computational cost of the evaluation phase and promotes population diversity and fairness by preserving promising networks based on their distribution. We evaluated the proposed progressive evaluation and sub-population preservation of neural architecture search (PEPNAS) algorithm on the CIFAR10, CIFAR100, and ImageNet benchmark datasets, and compare it with 36 state-of-the-art algorithms, including manually designed networks, reinforcement learning (RL) algorithms, gradient-based algorithms, and other EC-based ones. The experimental results demonstrate that PEP-NAS effectively identifies networks with competitive accuracy while also markedly improving the efficiency of the search process.
|
| 今後の研究の推進方策 |
Our plan for the next fiscal year is to incorporate our approach to auto-generate neural networks by proposing a progressive neural predictor that uses score-based sampling to improve the performance of the surrogate model with limited training data. Different from existing algorithms that rely on initial sample selection uses an online method to progressively select new samples of the surrogate model based on potential information from the previous search process. During the iterative process, the sampled scores are dynamically adjusted based on the prediction rankings in each round to keep track of good architectures, which gradually optimises the surrogate model. In this way, the processes of training the predictor and searching for architectures are jointly combined to improve the efficiency of sample utilization. In addition, the surrogate model with different degrees of training is assigned prediction confidence equal to the accuracy of the current stage.
|
