2023 Fiscal Year Research-status Report
Baldwinian Evolution of Task-Specialised High-Efficiency Learning in Neural Networks
| Project/Area Number |
23K11262
|
|---|
| Research Institution | Shinshu University |
|---|
Principal Investigator |
Arnold Solvi 信州大学, 工学部, 准教授(特定雇用) (80764935)
|
|---|
| Co-Investigator(Kenkyū-buntansha) |
有田 隆也 名古屋大学, 情報学研究科, 教授 (40202759)
鈴木 麗璽 名古屋大学, 情報学研究科, 准教授 (20362296)
|
|---|
| Project Period (FY) |
2023-04-01 – 2026-03-31
|
| Keywords | neural networks / artificial intelligence / learning algorithms / evolution of learning / meta-learning / Baldwin effect / artificial life |
|---|
| Outline of Annual Research Achievements |
The main goal for this year was a proof-of-concept implementation of the hypothesised evolutionary scenario. We developed a model for evolution of neural networks with mechanisms for both reward-driven learning (an existing Reinforcement Learning algorithm) and direct synaptic weight modification via neuromodulation (a novel implementation of the neuromodulation concept that considers columnar neural structures). We designed 2D and 3D task domains consisting of navigation tasks that require individual learning to solve. We let neural network populations evolve on these domains, and analysed how learning abilities evolved. The resulting evolutionary dynamics are consistent with our theory: first reward-driven learning appears, then non-reward information is gradually integrated into the learning process via the neuromodulation mechanism, thereby improving the efficiency of the learning process. On the present task domains, evolution eventually eliminates the need for reward signals altogether, enabling reward-agnostic learning of the tasks. We performed a quantitative comparison with a representative non-evolutionary Reinforcement Learning algorithm, and found that the learning abilities evolved in our model learn over 300 times faster on the task domain. These results support our theory and indicate its potential for improving learning ability in neural networks. We prepared a conference paper discussing our theory and results.
|
| Current Status of Research Progress |
Current Status of Research Progress
2: Research has progressed on the whole more than it was originally planned.
Reason
The project is mostly proceeding as planned. We switched from a locomotion task to a navigation task for the initial proof of concept because the latter is computationally lightweight, allowing us to experiment more effectively with various neural network implementations during the early stages of the project. Results on the navigation task exceeded expectations. The originally planned locomotion task has also been developed, and we plan to run experiments on this task in FY2024.
|
| Strategy for Future Research Activity |
In FY2024 so far, we submitted a conference paper on the theory and our first results, and made this paper publicly available as a pre-print. The main research direction for FY2024 will be to diversify the tasks we apply the system to. From a theoretical point of view, this should help clarify the role of the hypothesised evolutionary dynamic in biological evolution. From a practical point of view, this will clarify what sort of tasks the system solves well and what sort of tasks will require further development. We also plan to release source code to allow others to experiment with the approach.
|
| Causes of Carryover |
In the initial plan, research meetings would take place at both the PI’s institute and the Co-PIs’ institute. However, it turned out more convenient for the PI to visit the Co-PIs’ institute, resulting in the Co-PIs’ travel funds (80000 yen each for two Co-PIs) remaining unused. We plan to the present the work at a conference in FY2024, so we expect to use these funds for travel or conference registration expenses this year. The 33300 yen left over in the PI’s budget will be put towards literature or hardware.
|
| Remarks |
A paper reporting the results obtained so far, accepted for presentation at The 2024 Conference on Artificial Life. We made this paper publicly available as a pre-print at the provided URL.
|
