| Project/Area Number |
22K17900
|
|---|
| Research Category |
Grant-in-Aid for Early-Career Scientists
|
| Allocation Type | Multi-year Fund |
|---|
| Review Section |
Basic Section 60090:High performance computing-related
|
|---|
| Research Institution | Institute of Physical and Chemical Research |
|---|
Principal Investigator |
Vatai Emil 国立研究開発法人理化学研究所, 計算科学研究センター, 研究員 (70889633)
|
|---|
| Project Period (FY) |
2022-04-01 – 2026-03-31
|
| Project Status |
Granted (Fiscal Year 2023)
|
| Budget Amount *help |
¥4,550,000 (Direct Cost: ¥3,500,000、Indirect Cost: ¥1,050,000)
Fiscal Year 2025: ¥1,040,000 (Direct Cost: ¥800,000、Indirect Cost: ¥240,000)
Fiscal Year 2024: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000)
Fiscal Year 2023: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000)
Fiscal Year 2022: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000)
|
|---|
| Keywords | sparsity / large language models / data locality / spin chemistry / radical pairs / numerical methods / machine learning / NLP / linear solvers / sparse matrix / iterative methods / graph algorithms |
|---|
| Outline of Research at the Start |
Extremely large problems in science can often be described and solved by mathematics and computations with special data-structures called sparse matrices. Because of the disparity between the speed of processors and memory, optimizing data movement is paramount for the irregular data access of computations with sparse matrices. Since graphs have a natural correspondence to sparse matrices, our research aims to address this issue by exploring the possibilities of graph and machine learning algorithms to generate optimally performing code for such large computations.
|
| Outline of Annual Research Achievements |
We made progress in sparsifying transformers (the building block of large language models). With the help of intern and now student trainee Tamas Ficsor, we've made progress in implementing locally sensitive hashes (LSHs) as a sparse alternative to multilayer perceptron.
We have further developed RadicalPy, our Python package for spin-chemistry simulations of radical pairs. We have implemented half a dozen functions corresponding to experiments and about 15 examples which users can modify and use as a template to fit their own experimental setup. We aimed to make RadicalPy as "professional" and as "user friendly" as possible, so we aim to maximise automation, i.e., CI/CD: everything is documented, with a documentation website being automatically generated from the code; each release is automatically published as a python package to PYPI.
The most notable addition to the software is the novel λt-MARY simulation approach. This method incorporates chemical rate equations into quantum simulation models. It also allows reference spectra to be included in the simulation to produce time- and wavelength-resolved magnetic field effects. Our approach achieves better accuracy than quantum simulations, at a constant amount of memory, which is minuscule compared to the exponential (in the number of simulated nuclei) memory requirement of the quantum method.
|
| Current Status of Research Progress |
Current Status of Research Progress
2: Research has progressed on the whole more than it was originally planned.
Reason
The possibilities and performance of LSHs as replacements for MLP is somewhat uncertain, and implementing it on fugaku is slow and challenging. However, we've made good progress with RadicalPy.
RadicalPy is publicly available on GitHub, the documentation on ReadTheDocs.org and the Python package can be installed from pypi.org. The current released version of the software is v0.7.2 indicating that the API is close to stabilising. While the infrastructure is set up for CI/CD, automatic testing and document generation, some pieces of the code are not yet fully tested and documented. The parts of the code lacking testing (and sometimes documentation) are the newer additions, where the code itself needs some refactoring, as well es update to the tests to reflect some previous refactoring. RadicalPy is used by practitioners at the Oxford by the leading research lab on magnetoreception, and we received a lot of positive feedback.
RadicalPy and the new results about the superb accuracy of the λt-MARY is summarised in a paper currently submitted to Nature Computation Science. The journal has an "Article" track (for novel results in computational science), and "Resources" for tooling such as RadicalPy. At an earlier stage of the paper (which did not include the λt-MARY results) the paper was a better fit for a "Resource", however with the with the positive (and somewhat surprising) new results from the λt-MARY approach, we hope it will be accepted as an "Article". I also gave a talk about RadicalPy at the QuTip workshop.
|
| Strategy for Future Research Activity |
The LSHs still need to be implemented and performance (both in terms of accuracy and run-time) needs to be measured.
RadicalPy was originally planned as a (quick and dirty, not performant) "test implementation" for a large-scale, high performance, distributed simulation software for supercomputers, but it grew to be a full blown package (implemented in NumPy and SciPy). However, the goal of the high performance distributed version is still relevant and we intend to implement it using the PETSc/SLEPc scientific library.
Additionally, considering the fairly regular and structured but large Hamiltonians, to combat the memory requirements of quantum methods, we suggest an implementation using PETSc's "shell matrices". Shell matrices don't explicitly store the weights of the matrix, but implement other operations with the matrix, such as matrix-vector product. This way instead of storing the usually repetitive entries of the Hamiltonian matrices, shell matrices can store the minimal information required to reconstruct the matrix, that is to emulate the resulting matirx-vector product. This idea was discussed with the developers of QuTip, a similar software package for quantum simulations, and was met with interest and positive feedback. Hence, building and measuring a prototype implementation of this shell matrix approach is the next major item to be executed, followed by implementing it as a backend, with RadicalPy as a frontend.
|
