• 研究課題をさがす
  • 研究者をさがす
  • KAKENの使い方
  1. 前のページに戻る

Data locality for sparse matrices via advanced optimisations in large-scale scientific programs

研究課題

研究課題/領域番号 22K17900
研究種目

若手研究

配分区分基金
審査区分 小区分60090:高性能計算関連
研究機関国立研究開発法人理化学研究所

研究代表者

Vatai Emil  国立研究開発法人理化学研究所, 計算科学研究センター, 研究員 (70889633)

研究期間 (年度) 2022-04-01 – 2026-03-31
研究課題ステータス 交付 (2023年度)
配分額 *注記
4,550千円 (直接経費: 3,500千円、間接経費: 1,050千円)
2025年度: 1,040千円 (直接経費: 800千円、間接経費: 240千円)
2024年度: 1,170千円 (直接経費: 900千円、間接経費: 270千円)
2023年度: 1,170千円 (直接経費: 900千円、間接経費: 270千円)
2022年度: 1,170千円 (直接経費: 900千円、間接経費: 270千円)
キーワードsparsity / large language models / data locality / spin chemistry / radical pairs / numerical methods / machine learning / NLP / linear solvers / sparse matrix / iterative methods / graph algorithms
研究開始時の研究の概要

Extremely large problems in science can often be described and solved by mathematics and computations with special data-structures called sparse matrices. Because of the disparity between the speed of processors and memory, optimizing data movement is paramount for the irregular data access of computations with sparse matrices. Since graphs have a natural correspondence to sparse matrices, our research aims to address this issue by exploring the possibilities of graph and machine learning algorithms to generate optimally performing code for such large computations.

研究実績の概要

We made progress in sparsifying transformers (the building block of large language models). With the help of intern and now student trainee Tamas Ficsor, we've made progress in implementing locally sensitive hashes (LSHs) as a sparse alternative to multilayer perceptron.

We have further developed RadicalPy, our Python package for spin-chemistry simulations of radical pairs. We have implemented half a dozen functions corresponding to experiments and about 15 examples which users can modify and use as a template to fit their own experimental setup. We aimed to make RadicalPy as "professional" and as "user friendly" as possible, so we aim to maximise automation, i.e., CI/CD: everything is documented, with a documentation website being automatically generated from the code; each release is automatically published as a python package to PYPI.

The most notable addition to the software is the novel λt-MARY simulation approach. This method incorporates chemical rate equations into quantum simulation models. It also allows reference spectra to be included in the simulation to produce time- and wavelength-resolved magnetic field effects. Our approach achieves better accuracy than quantum simulations, at a constant amount of memory, which is minuscule compared to the exponential (in the number of simulated nuclei) memory requirement of the quantum method.

現在までの達成度 (区分)
現在までの達成度 (区分)

2: おおむね順調に進展している

理由

The possibilities and performance of LSHs as replacements for MLP is somewhat uncertain, and implementing it on fugaku is slow and challenging. However, we've made good progress with RadicalPy.

RadicalPy is publicly available on GitHub, the documentation on ReadTheDocs.org and the Python package can be installed from pypi.org. The current released version of the software is v0.7.2 indicating that the API is close to stabilising. While the infrastructure is set up for CI/CD, automatic testing and document generation, some pieces of the code are not yet fully tested and documented. The parts of the code lacking testing (and sometimes documentation) are the newer additions, where the code itself needs some refactoring, as well es update to the tests to reflect some previous refactoring. RadicalPy is used by practitioners at the Oxford by the leading research lab on magnetoreception, and we received a lot of positive feedback.

RadicalPy and the new results about the superb accuracy of the λt-MARY is summarised in a paper currently submitted to Nature Computation Science. The journal has an "Article" track (for novel results in computational science), and "Resources" for tooling such as RadicalPy. At an earlier stage of the paper (which did not include the λt-MARY results) the paper was a better fit for a "Resource", however with the with the positive (and somewhat surprising) new results from the λt-MARY approach, we hope it will be accepted as an "Article". I also gave a talk about RadicalPy at the QuTip workshop.

今後の研究の推進方策

The LSHs still need to be implemented and performance (both in terms of accuracy and run-time) needs to be measured.

RadicalPy was originally planned as a (quick and dirty, not performant) "test implementation" for a large-scale, high performance, distributed simulation software for supercomputers, but it grew to be a full blown package (implemented in NumPy and SciPy). However, the goal of the high performance distributed version is still relevant and we intend to implement it using the PETSc/SLEPc scientific library.

Additionally, considering the fairly regular and structured but large Hamiltonians, to combat the memory requirements of quantum methods, we suggest an implementation using PETSc's "shell matrices". Shell matrices don't explicitly store the weights of the matrix, but implement other operations with the matrix, such as matrix-vector product. This way instead of storing the usually repetitive entries of the Hamiltonian matrices, shell matrices can store the minimal information required to reconstruct the matrix, that is to emulate the resulting matirx-vector product. This idea was discussed with the developers of QuTip, a similar software package for quantum simulations, and was met with interest and positive feedback. Hence, building and measuring a prototype implementation of this shell matrix approach is the next major item to be executed, followed by implementing it as a backend, with RadicalPy as a frontend.

報告書

(2件)
  • 2023 実施状況報告書
  • 2022 実施状況報告書
  • 研究成果

    (7件)

すべて 2024 2022 その他

すべて 学会発表 (2件) (うち国際学会 1件、 招待講演 1件) 備考 (5件)

  • [学会発表] Development of Radicalpy and the lessons learned2024

    • 著者名/発表者名
      Emil Vatai
    • 学会等名
      Qutip 2024 Developers workshop
    • 関連する報告書
      2023 実施状況報告書
    • 国際学会 / 招待講演
  • [学会発表] RadicalPy: radical pair spin dynamics for the masses2022

    • 著者名/発表者名
      Antill Lewis , Vatai Emil
    • 学会等名
      第61回電子スピンサイエンス学会年会
    • 関連する報告書
      2022 実施状況報告書
  • [備考] RadicalPy GitHub repo

    • URL

      https://github.com/Spin-Chemistry-Labs/radicalpy

    • 関連する報告書
      2023 実施状況報告書
  • [備考] RadicalPy Docs

    • URL

      https://radicalpy.readthedocs.io/en/latest/

    • 関連する報告書
      2023 実施状況報告書
  • [備考] RadicalPy [source code]

    • URL

      https://github.com/Spin-Chemistry-Labs/radicalpy

    • 関連する報告書
      2022 実施状況報告書
  • [備考] RadicalPy [documentation]

    • URL

      https://radicalpy.readthedocs.io/en/latest/

    • 関連する報告書
      2022 実施状況報告書
  • [備考] RadicalPy [pypi package]

    • URL

      https://pypi.org/project/radicalpy/

    • 関連する報告書
      2022 実施状況報告書

URL: 

公開日: 2022-04-19   更新日: 2024-12-25  

サービス概要 検索マニュアル よくある質問 お知らせ 利用規程 科研費による研究の帰属

Powered by NII kakenhi