• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Distributed Data Management Library for Large-Scale Many-Core Clusters and its Integration with Dynamic Load Balancers

Research Project

Project/Area Number 20K11841
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeMulti-year Fund
Section一般
Review Section Basic Section 60090:High performance computing-related
Research InstitutionKonan University (2022)
Kobe University (2020-2021)

Principal Investigator

Kamada Tomio  甲南大学, 知能情報学部, 准教授 (20304131)

Co-Investigator(Kenkyū-buntansha) Finnerty Patrick・Martin  神戸大学, システム情報学研究科, 助教 (50957628)
Project Period (FY) 2020-04-01 – 2023-03-31
Project Status Completed (Fiscal Year 2022)
Budget Amount *help
¥4,420,000 (Direct Cost: ¥3,400,000、Indirect Cost: ¥1,020,000)
Fiscal Year 2022: ¥780,000 (Direct Cost: ¥600,000、Indirect Cost: ¥180,000)
Fiscal Year 2021: ¥650,000 (Direct Cost: ¥500,000、Indirect Cost: ¥150,000)
Fiscal Year 2020: ¥2,990,000 (Direct Cost: ¥2,300,000、Indirect Cost: ¥690,000)
Keywords動的負荷分散 / 分散集合ライブラリ / メニーコアクラスタ / 通信と計算のオーバーラップ / メニ―コアクラスタ / メニ―コア / 分散データ / データ再配置
Outline of Research at the Start

本研究の目的は,社会事象シミュレーションのように不規則性の強い問題を対象に,
プログラマが容易に大規模分散データを作成し,計算状況に応じて容易に適切な計算ノードにデータを配置できる環境を実現することにある.
データの論理的な構造とデータ配置を分離し,計算局面に応じて各データ部位を計算ノードに(再)配置したりキャッシュすることができ,動的負荷分散などによるタスク移動との連携も可能とする.

Outline of Final Research Achievements

This research aims to provide distributed collection libraries for many-core large-scale clusters and enables dynamic load-balancing over them. We developed (1) a self-adjusting task granularity mechanism for our global load balancer library to avoid contention on many‐core clusters and (2) a series of relocatable distributed collections featuring inter/intra-node parallelism.
For integrating load-balancer and distributed collections, we developed (3a) a global load balancer for distributed arrays involving range-based element relocation. In addition, we developed (3b) a distributed cell set that allows the easy description of communication/computation overlapping and relationships between inter-node communication and its dependent computations.

Academic Significance and Societal Importance of the Research Achievements

今後、スーパーコンピュータの用途が広がりや計算の高知能化により、並列プログラムの不規則化が予想される。例えば、状況に応じて大規模な計算をする知的なエージェントをシミュレートする場合、負荷状況に応じた計算資源の再割り当てが必要になる。一方で、現在のスーパーコンピュータでは、メニーコアプロセッサが一般化するなど、より大規模化・複雑化が進んでいる。
ノード間にまたがるデータ・タスク配置管理の今後のさらなる複雑化に対応するため、本研究では、メニーコア環境における要素の再配置可能な大規模分散データ管理ライブラリを開発するとともに、自動負荷分散機構との融合に向けた研究をおこなった。

Report

(4 results)
  • 2022 Annual Research Report   Final Research Report ( PDF )
  • 2021 Research-status Report
  • 2020 Research-status Report
  • Research Products

    (19 results)

All 2023 2022 2021 2020 Other

All Journal Article (6 results) (of which Int'l Joint Research: 1 results,  Peer Reviewed: 6 results,  Open Access: 4 results) Presentation (5 results) (of which Int'l Joint Research: 4 results) Remarks (8 results)

  • [Journal Article] Automatically balancing relocatable distributed collections2023

    • Author(s)
      Finnerty Patrick、Kamada Tomio、Ohta Chikara
    • Journal Title

      Concurrency and Computation: Practice and Experience

      Volume: n/a Issue: 27

    • DOI

      10.1002/cpe.7717

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Topic-based distributed publish-process-subscribe system with metrics on geographic distance and permissible delay2022

    • Author(s)
      Matsumoto Ryota、Kamada Tomio、Finnerty Patrick、Ohta Chikara
    • Journal Title

      IEICE Communications Express

      Volume: 11 Issue: 12 Pages: 748-753

    • DOI

      10.1587/comex.2022COL0009

    • ISSN
      2187-0136
    • Year and Date
      2022-12-01
    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Supercharging the APGAS Programming Model with Relocatable Distributed Collections2022

    • Author(s)
      Finnerty Patrick、Kawanishi Yoshiki、Kamada Tomio、Ohta Chikara
    • Journal Title

      Scientific Programming

      Volume: 2022 Pages: 1-27

    • DOI

      10.1155/2022/5092422

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Topic allocation method on edge servers for latency‐sensitive notification service2021

    • Author(s)
      Tanaka Tomoya、Kamada Tomio、Ohta Chikara
    • Journal Title

      International Journal of Network Management

      Volume: 31 Issue: 6

    • DOI

      10.1002/nem.2173

    • Related Report
      2021 Research-status Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] A self‐adjusting task granularity mechanism for the Java lifeline‐based global load balancer library on many‐core clusters2021

    • Author(s)
      Finnerty Patrick、Kamada Tomio、Ohta Chikara
    • Journal Title

      Concurrency and Computation: Practice and Experience

      Volume: - Issue: 2

    • DOI

      10.1002/cpe.6224

    • Related Report
      2020 Research-status Report
    • Peer Reviewed
  • [Journal Article] Distributed topic management in publish-process-subscribe systems on edge-servers for real-time notification service2020

    • Author(s)
      Tanaka Tomoya、Kamada Tomio、Ohta Chikara
    • Journal Title

      IEICE Communications Express

      Volume: 9 Issue: 12 Pages: 616-621

    • DOI

      10.1587/comex.2020COL0040

    • NAID

      130007949083

    • ISSN
      2187-0136
    • Year and Date
      2020-12-01
    • Related Report
      2020 Research-status Report
    • Peer Reviewed / Open Access
  • [Presentation] Distributed Cell Set : A Library for Space-Dependent Communication/Computation Overlap on Many Core Cluster2023

    • Author(s)
      Yoshiki Kawanishi, Patrick Finnerty, Tomio Kamada, Chikara Ohta
    • Organizer
      The 14th International Workshop on Programming Models and Applications for Multicores and Manycores (PMAM `23)
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Integrating a global load balancer to an APGAS distributed collections library2022

    • Author(s)
      Patrick Finnerty, Tomio Kamada, and Chikara Ohta
    • Organizer
      n Proceedings of the Thirteenth International Workshop on Programming Models and Applications for Multicores and Manycores (PMAM ‘22)
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Integrating a global load balancer to an APGAS distributed collections library2022

    • Author(s)
      Patrick Finnerty, Tomio Kamada, and Chikara Ohta
    • Organizer
      Proceedings of the Thirteenth International Workshop on Programming Models and Applications for Multicores and Manycores (PMAM '22)
    • Related Report
      2021 Research-status Report
    • Int'l Joint Research
  • [Presentation] Experience in testing MPI+Java parallel and distributed programs with JUnit2021

    • Author(s)
      Patrick Finnerty, Yoshiki Kawanishi, Tomio Kamada, Chikara Ohta
    • Organizer
      Summer United Workshops on Parallel, Distributed and Cooperative Processing (SWoPP2021), 2021
    • Related Report
      2021 Research-status Report
  • [Presentation] Topic-based Allocation of Distributed Message Processors on Edge-Servers for Real-time Notification Service2020

    • Author(s)
      Tomoya Tanaka, Tomio Kamada, Chikara Ohta
    • Organizer
      Proc. of The 21st Asia-Pacific Network Operations and Management Symposium (APNOMS 2020)
    • Related Report
      2020 Research-status Report
    • Int'l Joint Research
  • [Remarks] Java GLB (動的負荷分散ライブラリ)

    • URL

      https://github.com/handist/JavaGLB

    • Related Report
      2022 Annual Research Report
  • [Remarks] Handist Collections (分散集合ライブラリ)

    • URL

      https://github.com/handist/collections

    • Related Report
      2022 Annual Research Report 2021 Research-status Report
  • [Remarks] MPI Junit (MPI プログラムのテスト環境)

    • URL

      https://github.com/handist/mpi-junit

    • Related Report
      2022 Annual Research Report 2021 Research-status Report
  • [Remarks] Hnadist Collections Benchmarks (ベンチマーク群)

    • URL

      https://github.com/handist/collections-benchmarks

    • Related Report
      2022 Annual Research Report
  • [Remarks] PlhamJ (人工市場シミュレーション基盤)

    • URL

      https://github.com/plham/plhamJ

    • Related Report
      2022 Annual Research Report 2021 Research-status Report
  • [Remarks] Handist Collections Benchmarks (ベンチマーク)

    • URL

      https://github.com/handist/collections-benchmarks

    • Related Report
      2021 Research-status Report
  • [Remarks] Java GLB

    • URL

      https://github.com/handist/JavaGLB

    • Related Report
      2020 Research-status Report
  • [Remarks] Handist Collections

    • URL

      https://github.com/handist/collections

    • Related Report
      2020 Research-status Report

URL: 

Published: 2020-04-28   Modified: 2024-01-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi