A Coordination Structure for MPI Execution and Communication on OpenFlow-based Cluster System

• Project/Area Number: 17K00168
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Multi-year Fund
• Section: 一般
• Research Field: High performance computing
• Research Institution: Osaka University
• Principal Investigator: Susumu Date 大阪大学, サイバーメディアセンター, 准教授 (20346175)
• Project Period (FY): 2017-04-01 – 2023-03-31
• Project Status: Completed (Fiscal Year 2022)
• Budget Amount: ¥4,680,000 (Direct Cost: ¥3,600,000、Indirect Cost: ¥1,080,000); Fiscal Year 2019: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000); Fiscal Year 2018: ¥1,430,000 (Direct Cost: ¥1,100,000、Indirect Cost: ¥330,000); Fiscal Year 2017: ¥2,080,000 (Direct Cost: ¥1,600,000、Indirect Cost: ¥480,000)
• Keywords: MPI / SDN / OpenFlow / 分散処理

Outline of Final Research Achievements

In this research, acceleration of MPI communication was achieved based on a novel approach of seamlessly coordinating MPI program characteristics and network programmability brought by SDN. By overturning the assumption that network is a static resource that cannot be controlled, we dynamically control packet flows on the interconnect in a dynamic programming manner

Academic Significance and Societal Importance of the Research Achievements

ネットワークは制御できない静的な資源であるという前提から、動的に制御できる資源であるという視点に基づき、クラスタシステムのインターコネクト (相互結合網)を動的に制御可能なネットワーク資源ととらえることにより、近年の高性能計算機システム上での並列分散計算でのデファクトスタンダードなmessage passing方式に基づくMPIプログラムにより発生するパケットフローを動的に制御する。このことにより、新たな並列分散計算基盤の可能性を示した。「

Research Products

• [Journal Article] Dynamic Traffic Control of Staging Traffic on the Interconnect of the HPC Cluster System (2020)
  • Author(s): Endo Arata、Ohtsuji Hiroki、Hayashi Erika、Yoshida Eiji、Lee Chunghan、Date Susumu、Shimojo Shinji
  • Journal Title: IEEE Access Volume: 8 Pages: 198518-198531
  • DOI: 10.1109/access.2020.3035158
  • Open Access
• [Journal Article] UnisonFlow: A Software-Defined Coordination Mechanism for Message-Passing Communication and Computation (2018)
  • Author(s): Keichi Takahashi, Susumu Date, Dashdavaa Khureltulga, Yoshiyuki Kido, Hiroaki Yamanaka, Eiji Kawai, Shinji Shimojo
  • Journal Title: IEEE Access Volume: 印刷中 Pages: 23372-23382
  • DOI: 10.1109/access.2018.2829532
  • Peer Reviewed / Open Access
• [Presentation] Research Experiences of an On-Time Data Transfer Framework in Cooperation with Scheduler System and Future Directions (2022)
  • Author(s): Susumu Date
  • Organizer: The 3rd Global Research Platform Workshop (3GRP)
  • Int'l Joint Research
• [Presentation] Towards Practical Interconnect Design: Integrating Job Scheduling and Network Simulation (2022)
  • Author(s): Kohei Taniguchi, Arata Endo, Chunghan Lee, Masatoshi Saitoh, Susumu Date
  • Organizer: The 23rd International Conference on Parallel and Distributed Computing, Applications and Technologies (PDCAT’22),
  • Int'l Joint Research
• [Presentation] Prototyping an MPI-communication Logging Module on Data Processing Unit (2022)
  • Author(s): Kazuki Takashima, Arata Endo, Susumu Date
  • Organizer: The 23rd International Conference on Parallel and Distributed Computing, Applications and Technologies (PDCAT’22)
  • Int'l Joint Research
• [Presentation] DPUを用いたMPI通信ロギング手法の試作 (2022)
  • Author(s): 高嶋 和貴，遠藤 新, 伊達 進，下條 真司
  • Organizer: 第155回 システムソフトウェアとオペレーティング・システム研究会
• [Presentation] Osaka University’s Research Infrastructure Towards Acceleration of Global e-Science Research (2021)
  • Author(s): Shinji Shimojo, Susumu Date,
  • Organizer: 17th IEEE International Conference on eScience 2021
  • Int'l Joint Research
• [Presentation] ステージングによるトラヒック競合を自動抽出可能なパケットフロー分析ツール (2019)
  • Author(s): 遠藤 新, Chunghan Lee, 伊達 進, 木戸 善之, 渡場 康弘, 下條 真司
  • Organizer: 第17回 ディペンダブルシステムワークショップ, 日本ソフトウェア科学会
• [Presentation] Integrating SDN-Enhanced MPI with Job Scheduler to Support Shared Clusters (2019)
  • Author(s): Keichi Takahashi, Susumu Date, Yasuhiro Watashiba, Yoshiyuki Kido, Shinji Shimojo
  • Organizer: 28th Workshop on Sustained Simulation Performance
  • Int'l Joint Research
• [Presentation] A Traffic Simulator with Intra-node Parallelism for Designing High-performance Interconnects (2018)
  • Author(s): Yohei Takigawa, Keichi Takahashi, Susumu Date, Yoshiyuki Kido, Shinji Shimojo
  • Organizer: The 2018 International Conference on High Performance Computing & Simulation (HPCS 2018)
  • Int'l Joint Research
• [Presentation] インターコネクトにおけるステージング I/O 通信とノード間通信の分離による性能評価 (2018)
  • Author(s): 遠藤新, 伊達進, 木戸善之, 渡場康弘, 下條真司
  • Organizer: 日本ソフトウェア科学会 第16回ディペンダブルシステムワークショップ
• [Presentation] PFAnalyzer: A Toolset for Analyzing Application-aware Dynamic Interconnects (2017)
  • Author(s): Keichi Takahashi, Susumu Date, Khureltulga Dashdavaa, Yoshiyuki Kido, Shinji Shimojo
  • Organizer: The Monitoring and Analysis for High Performance Computing Systems Plus Applications (HPCMASPA) Workshop, Cluster 2017
  • Int'l Joint Research