Characterization of data and application behaviors of large stream data analyses(Fostering Joint International Research)

2022 Fiscal Year Final Research Report

• Project/Area Number: 16KK0008
• Research Category: Fund for the Promotion of Joint International Research (Fostering Joint International Research)
• Allocation Type: Multi-year Fund
• Research Field: Software
• Research Institution: Meiji University
• Principal Investigator: Akioka Sayaka 明治大学, 総合数理学部, 専任教授 (90333533)
• Project Period (FY): 2017 – 2022
• Keywords: 大規模データストリーム解析 / モデル化

Outline of Final Research Achievements

The problem of this project arose from the experience in another large-scale data stream analysis project. That is, the behavior of analysis programs can vary significantly depending on the characteristics of the input data. Such tendencies can also be observed in other applications. In particular, in sparse matrix computations, it is quite common to switch the preconditioners based on the characteristics of the input matrix. Therefore, the collaboration with experts in sparse matrix computations is expected to be a great way to attack the project target. In the COVID-19 pandemic, the collaboration became difficult. Therefore, we explored various applications beyond data stream analysis, and attempted to model the target based on their characteristic differences. However, we were unable to obtain a clear model by the end of this project.

Free Research Field

並列分散処理

Academic Significance and Societal Importance of the Research Achievements

昨今、社会的重要性を高めている機械学習や強化学習のプログラム的特徴は、本プロジェクトで対象とした大規模データストリーム解析とよく似ている。機械学習や強化学習は、その学習過程で並列化による高速化を行うことが難しい部分も多く、部分的な高速化しか成し得ていない。また、効率よく優れたモデルを獲得するためには、学習データの順序や選択について、知見に基づいた試行錯誤が必要な場合も多い。つまり、大規模データストリーム解析と入力データの挙動をモデリングすることは、機械学習や強化学習の高速化や効率化に繋がる。チャレンジングな問題ではあるが、様々なアプローチを模索しながら、引き続きこの問題に取り組んで行きたい。