• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Knowledge mining of large-scale network operational data for troubleshooting and predictive analysis

Research Project

Project/Area Number 19K20262
Research Category

Grant-in-Aid for Early-Career Scientists

Allocation TypeMulti-year Fund
Review Section Basic Section 60060:Information network-related
Research InstitutionNational Institute of Informatics

Principal Investigator

Kobayashi Satoru  国立情報学研究所, アーキテクチャ科学研究系, 特任研究員 (40824107)

Project Period (FY) 2019-04-01 – 2021-03-31
Project Status Completed (Fiscal Year 2020)
Budget Amount *help
¥4,160,000 (Direct Cost: ¥3,200,000、Indirect Cost: ¥960,000)
Fiscal Year 2020: ¥1,820,000 (Direct Cost: ¥1,400,000、Indirect Cost: ¥420,000)
Fiscal Year 2019: ¥2,340,000 (Direct Cost: ¥1,800,000、Indirect Cost: ¥540,000)
Keywordsネットワーク運用 / データマイニング / 因果推論 / 転移学習 / ログ解析 / 異常検知 / インターネット / 知識ベ一スシステム
Outline of Research at the Start

大規模ネットワークの運用支援のため障害の検知や原因究明を実現するには、運用データから機械的に情報抽出を行う必要がある。本研究では因果推論に基づく解析技術により運用データからシステムの振る舞いをより直感的な形で抽出する技術の確立を目指す。特に多様な運用データをスケーラブルに扱うドメイン間因果推論技術、および過去の振る舞いを元に情報の具体化や予兆解析を実現する因果知識ベースの技術の開発に取り組む。

Outline of Final Research Achievements

In this research, we proposed a new analysis approach for network operational data on the basis of causal inference, to help troubleshooting of large-scale networks. We proposed two analysis frameworks: "amulog", a general log analysis framework to estimate log templates and classify messages with them in online processing, and "logdag", a causal analysis framework designed for network operational data including logs and measured SNMP data. The proposed technology will help operators to extract contextual information in operational data automatically.

Academic Significance and Societal Importance of the Research Achievements

ネットワーク運用データに見られるシステムの振る舞いが、データからより直感的な関係性として示されあるいは視覚化することが可能となり、ネットワーク障害の実態を素早く把握し対処する上で大きな助けになると期待される。また特にログ解析に関する知見はネットワーク以外のシステム運用分野にも応用可能であり、データドリヴンなシステム運用自動化を助ける重要な技術として幅広く活用されることが期待される。

Report

(3 results)
  • 2020 Annual Research Report   Final Research Report ( PDF )
  • 2019 Research-status Report
  • Research Products

    (9 results)

All 2021 2020 2019 Other

All Journal Article (1 results) (of which Peer Reviewed: 1 results) Presentation (6 results) (of which Int'l Joint Research: 4 results,  Invited: 1 results) Remarks (2 results)

  • [Journal Article] Latent Variable Based Anomaly Detection in Network System Logs2019

    • Author(s)
      Kazuki Otomo, Satoru Kobayashi, Kensuke Fukuda, Hiroshi Esaki
    • Journal Title

      IEICE Transactions on Information and Systems

      Volume: E102.D Issue: 9 Pages: 1644-1652

    • DOI

      10.1587/transinf.2018OFP0007

    • NAID

      130007699784

    • ISSN
      0916-8532, 1745-1361
    • Year and Date
      2019-09-01
    • Related Report
      2019 Research-status Report
    • Peer Reviewed
  • [Presentation] Latent Semantics Approach for Network Log Analysis: Modeling and its application2021

    • Author(s)
      Kazuki Otomo, Satoru Kobayashi, Kensuke Fukuda, Hiroshi Esaki
    • Organizer
      17th IFIP/IEEE International Symposium on Integrated Network Management (IM2021)
    • Related Report
      2020 Annual Research Report
    • Int'l Joint Research
  • [Presentation] LogDTL: Network Log Template Generation with Deep Transfer Learning2021

    • Author(s)
      Thieu Nguyen, Satoru Kobayashi, Kensuke Fukuda
    • Organizer
      6th IEEE/IFIP International Workshop on Analytics for Network and Service Management (AnNet 2021)
    • Related Report
      2020 Annual Research Report
    • Int'l Joint Research
  • [Presentation] amulog: A General Log Analysis Framework for Diverse Template Generation Methods2020

    • Author(s)
      Satoru Kobayashi, Yuya Yamashiro, Kazuki Otomo, Kensuke Fukuda
    • Organizer
      16th International Conference on Network and Service Management (CNSM2020)
    • Related Report
      2020 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Causal Analysis of Network Log Events2020

    • Author(s)
      Satoru Kobayashi
    • Organizer
      JFLI Workshop 2020 on Next Generation Networking
    • Related Report
      2019 Research-status Report
    • Invited
  • [Presentation] Causal analysis of network logs with layered protocols and topology knowledge2019

    • Author(s)
      Satoru Kobayashi, Kazuki Otomo, Kensuke Fukuda
    • Organizer
      15th International Conference on Network and Service Management (CNSM'19)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] Bridged Refinementによるログテンプレート推定手法の検討2019

    • Author(s)
      山城 裕陽, 小林 諭, 福田 健介, 江崎 浩
    • Organizer
      電子情報通信学会IA研究会
    • Related Report
      2019 Research-status Report
  • [Remarks] amulog

    • URL

      https://github.com/cpflat/amulog

    • Related Report
      2020 Annual Research Report
  • [Remarks] logdag

    • URL

      https://github.com/cpflat/logdag

    • Related Report
      2019 Research-status Report

URL: 

Published: 2019-04-18   Modified: 2022-01-27  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi