• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Large-scale machine learning system

Research Project

Project/Area Number 16K00116
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeMulti-year Fund
Section一般
Research Field Software
Research InstitutionNational Institute of Advanced Industrial Science and Technology

Principal Investigator

NAKADA HIDEMOTO  国立研究開発法人産業技術総合研究所, 情報・人間工学領域, 研究主幹 (80357631)

Project Period (FY) 2016-04-01 – 2019-03-31
Project Status Completed (Fiscal Year 2018)
Budget Amount *help
¥4,550,000 (Direct Cost: ¥3,500,000、Indirect Cost: ¥1,050,000)
Fiscal Year 2018: ¥1,300,000 (Direct Cost: ¥1,000,000、Indirect Cost: ¥300,000)
Fiscal Year 2017: ¥1,430,000 (Direct Cost: ¥1,100,000、Indirect Cost: ¥330,000)
Fiscal Year 2016: ¥1,820,000 (Direct Cost: ¥1,400,000、Indirect Cost: ¥420,000)
Keywords分散計算 / 機械学習 / ディープラーニング / 並列計算 / 並列システム / 分散システム / 強化学習 / 分散ファイルシステム / ネットワーク構成 / 耐故障性 / IoT
Outline of Final Research Achievements

Aiming at the construction of large-scale machine learning system, we conducted researches in the following three directions; 1) the system architecture in terms of network configuration, 2) effects on convergence, 3) machine learning applications.1) we investigated the relationship between network configuration and distribution method, using existing simulator. We found that relatively poor network configuration suffice the machine learning applications.2) we developed a novel hybrid simulator and investigated the effect, and found that the learning rate is quite important for parallelization. 3) we studied reinforcement learning and image generation.

Academic Significance and Societal Importance of the Research Achievements

ディープラーニングに代表される機械学習技術が広く普及しつつあるが、これらは大量の計算を伴うため並列分散化して実行することが非常に重要である。われわれは、大規模な並列分散機械学習システムを構成する方法に取り組み、このような計算システムを比較的安価かつ効率的に運用するために必要とされるハードウェアの構成を検討し、比較的安価なネットワークでも十分な性能が得られることを示した。さらに、パラメータを調整することで大規模に並列化しても計算の収束に影響しないように制御することが可能であることを示した。

Report

(4 results)
  • 2018 Annual Research Report   Final Research Report ( PDF )
  • 2017 Research-status Report
  • 2016 Research-status Report
  • Research Products

    (29 results)

All 2019 2018 2017 2016

All Journal Article (9 results) (of which Peer Reviewed: 3 results,  Acknowledgement Compliant: 3 results) Presentation (20 results) (of which Int'l Joint Research: 1 results)

  • [Journal Article] Storage-Side Processing for Spark with Tiered Storage2018

    • Author(s)
      Kaihui Zhang, Yusuke Tanimura, Hidemoto Nakada, Hirotaka Ogawa
    • Journal Title

      研究報告ハイパフォーマンス・コンピューティング(HPC)2017-HPC-163(7)

      Volume: 163 Pages: 1-6

    • Related Report
      2017 Research-status Report
  • [Journal Article] A Performance Evaluation of Distributed TensorFlow2018

    • Author(s)
      Tianlun Wang, Yusuke Tanimura, Hirotaka Ogawa, Hidemoto Nakada
    • Journal Title

      研究報告ハイパフォーマンス・コンピューティング(HPC)2017-HPC-161(1)

      Volume: 161 Pages: 1-6

    • Related Report
      2017 Research-status Report
  • [Journal Article] Toward image inbetweening using Latent Model2018

    • Author(s)
      Paulino Cristovao, Yusuke Tanimura, Hidemoto Nakada, Hideki Asoh
    • Journal Title

      信学技法 IEICE-PRMU2017-185, vol. IEICE-117, no.514

      Volume: 117 Pages: 79-84

    • Related Report
      2017 Research-status Report
  • [Journal Article] A study on Network Structure and Parameter Exchange Method in large-scale Cluster for Machine Learning2017

    • Author(s)
      Dou Zhang, Rei Mingxi, Yusuke Tanimura, Hidemoto Nakada
    • Journal Title

      信学技報, vol. 117, no. 153, CPSY2017-29

      Volume: 117 Pages: 145-150

    • Related Report
      2017 Research-status Report
  • [Journal Article] A Quantitative Analysis on Required Network Bandwidth for Large-Scale Parallel Machine Learning2017

    • Author(s)
      Mingxi Li, Yusuke Tanimura, Hidemoto Nakada
    • Journal Title

      MOD 2017 - (The Third International Conference on Machine Learning, Optimization and Big Data) , LNCS vol.10710

      Volume: 10710 Pages: 389-400

    • Related Report
      2017 Research-status Report
    • Peer Reviewed
  • [Journal Article] Understanding and Improving Disk-based Intermediate Data Caching in Spark2017

    • Author(s)
      Kaihui Zhang, Yusuke Tanimura, Hidemoto Nakada, Hirotaka Ogawa
    • Journal Title

      Scalable Cloud Data Management Workshop 2017 in IEEE BigData

      Volume: - Pages: 2426-2435

    • Related Report
      2017 Research-status Report
    • Peer Reviewed
  • [Journal Article] A Quantitative Analysis of Fault Tolerance Mechanisms for Parallel Machine Learning Systems with Parameter Servers2017

    • Author(s)
      Mingxi Li, Yuusuke Tanimura, Hidemoto Nakada
    • Journal Title

      Proc. of ACM IMCOM 2017

      Volume: -

    • Related Report
      2016 Research-status Report
    • Peer Reviewed / Acknowledgement Compliant
  • [Journal Article] Spark RDDの入出力性能の高速化に関する検討2016

    • Author(s)
      張 凱輝, 谷村 勇輔, 中田 秀基, 小川 宏高
    • Journal Title

      信学技報

      Volume: 177 Pages: 77-82

    • Related Report
      2016 Research-status Report
    • Acknowledgement Compliant
  • [Journal Article] パラメータサーバを用いた並列機械学習システムにおける耐故障性のシミュレーション2016

    • Author(s)
      黎 明曦, 谷村 勇輔, 中田 秀
    • Journal Title

      信学技報

      Volume: 177 Pages: 125-130

    • Related Report
      2016 Research-status Report
    • Acknowledgement Compliant
  • [Presentation] A Sub-policy Pruning Method for Meta Learning Shared Hierarchies2019

    • Author(s)
      Qing Hong,Yusuke Tanimura,Hidemoto Nakada
    • Organizer
      人工知能学会全国大会
    • Related Report
      2018 Annual Research Report
  • [Presentation] One-shot Learning using Triplet Network with kNN classifier2019

    • Author(s)
      Zhou Mu,Yusuke Tanimura,Hidemoto Nakada
    • Organizer
      人工知能学会全国大会
    • Related Report
      2018 Annual Research Report
  • [Presentation] Design a Loss Function which Generates a Spatial configuration of Image In-betweening2019

    • Author(s)
      Paulino Cristovao, Yusuke Tanimura, Hidemoto Nakada,Hideki Asoh
    • Organizer
      人工知能学会全国大会
    • Related Report
      2018 Annual Research Report
  • [Presentation] Wasserstein Autoencoderを用いた画像スタイル変換2019

    • Author(s)
      中田 秀基, 麻生 英樹
    • Organizer
      人工知能学会全国大会
    • Related Report
      2018 Annual Research Report
  • [Presentation] ニューラルネットワークを用いた任意人物・姿勢画像の生成2019

    • Author(s)
      中田 秀基,麻生 英樹
    • Organizer
      電子情報通信学会 パターン認識・メディア理解研究会
    • Related Report
      2018 Annual Research Report
  • [Presentation] Designing Loss Function which Generate Latent Space for image in-betweening2018

    • Author(s)
      Paulino Cristovao, Hidemoto Nakada, Yusuke Tanimura, Hideki Asoh
    • Organizer
      9th Annual Symosium of Indian Scientists association in japan
    • Related Report
      2018 Annual Research Report
  • [Presentation] One-shot Learning using Triplet Networks with KNN2018

    • Author(s)
      Mu Zhou, Hidemoto Nakada, Yusuke Tanimura
    • Organizer
      第21回情報論的学習理論ワークショップ(IBIS2018)ディスカッショントラック
    • Related Report
      2018 Annual Research Report
  • [Presentation] Designing Loss Function which Generate Latent Space for image in-betweening2018

    • Author(s)
      Paulino Cristovao, Hidemoto Nakada, Yusuke Tanimura, Hideki Asoh
    • Organizer
      第21回情報論的学習理論ワークショップ(IBIS2018)ディスカッショントラック
    • Related Report
      2018 Annual Research Report
  • [Presentation] Adaptation of Ray, a distributed framework for machine learning, to MPI-based environment2018

    • Author(s)
      Tianlun WANG, Yusuke Tanimura, Hidemoto Nakada
    • Organizer
      信学技法 IEICE-CPSY
    • Related Report
      2018 Annual Research Report
  • [Presentation] Asynchronous Deep Learning Test-bed to Analyze Gradient Staleness Effect2018

    • Author(s)
      Duo Zhang, Yusuke Tanimura, Hidemoto Nakada
    • Organizer
      信学技法 IEICE-CPSY
    • Related Report
      2018 Annual Research Report
  • [Presentation] Sub-policy pruning in Meta Learning Shared Hierarchies2018

    • Author(s)
      Ging Hong, Yusuke Tanimura, Hidemoto Nakada
    • Organizer
      34th meeting of the Pacific Rim Applications and Grid Middleware Assembly (PRAGMA 34)
    • Related Report
      2018 Annual Research Report
  • [Presentation] Toward image inbetweening using Latent Model2018

    • Author(s)
      Paulino Cristovao, Yusuke Tanimura, Hidemoto Nakada, Hideki Asoh
    • Organizer
      34th meeting of the Pacific Rim Applications and Grid Middleware Assembly (PRAGMA 34)
    • Related Report
      2018 Annual Research Report
  • [Presentation] Sub-policy pruning in Meta Learning Shared Hierarchies2018

    • Author(s)
      Ging Hong, Yusuke Tanimura, Hidemoto Nakada
    • Organizer
      人工知能学会汎用人工知能研究会
    • Related Report
      2017 Research-status Report
  • [Presentation] Spark RDD の入出力性能の高速化2017

    • Author(s)
      張 凱輝, 谷村 勇輔, 中田 秀基, 小川 宏高
    • Organizer
      cross-disciplinary Workshop on Computing Systems, Infrastructures, and Programming
    • Place of Presentation
      虎ノ門ヒルズフォーラム(東京)
    • Year and Date
      2017-04-24
    • Related Report
      2016 Research-status Report
  • [Presentation] 大規模機械学習向けクラスタにおけるネットワークバンド幅とパラメータ交換手法に関する考察2017

    • Author(s)
      黎 明曦, 谷村 勇輔, 中田 秀基
    • Organizer
      cross-disciplinary Workshop on Computing Systems, Infrastructures, and Programming
    • Place of Presentation
      虎ノ門ヒルズフォーラム(東京)
    • Year and Date
      2017-04-24
    • Related Report
      2016 Research-status Report
  • [Presentation] Spark におけるディスクを用いた RDD キャッシングの高速化と効果的な利用に関する検討2017

    • Author(s)
      張 凱輝, 谷村 勇輔, 中田 秀基, 小川 宏高
    • Organizer
      cross-disciplinary Workshop on Computing Systems, Infrastructures, and Programming
    • Place of Presentation
      虎ノ門ヒルズフォーラム(東京)
    • Year and Date
      2017-04-24
    • Related Report
      2016 Research-status Report
  • [Presentation] Spark におけるディスクを用いた RDD キャッシングの高速化と 効果的な利用に関する検討2017

    • Author(s)
      張 凱輝, 谷村 勇輔, 中田 秀基, 小川 宏高
    • Organizer
      cross-disciplinary Workshop on Computing Systems, Infrastructures, and Programming
    • Related Report
      2017 Research-status Report
  • [Presentation] 大規模機械学習向けクラスタにおけるネットワーク構造とパラメータ交換手法2017

    • Author(s)
      黎 明曦, 谷村 勇輔, 中田 秀基
    • Organizer
      cross-disciplinary Workshop on Computing Systems, Infrastructures, and Programming
    • Related Report
      2017 Research-status Report 2016 Research-status Report
  • [Presentation] A study on the performance of DDPG (Deep Deterministic Policy Grandient)2017

    • Author(s)
      Ging Hong, Yusuke Tanimura, Hidemoto Nakada
    • Organizer
      第20回情報論的学習理論ワークショップ ディスカッショントラック
    • Related Report
      2017 Research-status Report
  • [Presentation] How Much Should We Invest for Network Facility: Quantitative Analysis on Network 'Fatness' and Machine Learning Performance2017

    • Author(s)
      Duo Zhang, Mingxi LI, Yusuke Tanimura, Hidemoto Nakada
    • Organizer
      Workshop on ML Systems in NIPS 2017
    • Related Report
      2017 Research-status Report
    • Int'l Joint Research

URL: 

Published: 2016-04-21   Modified: 2020-03-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi