• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Life-Long Deep Learning using Bayesian Principles

Research Project

Project/Area Number 20H04247
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeSingle-year Grants
Section一般
Review Section Basic Section 61030:Intelligent informatics-related
Research InstitutionInstitute of Physical and Chemical Research

Principal Investigator

Khan Emtiyaz  国立研究開発法人理化学研究所, 革新知能統合研究センター, チームリーダー (30858022)

Co-Investigator(Kenkyū-buntansha) Alquier Pierre  国立研究開発法人理化学研究所, 革新知能統合研究センター, 研究員 (10865645)
横田 理央  東京工業大学, 学術国際情報センター, 教授 (20760573)
Project Period (FY) 2020-04-01 – 2023-03-31
Project Status Completed (Fiscal Year 2022)
Budget Amount *help
¥18,200,000 (Direct Cost: ¥14,000,000、Indirect Cost: ¥4,200,000)
Fiscal Year 2022: ¥2,470,000 (Direct Cost: ¥1,900,000、Indirect Cost: ¥570,000)
Fiscal Year 2021: ¥3,640,000 (Direct Cost: ¥2,800,000、Indirect Cost: ¥840,000)
Fiscal Year 2020: ¥12,090,000 (Direct Cost: ¥9,300,000、Indirect Cost: ¥2,790,000)
KeywordsContinual learning / Bayesian deep learning / Lifelong learning / continual learning / deep learning / Deep Learning / Continual Learning / Bayesian principles / adaptation / lifelong learning / reinforcement learning / active learning
Outline of Research at the Start

By using Bayesian principles to “identify, memorize, and recall” useful past experiences during training, our goal is to design life-long learning AI systems. We expect our new methods to enable application of deep learning in more realistic settings than before.

Outline of Final Research Achievements

Current deep learning method cannot learn continually, and can easily forget the past information seen a long time ago. We developed new methods for continual deep learning where we reduce the forgetting. We do so by identifying and reusing a memory of the past. We show that our methods are universal, that is, any method that work well must have similar properties to ours. Our method is scalable and can be applied in practical settings.

Academic Significance and Societal Importance of the Research Achievements

Deep-learning methods require a huge amount of computing resources and also a lot of data. Our work reduces the dependencies on such resources. We aim to design AI systems that continue to learn and improve throughout their lifetime.

Report

(4 results)
  • 2022 Annual Research Report   Final Research Report ( PDF )
  • 2021 Annual Research Report
  • 2020 Annual Research Report
  • Research Products

    (5 results)

All 2021 2020

All Journal Article (2 results) (of which Int'l Joint Research: 2 results,  Peer Reviewed: 2 results,  Open Access: 2 results) Presentation (3 results) (of which Invited: 3 results)

  • [Journal Article] Knowledge-Adaptation Priors2021

    • Author(s)
      Khan, Mohammad Emtiyaz E and Swaroop, Siddharth
    • Journal Title

      Advances in Neural Information Processing Systems

      Volume: 34

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Continual deep learning by functional regularization of the memorable past2020

    • Author(s)
      Pan, P., Swaroop, S., Immer, A., Eschenhagen, R., Turner, R. and Khan, M. E.
    • Journal Title

      Advances in Neural Information Processing Systems

      Volume: 374 Pages: 4453-4464

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Presentation] K-priors: A General Principle of Adaptation2021

    • Author(s)
      Mohammad Emtiyaz Khan
    • Organizer
      ICML 2021 workshop on Theory of Continual Learning
    • Related Report
      2021 Annual Research Report
    • Invited
  • [Presentation] K-priors: A General Principle of Adaptation2021

    • Author(s)
      Mohammad Emtiyaz Khan
    • Organizer
      KDD 2021 Workshop on Model Mining
    • Related Report
      2021 Annual Research Report
    • Invited
  • [Presentation] Adaptive and Robust (Deep) Learning with Bayes2021

    • Author(s)
      Mohammad Emtiyaz Khan, Dharmesh Tailor, Siddharth Swaroop
    • Organizer
      NeurIPS 2021 Bayesian deep learning workshop
    • Related Report
      2021 Annual Research Report
    • Invited

URL: 

Published: 2020-04-28   Modified: 2024-01-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi