Theoretical Analysis of Transfer Learning and Its Applications

2022 Fiscal Year Final Research Report

• Project/Area Number: 17K12653
• Research Category: Grant-in-Aid for Young Scientists (B)
• Allocation Type: Multi-year Fund
• Research Field: Statistical science
• Research Institution: The University of Tokyo (2020-2022); Institute of Physical and Chemical Research (2017-2019)
• Principal Investigator: Kumagai Wataru 東京大学, 大学院工学系研究科(工学部), 特任助教 (20747167)
• Project Period (FY): 2017-04-01 – 2023-03-31
• Keywords: 転移学習 / メタ学習 / パラメータ転移

Outline of Final Research Achievements

As the first main result, we derived theoretical bounds for transferring parametric models across domains. Notably, this can involve employing complex parametric models as feature extractors, enabling the theoretical treatment of deep neural networks and sparse coding. As the second main result, we demonstrated the universality of meta-learners when considering an algebraic property called equivariance. Equivariance naturally emerges in data processing and natural processes, and it offers the advantage of enhancing learning efficiency through processing with covariant neural architectures. As the third main result, we derived a decomposition theorem for the difference in expected risks with respect to joint distributions.

Free Research Field

人工知能

Academic Significance and Societal Importance of the Research Achievements

初めに学術的意義について述べる．転移学習は現在の機械学習や人工知能の研究において欠かせない技術である．本研究により，転移学習の理論的側面の一端が明らかになり，効率的なモデルの構築や転移学習手法の構築に資することが期待できる．特にメタ学習において同変性を用いた新規のモデルを提案したが，これはデータ内の対称性という代数的性質を学習の効率化に結びつけるために重要な結果と言える．
次に社会的な意義について述べる．転移学習技術は多数のドメインでの学習をサポートするもので，幅広い応用で成功を収めている．本研究結果は今後の転移学習の応用においてその理論的基盤の構築に貢献するものである．