• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Resolution of singularities in neural networks

Research Project

Project/Area Number 16K00347
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeMulti-year Fund
Section一般
Research Field Soft computing
Research InstitutionRikkyo University (2021-2022)
National Institute of Advanced Industrial Science and Technology (2016-2020)

Principal Investigator

NITTA Tohru  立教大学, 人工知能科学研究科, 特任教授 (20357726)

Project Period (FY) 2016-04-01 – 2023-03-31
Project Status Completed (Fiscal Year 2022)
Budget Amount *help
¥4,550,000 (Direct Cost: ¥3,500,000、Indirect Cost: ¥1,050,000)
Fiscal Year 2020: ¥780,000 (Direct Cost: ¥600,000、Indirect Cost: ¥180,000)
Fiscal Year 2019: ¥780,000 (Direct Cost: ¥600,000、Indirect Cost: ¥180,000)
Fiscal Year 2018: ¥780,000 (Direct Cost: ¥600,000、Indirect Cost: ¥180,000)
Fiscal Year 2017: ¥1,430,000 (Direct Cost: ¥1,100,000、Indirect Cost: ¥330,000)
Fiscal Year 2016: ¥780,000 (Direct Cost: ¥600,000、Indirect Cost: ¥180,000)
Keywordsニューラルネットワーク / 特異点 / 深層学習 / ディープラーニング / 危点 / 学習 / 人工知能 / 機械学習 / ソフトコンピューティング / アルゴリズム
Outline of Final Research Achievements

We have revealed some of the characteristics of singularities in neural networks. In other words, we mathematically clarified that deep neural networks have a number of "hierarchical structure-based singularities" and derived the sufficient conditions for not having such singularities. In addition, we mathematically showed that some singularities in deep real-valued neural networks, which are equivalent to deep complex-valued neural networks, are naturally resolved due to their complex number-based nature. Furthermore, we derived the sufficient conditions for nonlinear deep neural networks not to have bad local minima with large learning errors which have a bad influence on learning performance.

Academic Significance and Societal Importance of the Research Achievements

深層学習技術では、大量のパラメータの調整が必要であるため、学習が行える適切な条件を特定するのに大変な労力がかかっている。ニューラルネットワークには、学習に悪い影響を与える多くの特異点が存在するからである。本研究では、ニューラルネットワークの特異点の特性を明らかにし、特異点を持たないための十分条件を導いた。また、特異点を持たないタイプのニューラルネットワークを提示した。これらの研究成果は深層ニューラルネットワークの学習性能向上に、引いては現在世界的に進められている深層学習を利用した社会実装に資するものと考えられる。

Report

(8 results)
  • 2022 Annual Research Report   Final Research Report ( PDF )
  • 2021 Research-status Report
  • 2020 Research-status Report
  • 2019 Research-status Report
  • 2018 Research-status Report
  • 2017 Research-status Report
  • 2016 Research-status Report
  • Research Products

    (20 results)

All 2023 2022 2021 2020 2019 2018 2017 2016

All Journal Article (5 results) (of which Int'l Joint Research: 1 results,  Peer Reviewed: 5 results,  Open Access: 1 results,  Acknowledgement Compliant: 1 results) Presentation (15 results) (of which Int'l Joint Research: 6 results,  Invited: 2 results)

  • [Journal Article] Proposal of fully augmented complex-valued neural networks2023

    • Author(s)
      Nitta Tohru
    • Journal Title

      Nonlinear Theory and Its Applications, IEICE

      Volume: 14 Issue: 2 Pages: 175-192

    • DOI

      10.1587/nolta.14.175

    • ISSN
      2185-4106
    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Hypercomplex Widely Linear Estimation Through the Lens of Underpinning Geometry2019

    • Author(s)
      Tohru Nitta, Masaki Kobayashi, Danilo P. Mandic
    • Journal Title

      IEEE Transactions on Signal Processing

      Volume: 67 Issue: 15 Pages: 3985-3994

    • DOI

      10.1109/tsp.2019.2922151

    • Related Report
      2019 Research-status Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Resolution of Singularities via Deep Complex-Valued Neural Networks2018

    • Author(s)
      Nitta Tohru
    • Journal Title

      Mathematical Methods in the Applied Sciences

      Volume: 41 Issue: 11 Pages: 4170-4178

    • DOI

      10.1002/mma.4434

    • Related Report
      2018 Research-status Report
    • Peer Reviewed
  • [Journal Article] Hyperbolic Gradient Operator and Hyperbolic Back-Propagation Learning Algorithms2018

    • Author(s)
      Nitta Tohru、Kuroe Yasuaki
    • Journal Title

      IEEE Transactions on Neural Networks and Learning Systems

      Volume: 29 Issue: 5 Pages: 1689-1702

    • DOI

      10.1109/tnnls.2017.2677446

    • Related Report
      2017 Research-status Report
    • Peer Reviewed
  • [Journal Article] Resolution of Singularities Introduced by Hierarchical Structure in Deep Neural Networks2017

    • Author(s)
      新田徹
    • Journal Title

      IEEE Trans. Neural Networks and Learning Systems

      Volume: 印刷中 Issue: 10 Pages: 2282-2293

    • DOI

      10.1109/tnnls.2016.2580741

    • Related Report
      2016 Research-status Report
    • Peer Reviewed / Acknowledgement Compliant
  • [Presentation] Sequential Learning on sEMGs in Short- and Long-term Situations via Self-Training Semi-Supervised Support Vector Machine2022

    • Author(s)
      Y. Okawa, S. Kanoga, T. Hoshino and T. Nitta
    • Organizer
      Proceedings of the 44th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC2022), Glasgow, UK, July 11-15, pp.3232-3235
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Fully Augmented Complex-Valued Neural Networks2022

    • Author(s)
      T. Nitta
    • Organizer
      Proceedings of the 2022 International Symposium on Nonlinear Theory and its Applications (NOLTA2022) (Full-Online), Dec. 12-15, pp.248-251
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 高次元ニューラルネットワーク2022

    • Author(s)
      新田徹
    • Organizer
      東京女子大学学会
    • Related Report
      2022 Annual Research Report
    • Invited
  • [Presentation] 双対数を用いたニューラルネットワークとその学習特性2022

    • Author(s)
      大河勇斗、新田徹
    • Organizer
      東北大学 電気通信研究所 共同プロジェクト研究研究会:「高次元・時空間ニューロダイナミクスとそれに基づくシステム構築への展開」
    • Related Report
      2021 Research-status Report
  • [Presentation] Learning Properties of Feedforward Neural Networks Using Dual Numbers2021

    • Author(s)
      Yuto Okawa, Tohru Nitta
    • Organizer
      13th Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)
    • Related Report
      2021 Research-status Report
    • Int'l Joint Research
  • [Presentation] 直交変数可換四元数ニューロンの基本構造2021

    • Author(s)
      新田 徹,Hui Hu GAN
    • Organizer
      東北大学電気通信研究所共同プロジェクト研究研究会:「高次元ニューロダイナミクスとそのニューロハードウェア構築への展開」
    • Related Report
      2020 Research-status Report
  • [Presentation] Fundamental Structure of Orthogonal Variable Commutative Quaternion Neurons2020

    • Author(s)
      Tohru Nitta, Hui Hu GAN
    • Organizer
      Joint 11th International Conference on Soft Computing and Intelligent Systems and 21st International Symposium on Advanced Intelligent Systems, SCIS & ISIS2020
    • Related Report
      2020 Research-status Report
    • Int'l Joint Research
  • [Presentation] 双曲勾配オペレータと階層型双曲ニューラルネットワーク2019

    • Author(s)
      新田 徹
    • Organizer
      福岡大学情報数理セミナー
    • Related Report
      2019 Research-status Report
    • Invited
  • [Presentation] On the Equivalence between Hypercomplex Widely Linear Estimation and its Real Vector Counterpart2018

    • Author(s)
      新田徹、小林正樹、Danilo, P. Mandic
    • Organizer
      東北大学電気通信研究所共同プロジェクト研究研究会「高次元ニューロダイナミクスとそのニューロハードウエア構築への展開」
    • Related Report
      2018 Research-status Report
  • [Presentation] Resolution of Singularities via Deep Complex-Valued Neural Networks2017

    • Author(s)
      新田徹
    • Organizer
      Empowering Novel Geometric Algebra for Graphics & Engineering Workshop
    • Related Report
      2017 Research-status Report
    • Int'l Joint Research
  • [Presentation] 階層型双曲ニューラルネットワークの学習特性2017

    • Author(s)
      新田徹、黒江康明
    • Organizer
      東北大学電気通信研究所共同プロジェクト研究研究会「高次元ニューラルネットワークにおける情報表現の最適化」
    • Related Report
      2017 Research-status Report
  • [Presentation] 深層複素ニューラルネットワークの学習特性2016

    • Author(s)
      新田徹
    • Organizer
      計測自動制御学会 システム・情報部門 学術講演会
    • Place of Presentation
      滋賀県立体育館(滋賀県大津市)
    • Year and Date
      2016-12-06
    • Related Report
      2016 Research-status Report
  • [Presentation] On the Singularity in Deep Neural Networks2016

    • Author(s)
      新田徹
    • Organizer
      The 23rd International Conference on Neural Information Processing
    • Place of Presentation
      京都大学(京都府京都市)
    • Year and Date
      2016-10-16
    • Related Report
      2016 Research-status Report
    • Int'l Joint Research
  • [Presentation] 深層複素ニューラルネットワークの学習特性2016

    • Author(s)
      新田徹
    • Organizer
      計測自動制御学会 第9回コンピュテーショナル・インテリジェンス研究会
    • Place of Presentation
      千葉大学(千葉県千葉市)
    • Year and Date
      2016-07-08
    • Related Report
      2016 Research-status Report
  • [Presentation] 複素ニューラルネットワークによるTwo Spirals Problemの求解2016

    • Author(s)
      新田徹
    • Organizer
      東北大学電気通信研究所共同プロジェクト研究研究会「高次元ニューラルネットワークにおける情報表現の最適化」
    • Place of Presentation
      東北大学電気通信研究所(宮城県仙台市)
    • Related Report
      2016 Research-status Report

URL: 

Published: 2016-04-21   Modified: 2024-01-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi