• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Generalized competitive learning for improving and interpreting neural networks

Research Project

Project/Area Number 16K00339
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeMulti-year Fund
Section一般
Research Field Soft computing
Research InstitutionTokai University

Principal Investigator

Kamimura Ryotaro  東海大学, 情報教育センター, 非常勤講師 (80176643)

Project Period (FY) 2016-04-01 – 2019-03-31
Project Status Completed (Fiscal Year 2018)
Budget Amount *help
¥4,680,000 (Direct Cost: ¥3,600,000、Indirect Cost: ¥1,080,000)
Fiscal Year 2018: ¥1,430,000 (Direct Cost: ¥1,100,000、Indirect Cost: ¥330,000)
Fiscal Year 2017: ¥1,430,000 (Direct Cost: ¥1,100,000、Indirect Cost: ¥330,000)
Fiscal Year 2016: ¥1,820,000 (Direct Cost: ¥1,400,000、Indirect Cost: ¥420,000)
Keywordsニューラルネットワーク / 情報理論 / 汎化能力 / 解釈 / 情報圧縮 / 解釈能力 / 相互情報量 / 競合学習 / 内部表現 / 情報劣化 / ディープラーニング / 多層ニューラルネットワーク / 集合的解釈 / デープラーニング / 自己符号化器 / 中間層 / 情報量最大化 / ソフトコンピューティング / 人工知能
Outline of Final Research Achievements

The present study tried to extend the competitive learning methods to more generalized methods. The generalized competitive learning can be used to maximize mutual information between neurons and input patterns, disentangling complex patterns into a set of simple features. Thus, maximized mutual information can be compressed to be represented by the simplest neural networks without hidden layers. Then, it becomes easier to interpret the inference mechanism of complex neural networks by using the simplest networks. Applied to the real business data sets, it was found that the information maximization and compression could be used to create simpler and easily interpretable representations on the relations between inputs and outputs.

Academic Significance and Societal Importance of the Research Achievements

意義は,情報量最大化法の単純化,情報の圧縮,さらに解釈可能なニューラルネットワークの開発の3点に要約できる.まず,これまで最大の問題であったニューラルネットワークの持つ情報量の制御を非常に簡単な競合学習で行うことができることがわかった.また,情報量を圧縮することも容易になり,圧縮された情報量を読み取ることが可能となり,解釈へ応用できる可能性が示された.推論過程の解釈が可能となり,より深く社会に受け入れられる方法へ発展する可能性を示したと考える.

Report

(4 results)
  • 2018 Annual Research Report   Final Research Report ( PDF )
  • 2017 Research-status Report
  • 2016 Research-status Report
  • Research Products

    (23 results)

All 2019 2018 2017 2016

All Journal Article (9 results) (of which Peer Reviewed: 9 results,  Open Access: 3 results) Presentation (14 results) (of which Int'l Joint Research: 14 results)

  • [Journal Article] Sparse semi-autoencoders to solve the vanishing information problem in multi-layered neural networks2019

    • Author(s)
      Kamimura Ryotaro and Takeuchi Haruhiko
    • Journal Title

      Applied Intelligence

      Volume: - Issue: 7 Pages: 1-24

    • DOI

      10.1007/s10489-018-1393-x

    • Related Report
      2018 Annual Research Report
    • Peer Reviewed
  • [Journal Article] SOM-based information maximization to improve and interpret multi-layered neural networks: From information reduction to information augmentation approach to create new information2019

    • Author(s)
      Ryotaro Kamimura
    • Journal Title

      Expert systems with applications

      Volume: 125 Pages: 397-411

    • DOI

      10.1016/j.eswa.2019.01.056

    • Related Report
      2018 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Neural self-compressor: Collective interpretation by compressing multi-layered neural networks into non-layered networks2019

    • Author(s)
      Ryotaro Kamimura
    • Journal Title

      Neurocomputing

      Volume: 323 Pages: 12-36

    • DOI

      10.1016/j.neucom.2018.09.036

    • Related Report
      2018 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Supposed Maximum Mutual Information for Improving Generalization and Interpretation of Multi-Layered Neural Networks2019

    • Author(s)
      Ryotaro Kamimura
    • Journal Title

      Journal of Artificial Intelligence and Soft Computing Research

      Volume: 9.2 Issue: 2 Pages: 123-147

    • DOI

      10.2478/jaiscr-2018-0029

    • Related Report
      2018 Annual Research Report
    • Peer Reviewed
  • [Journal Article] Collective mutual information maximization to unify passive and positive approaches for improving interpretation and generalization2017

    • Author(s)
      Ryotaro Kamimura
    • Journal Title

      Neural Networks

      Volume: 90 Pages: 56-71

    • DOI

      10.1016/j.neunet.2017.03.001

    • Related Report
      2017 Research-status Report
    • Peer Reviewed
  • [Journal Article] Identifying Important Tweets by Considering the Potentiality of Neurons2017

    • Author(s)
      (6)Kitajima, R., Kamimura, R., Uchida, O., & Toriumi, F
    • Journal Title

      IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences

      Volume: 99 Pages: 1555-1559

    • NAID

      130005253730

    • Related Report
      2016 Research-status Report
    • Peer Reviewed
  • [Journal Article] Potential information maximization: Potentiality-driven information maximization and its application to Tweets classification and interpretation2016

    • Author(s)
      (1)Kitajima, R., Kamimura, R., Uchida, O., & Toriumi, F
    • Journal Title

      International Journal of Computer Information Systems and Industrial Management Applications

      Volume: 8 Pages: 42-51

    • Related Report
      2016 Research-status Report
    • Peer Reviewed / Open Access
  • [Journal Article] Simplified Information Maximization for Improving Generalization Performance in Multilayered Neural Networks.2016

    • Author(s)
      Ryotaro Kamimura
    • Journal Title

      Mathematical Problems in Engineering

      Volume: 2016 Pages: 1-17

    • DOI

      10.1155/2016/3015087

    • Related Report
      2016 Research-status Report
    • Peer Reviewed / Open Access
  • [Journal Article] Information-Theoretic Active SOM for Improving Generalization Performance2016

    • Author(s)
      Ryotaro Kamimura
    • Journal Title

      International Journal of Advanced Research in Artificial Intelligence

      Volume: 5 Issue: 8 Pages: 21-30

    • DOI

      10.14569/ijarai.2016.050804

    • Related Report
      2016 Research-status Report
    • Peer Reviewed / Open Access
  • [Presentation] Excessive, Selective and Collective Information Processing to Improve and Interpret Multi-layered Neural Networks2018

    • Author(s)
      Ryotaro Kamimura and Haruhiko Takeuchi
    • Organizer
      Proceedings of SAI intelligent systems conference
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Information-Theoretic Self-compression of Multi-layered Neural Networks2018

    • Author(s)
      Ryotaro Kamimura
    • Organizer
      International Conference on Theory and Practice of Natural Computing
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Autoeoncoders and Information Augmentation for Improved Generalization and Interpretation in Multi-layered Neural Networks2018

    • Author(s)
      Ryotaro Kamimura
    • Organizer
      International Symposium on Computational and Business Intelligence (ISCBI)
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Information Augmentation, Reduction and Compression for Interpreting Multi-layered Neural Networks2018

    • Author(s)
      Ryotaro Kamimura
    • Organizer
      Future of Information and Communication Conference
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Local Selective Learning for Interpreting Multi-Layered Neural Networks2018

    • Author(s)
      Ryotaro Kamimura, Ryozo Kitajima and Hiroyuki Sakai
    • Organizer
      Joint 10th International Conference on Soft Computing and Intelligent Systems and 19th International Symposium on Advanced Intelligent Systems
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Correlation-Constrained Mutual Information Maximization for Interpretable Multi-Layered Neural Networks2018

    • Author(s)
      Ryotaro Kamimura and Haruhiko Takeuchi
    • Organizer
      Joint 10th International Conference on Soft Computing and Intelligent Systems and 19th International Symposium on Advanced Intelligent Systems
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Autoeconder-Based Excessive Information Generation for Improving and Interpreting Multi-Layered Neural Network2018

    • Author(s)
      Ryotaro Kamimura and Haruhiko Takeuchi
    • Organizer
      6th International Conference on Smart Computing and Artificial Intelligence (SCAI 2018)
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Potential layer-wise supervised learning for training multi-layered neural networks2017

    • Author(s)
      Ryotaro Kamimura
    • Organizer
      International joint conference on neural networks
    • Related Report
      2017 Research-status Report
    • Int'l Joint Research
  • [Presentation] Selective and cooperative potentiality maximization for improving interpretation and generalization2017

    • Author(s)
      Ryotaro Kamimura
    • Organizer
      International joint conference on neural networks
    • Related Report
      2017 Research-status Report
    • Int'l Joint Research
  • [Presentation] Mutual information maximization for improving and interpreting multi-layered neural networks2017

    • Author(s)
      Ryotaro Kamimura
    • Organizer
      IEEE Symposium series on computational intelligence
    • Related Report
      2017 Research-status Report
    • Int'l Joint Research
  • [Presentation] Supervised semi-autoencoder learning for multi-layered neural networks2017

    • Author(s)
      Ryotaro Kamimura, Haruhiko Takeuchi
    • Organizer
      IFSA-SCIS
    • Related Report
      2017 Research-status Report
    • Int'l Joint Research
  • [Presentation] Simple and Stable Internal Representation by Potential Mutual Information Maximization. In International Conference on Engineering Applications of Neural Networks2016

    • Author(s)
      Ryotaro Kamimura
    • Organizer
      European conference on engineering applications of neural networks
    • Place of Presentation
      Aberdeen
    • Year and Date
      2016-09-02
    • Related Report
      2016 Research-status Report
    • Int'l Joint Research
  • [Presentation] Repeated potentiality assimilation: simplifying learning procedures by positive, independent and indirect operation for improving generalization and interpretation2016

    • Author(s)
      Ryotaro Kamimura
    • Organizer
      International Joint Conference on Neural Networks
    • Place of Presentation
      Vancouber
    • Related Report
      2016 Research-status Report
    • Int'l Joint Research
  • [Presentation] Solving the Vanishing Information Problem with Repeated Potential Mutual Information Maximization. In International Conference on Neural Information Processing2016

    • Author(s)
      Ryotaro Kamimura
    • Organizer
      International Conference on Neural Information Processing
    • Place of Presentation
      Kyoto
    • Related Report
      2016 Research-status Report
    • Int'l Joint Research

URL: 

Published: 2016-04-21   Modified: 2020-03-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi