• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Study on cooperation mechanism and it's dynamic behavior of many learning machines

Research Project

Project/Area Number 16500146
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeSingle-year Grants
Section一般
Research Field Sensitivity informatics/Soft computing
Research InstitutionTokyo metropolitan college of industrial technology (2006)
Tokyo Metropolitan College of Technology (2004-2005)

Principal Investigator

HARA Kazuyuki  Tokyo metropolitan college of industrial technology, Monozukuri engineering department, Professor, ものづくり工学科, 教授 (30311004)

Co-Investigator(Kenkyū-buntansha) MIYOSHI Seiji  Kobe city college of technology, Department of electronic engineering, Assistant professor, 電子工学科, 助教授 (10270307)
Project Period (FY) 2004 – 2006
Project Status Completed (Fiscal Year 2006)
Budget Amount *help
¥3,100,000 (Direct Cost: ¥3,100,000)
Fiscal Year 2006: ¥600,000 (Direct Cost: ¥600,000)
Fiscal Year 2005: ¥1,000,000 (Direct Cost: ¥1,000,000)
Fiscal Year 2004: ¥1,500,000 (Direct Cost: ¥1,500,000)
Keywordsensemble learning / cooperate mechanism / dynamic process / on-line learning / simple perceptron / statistical mechanics / アンサンブル学習 / 相互学習 / 線形パーセプトロン / 非線型パーセプトロン / 情報統計力学 / 多数決
Research Abstract

Ensemble learning algorithms, such as bagging and Ada-boost, try to improve upon the performance of a weak learning machine by using many weak learning machines ; such learning algorithms have recently received considerable attention. We have analyzed the dynamics of the generalization error of ensemble learning by using statistical mechanics methods within the framework of on-line learning. Within this framework, the overlap (or direction cosine) between the teacher and the initial student weight vectors plays important roles in ensemble learning. When overlaps between the teacher and the students are homogeneous, a simple average of the student outputs can be used as an integration method for ensemble learning (bagging). From our analysis, we found that the generalization error was equal to half that of a single linear perceptron when the number of linear perceptrons K became infinite for the no noise case. In addition, we found that the generalization error converged with that of th … More e infinite case with 0(1/K) when the number of linear perceptrons was finite for both the no noise case and the noisy case. In an inhomogeneous case, the generalization error can be improved by introducing weights to average the outputs of the learning machines (i.e., to use a weighted average rather than a simple average), and the weights should be adapted to minimize the generalization error (i.e., parallel boosting). In ensemble learning, there is no interaction between the students. In mutual learning, learning is performed between two students who learn from a teacher in advance. Therefore, the knowledge each student has obtained from the teacher is exchanged, which may improve the performance of the students. Moreover, the interaction may mimic the integration mechanism of ensemble learning. We showed that the mutual learning asymptotically converged into bagging. Moreover, a student with a larger initial overlap for mutual learning transiently passes through a state of parallel boosting during the learning in the limit of step size goes to zero. Less

Report

(4 results)
  • 2006 Annual Research Report   Final Research Report Summary
  • 2005 Annual Research Report
  • 2004 Annual Research Report
  • Research Products

    (24 results)

All 2007 2005 2004

All Journal Article (24 results)

  • [Journal Article] Statistical mechanics of mutual learning with a latent teacher2007

    • Author(s)
      Hara, K., Okada, M.
    • Journal Title

      Journal of physical society of Japan 76・1

      Pages: 14001-14001

    • NAID

      110006151543

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Statistical mechanics of mutual learning with a latent teacher2007

    • Author(s)
      Hara, K., Okada, M.
    • Journal Title

      Journal of physical society of Japan vol.76, no.1

      Pages: 14001-14001

    • NAID

      110006151543

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Statistical mechanics of mutual learning with a latent teacher2007

    • Author(s)
      Kazuyuki Hara
    • Journal Title

      Journal of the physical society of Japan 76.1

      Pages: 14001-14001

    • NAID

      110006151543

    • Related Report
      2006 Annual Research Report
  • [Journal Article] Analysis of ensemble learning using simple perceptrons based on on-line learning theory2005

    • Author(s)
      Miyoshi, S., Hara, K, Okada, M.
    • Journal Title

      Virtual journal of biological physics research 9・6

    • NAID

      110003232421

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Analysis of ensemble learning using simple perceptrons based on on-line learning theory2005

    • Author(s)
      Miyoshi, S., Hara, K, Okada, M.
    • Journal Title

      Progress of theoretical physics, supplement, 157

      Pages: 270-274

    • NAID

      110003232421

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Ensemble learning of linear perception : On-line learning theory2005

    • Author(s)
      Hara, K, Okada, M.
    • Journal Title

      Journal of physical society of Japan 74・11

      Pages: 2966-2972

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Analysis of ensemble learning using simple perceptrons based on on-line learning theory2005

    • Author(s)
      Miyoshi, S., Hara, K, Okada, M.
    • Journal Title

      System and Computer in Japan 36・12

      Pages: 63-74

    • NAID

      110003232421

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Analysis of ensemble learning using simple perceptrons based on on-line learning theory2005

    • Author(s)
      Miyoshi, S., Hara, K, Okada, M.
    • Journal Title

      Virtual journal of biological physics research vol.9,no.6

    • NAID

      110003232421

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Analysis of ensemble learning using simple perceptrons based on on-line learning theory2005

    • Author(s)
      Miyoshi, S., Hara, K, Okada, M.
    • Journal Title

      Progress of theoretical physics no.157(supplement)

      Pages: 270-274

    • NAID

      110003232421

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Ensemble learning of linear perceptron : On-line learning theory2005

    • Author(s)
      Hara, K, Okada, M.
    • Journal Title

      Journal of physical society of Japan vol.74, no.11

      Pages: 2966-2972

    • NAID

      110002325532

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Analysis of ensemble learning using simple perceptrons based on on-line learning theory2005

    • Author(s)
      Miyoshi, S., Hara, K, Okada, M.
    • Journal Title

      System and Computer in Japan vol.36, no.12

      Pages: 63-74

    • NAID

      110003232421

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Ensemble learning of linear perceptron : On-line learning theory2005

    • Author(s)
      K.Hara, M.Okada
    • Journal Title

      Journal of the Physical Society of Japan 74・11

      Pages: 7-7

    • NAID

      110002325532

    • Related Report
      2005 Annual Research Report
  • [Journal Article] Analysis of ensemble learning using simple perceptrons based on online learning theory,2005

    • Author(s)
      S.Miyoshi, K.Hara, M.Okada
    • Journal Title

      Virtual Journal of Biological Physics Research 9・6

      Pages: 11-11

    • NAID

      110001276022

    • Related Report
      2005 Annual Research Report
  • [Journal Article] Analysis of ensemble learning using simple perceptrons based on online learning theory2005

    • Author(s)
      S.Miyoshi, K.Hara, M.Okada
    • Journal Title

      Systems and Computers in Japan(John Wiley & Sons) 36・12

      Pages: 11-11

    • NAID

      110001276022

    • Related Report
      2005 Annual Research Report
  • [Journal Article] Analysis of ensemble learning using simple perceptrons based on online learning theory2005

    • Author(s)
      S.Miyoshi, K.Hara, M.Okada
    • Journal Title

      Physical Review E 71

      Pages: 11-11

    • NAID

      110001276022

    • Related Report
      2004 Annual Research Report
  • [Journal Article] オンライン学習理論に基づく単純パーセプトロンのアンサンブル学習の解析2004

    • Author(s)
      三好誠司, 原一之, 岡田真人
    • Journal Title

      電子情報通信学会論文誌 J87・7

      Pages: 1391-1401

    • NAID

      110003232421

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] On-line learning of a simple perceptron learning with margin2004

    • Author(s)
      Hara, K., Okada, M.
    • Journal Title

      System and Computer in Japan 35・7

      Pages: 98-105

    • NAID

      110004066696

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Analysis of ensemble learning using simple perceptions based on on-line learning theory2004

    • Author(s)
      Miyoshi, S., Hara, K, Okada, M.
    • Journal Title

      Physical Review E

      Pages: 36116-36116

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] 線形ウィークラーナーによるアンサンブル学習の汎化誤差の解析2004

    • Author(s)
      原一之, 岡田真人
    • Journal Title

      システム制御情報学会論文誌 17・12

      Pages: 28-36

    • NAID

      10014152118

    • Description
      「研究成果報告書概要(和文)」より
    • Related Report
      2006 Final Research Report Summary 2004 Annual Research Report
  • [Journal Article] Analysis of ensemble learning using simple perceptrons based on on-line learning theory2004

    • Author(s)
      Miyoshi, S., Hara, K., Okada, M.
    • Journal Title

      Transactions on Information and systems, The Institute of Electronics, Information and Communication Engieers vol.J87, no.7

      Pages: 1391-1401

    • NAID

      110003232421

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] On-line learning of a simple perceptron learning with margin2004

    • Author(s)
      Hara, K., Okada, M.
    • Journal Title

      System and Computer in Japan vol.35, no.7

      Pages: 98-105

    • NAID

      110004066696

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Analysis of ensemble learning using simple perceptrons based on on-line learning theory2004

    • Author(s)
      Miyoshi, S., Hara, K, Okada, M.
    • Journal Title

      Physical Review E

      Pages: 36116-36116

    • NAID

      110003232421

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] Analysis of generalization error of ensemble learning through linear weak learner2004

    • Author(s)
      Hara, K., Okada, M.
    • Journal Title

      Transactions of the institute of systems, control and information engineers vol.17, no.12

      Pages: 28-36

    • NAID

      10014152118

    • Description
      「研究成果報告書概要(欧文)」より
    • Related Report
      2006 Final Research Report Summary
  • [Journal Article] オンライン学習理論に基づく単純パーセプトロンのアンサンブル学習の解析2004

    • Author(s)
      三好誠司, 原一之, 岡田真人
    • Journal Title

      電子情報通信学会論文誌 J87-D-II・7

      Pages: 11-11

    • NAID

      110003232421

    • Related Report
      2004 Annual Research Report

URL: 

Published: 2004-04-01   Modified: 2016-04-21  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi