2001 Fiscal Year Final Research Report Summary
Generalization Capability of Memorization Leaning
| Project/Area Number |
11480072
|
|---|
| Research Category |
Grant-in-Aid for Scientific Research (B)
|
| Allocation Type | Single-year Grants |
|---|
| Section | 一般 |
|---|
| Research Field |
Intelligent informatics
|
|---|
| Research Institution | Tokyo Institute of Technology |
|---|
Principal Investigator |
OGAWA Hidemitsu Tokyo Institute of Technology, Department of Computer Science, Professor, 大学院・情報理工学研究科, 教授 (50016630)
|
|---|
| Co-Investigator(Kenkyū-buntansha) |
SUGIYAMA Masashi Tokyo Institute of Technology, Department of Computer Science, Research Associate, 大学院・情報理工学研究科, 助手 (90334515)
HIRABAYASHI Akira Yamaguchi University, Department of Computer Science and Systems Engineering, Lecturer, 工学部, 講師 (50272688)
KUMAZAWA Itsuo Tokyo Institute of Technology, Department of Computer Science, Associate Professor, 大学院・情報理工学研究科, 助教授 (70186469)
|
|---|
| Project Period (FY) |
1999 – 2001
|
| Keywords | supervised learning / generalization capability / error back-propagation / memorization learning / admissibility / a family of projection learnings / subspace information criterion / SIC |
|---|
| Research Abstract |
The purpose of supervised learning is to be able to answer correctly to queries that are not necessarily included in the training examples, i.e., to acquire a higher level of the generalization capability. However, most of the learning methods such as the error back-propagation are the so-called memorization learning, which is aimed at reducing the error only for training examples. Therefore, there is no theoretical guarantee for optimal generalization.
This gives rise to the following problems: First is to clarify the reason why a higher level of the generalization capability can be acquired by the memorization learning despite the fact that it does not require the generalization capability. The second problem is to clarify the range of applicability that: the memorization learning works effectively. Third is to develop methods for further expanding the range of applicability.
For the first problem, we gave a lucid explanation by introducing the concept of admissibility. For the second
… More
problem, we introduced the concept of a family of projection learnings which allows us to theoretically discuss an infinitely many kinds of learning methods simultaneously. Utilizing the concepts of admissibility and a family of projection learning, we clarified the range of applicability of the memorization learning in the narrow sense with respect to a family of projection learnings. For the third problem, we showed that there are a large number of solutions : We extended the concept of the memorization learning from the rote memorization learning to the error corrected memorization learning, which further enlarges the range of applicability. From the view point of active learning, we gave design methods of training examples that maximally enhance the generalization capability. Furthermore, from the standpoint of model selection, we proposed the subspace information criterion (SIC) , which is a model selection criterion with its effectiveness theoretically guaranteed for a finite number of training examples. Based on SIC, we gave, for example, a design method of the optimal regularization parameter. Less
|
