1998 Fiscal Year Final Research Report Summary
Neural network learning with regulatizers and generalization ability
| Project/Area Number |
09680371
|
|---|
| Research Category |
Grant-in-Aid for Scientific Research (C)
|
| Allocation Type | Single-year Grants |
|---|
| Section | 一般 |
|---|
| Research Field |
Intelligent informatics
|
|---|
| Research Institution | Kyushu Institute of Technology |
|---|
Principal Investigator |
ISHIKAWA Masumi Kyushu Institute of Technology Dept. of Control Engineering & Science, Professor, 情報工学部, 教授 (60222973)
|
|---|
| Co-Investigator(Kenkyū-buntansha) |
ZHANG Hong Kyushu Institute of Technology Dept. of Control Engineering & Science, Assistant Professor, 情報工学部, 助手 (30235709)
|
|---|
| Project Period (FY) |
1997 – 1998
|
| Keywords | generalization / regularization / Gaussian regularizer / Laplace regularizer / linear regression |
|---|
| Research Abstract |
The relation between neural network learning with regularizes and generalization ability is clarified both theoretically and empirically. In the present theoretical study, a Laplacian regularize, a Gaussian regularize and their combinations are considered.
In the first stage, various empirical procedures in a structural learning with forgetting proposed by the authors are theoretically clarified. For example, a structural learning with selective forgetting has something to do with the line process in vision under the assumption that a scene is almost smooth except a small number of discontinuous points.
In the second stage, a estimation of mean value using regularization is theoretically studied. It is the simplest case of multiple regression models. It is demonstrated that the proposed regularization method is effectively applied to real data.
In the third stage, excessive simplification in a formulation of generalization errors in multiple regression models is rectified. So for it has been assumed that input variables are mutually independent, and true model parameters and a noise variance are known a priori. We modified formulations so as to allow correlations between input variables. We propose a novel procedure for theoretically evaluating regularizes based on data. Firstly, we estimate model parameters and a noise variance from data. Secondly, assuming that these estimates are true, we calculate the optimal regularization parameters and model parameters by the previously proposed method. It provides, hopefully, their better estimates. Thirdly, assuming that the resulting estimates are true, we again calculate the optimal regularization parameters and model parameters. This procedure can be repeated iteratively. This iterative estimation is a key idea of the present study. Applications of the proposed method to real data demonstrates that better estimates with smaller generalization errors are obtained successfully.
|
