2002 Fiscal Year Final Research Report Summary
Mathematical foundation of Singular Learning Machines based on Algebraic Geometry and Analysis
| Project/Area Number |
12680370
|
|---|
| Research Category |
Grant-in-Aid for Scientific Research (C)
|
| Allocation Type | Single-year Grants |
|---|
| Section | 一般 |
|---|
| Research Field |
Intelligent informatics
|
|---|
| Research Institution | Tokyo Institute of Technology |
|---|
Principal Investigator |
WATANABE Sumio Precision and Intelligence Lab., Tokyo Institute of Technology, Professor, 精密工学研究所, 教授 (80273118)
|
|---|
| Project Period (FY) |
2000 – 2002
|
| Keywords | Singularities / algebraic geometry / algebraic analysis / Learning Theory / generalization error / stochastic complexity |
|---|
| Research Abstract |
A lot of learning machines are nonidentifiable, for example, artificial neural networks, normal mixtures, Bayesian networks, reduced rank approximations. In such learning machines, the mapping from the parameter to the probability distribution is not one-to-one, resulting that the Fisher information matrices are singular. It should be emphasized that there is no learning theory which can be applied to such singular learning machines. In this research, we developed a new learning theory which enables us to clarify the generalization errors of such learning machines. The main result is as follows. The generalization error that is defined as the average Kullback information from the true distribution to the estimated distribution can be asymptotically expanded as "a log n - (b-1)loglog n + c", where a, b, and c are constants. The important constants a and b are determined as the largest pole and its order of the zeta function of the Kullback in formation and the prior. They can be calculated by the blowing-up technology, which is well known in resolution of singularities in algebraic geometry.
|
Research Products
(17 results)
