2015 Fiscal Year Annual Research Report
Onsite Transfer Learning （現地の転移学習）
| Project/Area Number |
15H06823
|
|---|
| Research Institution | The Institute of Statistical Mathematics |
|---|
Principal Investigator |
柳 松 統計数理研究所, その他部局等, 研究員 (80760579)
|
|---|
| Project Period (FY) |
2015-08-28 – 2017-03-31
|
| Keywords | Posterior Ratio / Supervised Learning / Transfer Learning |
|---|
| Outline of Annual Research Achievements |
In this year, we have made some major progress toward the final goal. One of the key components of this research, the estimator of the "posterior ratio" has been successfully developed and tested. This ratio is used in transferring a "learned pattern" from one task to another task. Preliminary experiments have shown successful performance. The methodology and some theoretical investigation has been published in the proceedings of SIAM International Conference Data Mining, 2016.
The proposed estimator is constructed based on the methodology of the density ratio estimation, where the "ratio" of two probability density functions (PDFs) are directly learned from two sets of data without learning two PDFs. This method avoids the learning of an intermediate step and directly learns the quantity that is desired.
In the proposed research, we are more interested in the "conditional density ratio" where the ratio of two *conditional probability densities* are learned. This is similar to the traditional density ratio estimator but has some major differences: not only we have to learned the ratio function, but also we have to take the smoothness over the domain of the conditioning random variable into account. As we have shown in the theoretical analysis, such "smoothness" plays an important role in the success of such an estimator.
|
| Current Status of Research Progress |
Current Status of Research Progress
2: Research has progressed on the whole more than it was originally planned.
Reason
The proposed research is making progress as planned. Not only we have already developed a posterior ratio estimator but also some preliminary experiments/theoretical analysis has been conducted and shown promising results. Most of the plans in the first year have been successfully finished.
As we work on this project, some new questions that we have not expected also emerged: e.g., what applications are the most suitable to this framework? From the experiences we have in the first year, the "onsite" setting may not be suitable to all applications, since "similar tasks" may have very different representations in some applications. and learning their "difference" is not necessarily simple. Thus an interesting question is, can we formally define a set of problems that can be "onsite-transferable"?
Due to this unsolved problem, we have postponed the original plan of collecting datasets, since a good answer to this question may give us hints on the dataset selection procedure.
|
| Strategy for Future Research Activity |
In the next year, the proposed research will firstly conduct as planned: we will examine the theoretical property of the proposed algorithm. We would hope to establish the statistical consistency, convergence rate of the proposed method, under the case where sample size from two datasets D_1 and D_2 are extremely unbalanced. We believe such analysis will be useful in giving us guidance and insights on improving the performance of algorithms in practice.
As we have also explained in the progress report, we will keep investigating a curious question that "what problems are onsite-transferable" and "what are not". The answer to this question will help us select proper datasets to test the performance of the proposed method.
We plan to finish this project with a real-world proven algorithm and a sound theory in the next year.
|
Research Products
(8 results)
