2017 Fiscal Year Research-status Report
High-Order Deep Learning Models: Theoretical Study and Applications
| Project/Area Number |
17K00326
|
|---|
| Research Institution | Institute of Physical and Chemical Research |
|---|
Principal Investigator |
ZHAO QIBIN 国立研究開発法人理化学研究所, 革新知能統合研究センター, ユニットリーダー (30599618)
|
|---|
| Co-Investigator(Kenkyū-buntansha) |
曹 建庭 埼玉工業大学, 工学部, 教授 (20306989)
|
|---|
| Project Period (FY) |
2017-04-01 – 2020-03-31
|
| Keywords | Tensor decomposition / Deep neural network / PU learning / GAN |
|---|
| Outline of Annual Research Achievements |
We study tensor based deep learning model and algorithms. In the traditional deep learning methods, each layer is considered as a vector and the connection between layers is considered as a matrix. However, the real-world data is usually represented as a high order tensor. To this end, we formulate the deep learning framework by considering each layer as a tensor and the connection between the layers as multilinear operations based on multiple matrices.
We developed a new tensor based generative adverbial network, which use tensors as input and output, the fully connected layer can be modeled by multilinear product on each tensor mode. The experimental results show that our method can alleviate the mode collapse problem of GAN.
We studied the combination of multiple GANs to perform the important positive unlabeled learning problem. The objective is to learn two generators simutaneously by three discriminator. Each generator can capture one class distribution.
We introduced a new type of tensor decomposition model, which is called tensor ring decomposition. We studied the theoretical ground and the mathematical properties of our proposed model. Then, we developed several algorithms to solve this model. Finally, we applied it to represent the fully connected weight parameters, yielding a significant compression for model complexity.
|
| Current Status of Research Progress |
Current Status of Research Progress
2: Research has progressed on the whole more than it was originally planned.
Reason
The project is performing smoothly.
|
| Strategy for Future Research Activity |
In the next step, we will investigate how the newly proposed tensor ring decomposition can be applied to deep learning models.
1. We will investigate the low rank representation ability of tensor ring decomposition by applying it to CNN, LSTM, and RNN. The goal is to reduce the model complexity while keeping the same generalization ability.
2. We will study more efficient tensor networks and the corresponding algorithms to model the unknown variables in the general machine learning method. This will allow us to develop new machine learning method with high computational efficiency and compact model complexity.
|
| Causes of Carryover |
Some conference will be hold in the next fiscal year. We plan to use it for business trip.
|
