| 研究実績の概要 |
We study tensor based deep learning model and algorithms. In the traditional deep learning methods, each layer is considered as a vector and the connection between layers is considered as a matrix. However, the real-world data is usually represented as a high order tensor. To this end, we formulate the deep learning framework by considering each layer as a tensor and the connection between the layers as multilinear operations based on multiple matrices.
We developed a new tensor based generative adverbial network, which use tensors as input and output, the fully connected layer can be modeled by multilinear product on each tensor mode. The experimental results show that our method can alleviate the mode collapse problem of GAN.
We studied the combination of multiple GANs to perform the important positive unlabeled learning problem. The objective is to learn two generators simutaneously by three discriminator. Each generator can capture one class distribution.
We introduced a new type of tensor decomposition model, which is called tensor ring decomposition. We studied the theoretical ground and the mathematical properties of our proposed model. Then, we developed several algorithms to solve this model. Finally, we applied it to represent the fully connected weight parameters, yielding a significant compression for model complexity.
|
| 今後の研究の推進方策 |
In the next step, we will investigate how the newly proposed tensor ring decomposition can be applied to deep learning models.
1. We will investigate the low rank representation ability of tensor ring decomposition by applying it to CNN, LSTM, and RNN. The goal is to reduce the model complexity while keeping the same generalization ability.
2. We will study more efficient tensor networks and the corresponding algorithms to model the unknown variables in the general machine learning method. This will allow us to develop new machine learning method with high computational efficiency and compact model complexity.
|
