2018 Fiscal Year Research-status Report
High-Order Deep Learning Models: Theoretical Study and Applications
| Project/Area Number |
17K00326
|
|---|
| Research Institution | Institute of Physical and Chemical Research |
|---|
Principal Investigator |
ZHAO QIBIN 国立研究開発法人理化学研究所, 革新知能統合研究センター, ユニットリーダー (30599618)
|
|---|
| Co-Investigator(Kenkyū-buntansha) |
曹 建庭 埼玉工業大学, 工学部, 教授 (20306989)
|
|---|
| Project Period (FY) |
2017-04-01 – 2020-03-31
|
| Keywords | Tensor ring / Deep multi-task learning |
|---|
| Outline of Annual Research Achievements |
We studied multi-task deep learning models by using high-order tensor network technology. The existing multi-task deep learning models mostly based on sharing the lower layers and having individual upper layers for each task. The problem is that such framework cannot handle the case when different tasks have different input dimensions and when different tasks have heterogenous network architecture. To solve this challenging problem, we proposed a new framework, which allows us to use heterogenous network architecture for individual tasks, by leveraging tensor ring representation of weight parameters of each layer and some of latent core tensors sharing between tasks. The experiments demonstrated that our method is more flexible with high performance.
We studied tensor ring decomposition for data completion and developed several different methods to reconstruct the original tensor. The results show that tensor ring model is more powerful than traditional tensor decomposition models. We also give the theoretical support for our proposed methods.
We propose a new type of tensor decomposition to find the latent low-rank tensors under reshuffling operations. This is quite useful for image steganography. And the theoretical analysis shows the uniqueness condition of our new algorithm.
|
| Current Status of Research Progress |
Current Status of Research Progress
2: Research has progressed on the whole more than it was originally planned.
Reason
The project is performing smoothly.
|
| Strategy for Future Research Activity |
In this fiscal year, we plan to conduct following two research works.
1. The current CNN for 3D data like video clip needs computation cost that is exponentially with dimension of data, which is a challenging problem if the data is higher dimensions much larger than 3. We attempt to solve this problem by using tensor network representation to improve the computation cost to be linear with the dimension of data.
2. We will also study how to learning an optimal tensor network structure from given data. This is an open problem in tensor research field.
|
| Causes of Carryover |
The conference paper accepted in 2018 will be held in 2019.
|
