2020 Fiscal Year Research-status Report
Structured Tensor Approximation under Kronecker Graph and Its Application on Hydrological Data
| Project/Area Number |
20K19875
|
|---|
| Research Institution | Institute of Physical and Chemical Research |
|---|
Principal Investigator |
李 超 国立研究開発法人理化学研究所, 革新知能統合研究センター, 特別研究員 (10869837)
|
|---|
| Project Period (FY) |
2020-04-01 – 2022-03-31
|
| Keywords | tensor network / time series forecasting |
|---|
| Outline of Annual Research Achievements |
In the past year, we had the progress on the project from the follow aspects. We first focused on the structure search of tensor network decomposition, a challenging combination optimization task appearing in various learning methods. Second, we study the memory mechanism of tensor-power recurrent models, a family of nonlinear dynamics based on tensors. Last, we attempted a practical application task, where we addressed it by novel tensor-based algorithms. The progress we have not only reveals deeper understanding on the structured tensor approximation issue from the theoretical side, yet achieves superior performance on real-world tasks from the practical side.
Theoretically, we solve the topology-search issue for tensor network decomposition by evolutionary algorithm, where we encode the topological structures into binary strings, and develop a simple genetic meta-algorithm to search the optimal topology. In addition, we study the memory mechanism of tensor-power recurrent models, where we prove that a large model degree is an essential condition to achieve the long memory effect, yet it would lead to unstable dynamical behaviors.
Empirically, we propose a tensor-based method termed Time Product Fusion Network (TPFN) for the task of multimodal sentiment analysis. We construct the fused features by the tensor product along adjacent time-steps, such that richer modal and temporal interactions are utilized. Results on CMU-MOSI and CMU-MOSEI datasets show that TPFN can compete with state-of-the art approaches in cases of both random and structured missing values.
|
| Current Status of Research Progress |
Current Status of Research Progress
2: Research has progressed on the whole more than it was originally planned.
Reason
We have achieved part of the goals in the past year, especially on the theoretical study of the structure search issue and properties of the dynamical behaviour of tensor methods. These results can be used to partially answer the theoretical questions proposed in the project , i.e., the formulation and properties of the structured-tensor-based learning models. The progress is closed to that we expected in the project proposal.
|
| Strategy for Future Research Activity |
In the next year, we plan to pay more effort on the study of the connection between graph theory and tensor networks. Based on the previous works, we plan to continue the topic on the structure search issue, yet to focus on an unsolved question that how the symmetry of the topological structures impacts difficulty of the structure search issue. The topic will be discussed by the tools from group theory and discrete mathematic. The potential results would be useful to develop more efficient tensor methods for various learning tasks. Furthermore, we also have the plan to improve algorithms proposed in the past year and to evaluate their performance on the hydrological data.
|
| Causes of Carryover |
Due to the COVID-19 issue, the total cost was less than expected in the proposal, especially on the travel expense and personnel expenditure. In the next year, we have the plan to increase the virtual academic networking including on-line talks and seminars. In addition, several national travels are also considered this year yet it depends on the COVID condition. The article and personal cost will be also implemented in this year as planed in the proposal.
|
| Remarks |
The urls for the open-access papers are listed.
|
Research Products
(10 results)
