2022 Fiscal Year Annual Research Report
Tensor Network Representation for Machine Learning: Theoretical Study and Algorithms Development
| Project/Area Number |
20H04249
|
|---|
| Research Institution | Institute of Physical and Chemical Research |
|---|
Principal Investigator |
ZHAO QIBIN 国立研究開発法人理化学研究所, 革新知能統合研究センター, チームリーダー (30599618)
|
|---|
| Co-Investigator(Kenkyū-buntansha) |
曹 建庭 埼玉工業大学, 工学部, 教授 (20306989)
横田 達也 名古屋工業大学, 工学(系)研究科(研究院), 准教授 (80733964)
|
|---|
| Project Period (FY) |
2020-04-01 – 2024-03-31
|
| Keywords | tensor networks |
|---|
| Outline of Annual Research Achievements |
In this fiscal year, we further study tensor completion algorithms and developed several novel approach such as Bayesian tensor completion algorithm and fully connected tensor network algorithm. In addition, we also study an open problem of tensor network structure search. Finally, several applications to multi-task learning and hyper spectral image processing have been conducted by applying tensor methods.
The transformer is widely studied and demonstrated to be a powerful architecture. However, it is difficult to be adopted into multi-modal learning, we have applied tensor representation for attention mechanism and developed multi-way multi-modal transformer, which is more efficient and powerful for multimodal learning. One practical variant of tensor network structure search, dubbed TN permutation search (TN-PS), in which we search for good mappings from tensor modes onto TN vertices (core tensors) for compact TN representations. We conduct a theoretical investigation of TN-PS and propose a practically-efficient algorithm to resolve the problem. We propose a unified paradigm combining the spatial and spectral properties for HSI restoration. The proposed paradigm enjoys performance superiority from the non-local spatial denoising and light computation complexity from the low-rank orthogonal basis exploration.
|
| Current Status of Research Progress |
Current Status of Research Progress
2: Research has progressed on the whole more than it was originally planned.
Reason
Our research goes smoothly and well.
|
| Strategy for Future Research Activity |
In the next year, we will further study how tensor networks structure search problem can be solved by efficient algorithm and also conduct research on tensor methods based machine learning technology.
|
