• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Tensor Network Representation for Machine Learning: Theoretical Study and Algorithms Development

Research Project

Project/Area Number 20H04249
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeSingle-year Grants
Section一般
Review Section Basic Section 61030:Intelligent informatics-related
Research InstitutionInstitute of Physical and Chemical Research

Principal Investigator

ZHAO Qibin  国立研究開発法人理化学研究所, 革新知能統合研究センター, チームリーダー (30599618)

Co-Investigator(Kenkyū-buntansha) 曹 建庭  埼玉工業大学, 工学部, 教授 (20306989)
横田 達也  名古屋工業大学, 工学(系)研究科(研究院), 准教授 (80733964)
Project Period (FY) 2020-04-01 – 2024-03-31
Project Status Completed (Fiscal Year 2023)
Budget Amount *help
¥17,550,000 (Direct Cost: ¥13,500,000、Indirect Cost: ¥4,050,000)
Fiscal Year 2023: ¥4,290,000 (Direct Cost: ¥3,300,000、Indirect Cost: ¥990,000)
Fiscal Year 2022: ¥4,290,000 (Direct Cost: ¥3,300,000、Indirect Cost: ¥990,000)
Fiscal Year 2021: ¥4,290,000 (Direct Cost: ¥3,300,000、Indirect Cost: ¥990,000)
Fiscal Year 2020: ¥4,680,000 (Direct Cost: ¥3,600,000、Indirect Cost: ¥1,080,000)
Keywordstensor network / machine learning / adversarial robustness / tensor decomposition / Tensor networks / tensor networks / Tensor Networks / Machine Learning / Robustness
Outline of Research at the Start

Tensor networks (TNs) have recently gained increasing attentions in machine learning, data mining and computer vision fields due to its effectiveness in efficient computation and model compression in deep learning. However, there are many open problems that are still unexplored, which limits its impact in machine learning. Therefore, our research aims to investigate the fundamental theory and develop scalable and efficient learning algorithms for TN. Moreover, we will further explore what challenging problems in machine learning can be solved by TN technology.

Outline of Final Research Achievements

We studied and developed some advanced tensor decomposition and tensor network representation methods for incomplete and noisy data tensor. To improve its practicability, we also developed various algorithms for optimal tensor network structure search, and deep neural network based nonlinear flexible tensor decomposition methods. Moreover, we also studied adversarial robustness of deep neural networks under tensor representation of model parameters and developed several novel approaches for adversarial purification. Finally, our research findings can be adopted into some applications such as multi-model learning, hyperspectral image processing and etc.

Academic Significance and Societal Importance of the Research Achievements

Our research further promotes fundamental technology on tensor methods for machine learning. We have shown that tensor methods are powerful for structured data analysis and also practically useful for parameter representation of deep neural networks, resulting in more efficient and robust models.

Report

(5 results)
  • 2023 Annual Research Report   Final Research Report ( PDF )
  • 2022 Annual Research Report
  • 2021 Annual Research Report
  • 2020 Annual Research Report
  • Research Products

    (44 results)

All 2024 2023 2022 2021 2020

All Journal Article (37 results) (of which Int'l Joint Research: 37 results,  Peer Reviewed: 37 results,  Open Access: 16 results) Presentation (7 results) (of which Int'l Joint Research: 5 results,  Invited: 3 results)

  • [Journal Article] ADVERSARIAL TRAINING ON PURIFICATION (ATOP): ADVANCING BOTH ROBUSTNESS AND GENERALIZA- TION2024

    • Author(s)
      Guang Lin, Chao Li, Jianhai Zhang, Toshihisa Tanaka, Qibin Zhao
    • Journal Title

      The Twelfth International Conference on Learning Representations 2024

      Volume: -

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Efficient Nonparametric Tensor Decomposition for Binary and Count Data2024

    • Author(s)
      Tao Zerui、Tanaka Toshihisa、Zhao Qibin
    • Journal Title

      Proceedings of the AAAI Conference on Artificial Intelligence

      Volume: 38 Issue: 14 Pages: 15319-15327

    • DOI

      10.1609/aaai.v38i14.29456

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Nonparametric tensor ring decomposition with scalable amortized inference2024

    • Author(s)
      Tao Zerui、Tanaka Toshihisa、Zhao Qibin
    • Journal Title

      Neural Networks

      Volume: 169 Pages: 431-441

    • DOI

      10.1016/j.neunet.2023.10.031

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Bayesian tensor network structure search and its application to tensor completion2024

    • Author(s)
      Zeng Junhua、Zhou Guoxu、Qiu Yuning、Li Chao、Zhao Qibin
    • Journal Title

      Neural Networks

      Volume: 175 Pages: 106290-106290

    • DOI

      10.1016/j.neunet.2024.106290

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Hierarchical Attacks on Large-Scale Graph Neural Networks2024

    • Author(s)
      Zhang Jianfu、Hong Yan、Cheng Dawei、Zhang Liqing、Zhao Qibin
    • Journal Title

      Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing

      Volume: - Pages: 7635-7639

    • DOI

      10.1109/icassp48485.2024.10448076

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Alternating Local Enumeration (TnALE): Solving Tensor Network Structure Search with Fewer Evaluations2023

    • Author(s)
      Chao Li, Junhua Zeng, Chunmei Li, Cesar Caiafa, Qibin Zhao
    • Journal Title

      Proceedings of the 40th International Conference on Machine Learning

      Volume: -

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Fractional Tensor Recurrent Unit (fTRU): A Stable Forecasting Model With Long Memory2023

    • Author(s)
      Qiu Hejia、Li Chao、Weng Ying、Sun Zhun、Zhao Qibin
    • Journal Title

      IEEE Transactions on Neural Networks and Learning Systems

      Volume: - Issue: 2 Pages: 1-10

    • DOI

      10.1109/tnnls.2023.3338696

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Tree-Based Mix-Order Polynomial Fusion Network for Multimodal Sentiment Analysis2023

    • Author(s)
      Tang Jiajia、Hou Ming、Jin Xuanyu、Zhang Jianhai、Zhao Qibin、Kong Wanzeng
    • Journal Title

      Systems

      Volume: 11 Issue: 1 Pages: 44-44

    • DOI

      10.3390/systems11010044

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Scalable Bayesian Tensor Ring Factorization for Multiway Data Analysis2023

    • Author(s)
      Tao Zerui、Tanaka Toshihisa、Zhao Qibin
    • Journal Title

      Lecture Notes in Computer Science

      Volume: 14447 Pages: 490-503

    • DOI

      10.1007/978-981-99-8079-6_38

    • ISBN
      9789819980789, 9789819980796
    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Undirected Probabilistic Model for Tensor Decomposition2023

    • Author(s)
      Zerui Tao, Toshihisa Tanaka, Qibin Zhao
    • Journal Title

      Proceedings of the Thirty-eighth Annual Conference on Neural Information Processing Systems

      Volume: -

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Transformed Low-Rank Parameterization Can Help Robust Generalization for Tensor Neural Networks2023

    • Author(s)
      Andong Wang, Chao Li, Mingyuan Bai, Zhong Jin, Guoxu Zhou, Qibin Zhao
    • Journal Title

      Proceedings of the Thirty-eighth Annual Conference on Neural Information Processing Systems

      Volume: -

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Memorization Weights for Instance Reweighting in Adversarial Training2023

    • Author(s)
      Zhang Jianfu、Hong Yan、Zhao Qibin
    • Journal Title

      Proceedings of the AAAI Conference on Artificial Intelligence

      Volume: 37 Issue: 9 Pages: 11228-11236

    • DOI

      10.1609/aaai.v37i9.26329

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Classification of the Epileptic Seizure Onset Zone Based on Partial Annotation2022

    • Author(s)
      Xuyang Zhao, Qibin Zhao, Toshihisa Tanaka, Jordi Casals, Guoxu Zhou, Takumi Mitsuhashi, Hidenori Sugano, Noboru Yoshida, Jianting Cao
    • Journal Title

      Cognitive Neurodynamics, Springer

      Volume: 1 Issue: 3 Pages: 1-13

    • DOI

      10.1007/s11571-022-09857-4

    • Related Report
      2023 Annual Research Report 2022 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Fast Signal Completion Algorithm with Cyclic Convolutional Smoothing2022

    • Author(s)
      Takayama Hiromu、Yokota Tatsuya
    • Journal Title

      Proceedings of 2022 APSIPA Annual Summit and Conference

      Volume: - Pages: 364-371

    • DOI

      10.23919/apsipaasc55919.2022.9980284

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Accommodating Multiple Tasks’ Disparities With Distributed Knowledge-Sharing Mechanism2022

    • Author(s)
      Chen Xinqi、Zhou Guoxu、Wang Yanjiao、Hou Ming、Zhao Qibin、Xie Shengli
    • Journal Title

      IEEE Transactions on Cybernetics

      Volume: 52 Issue: 4 Pages: 2440-2452

    • DOI

      10.1109/tcyb.2020.3002911

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Permutation Search of Tensor Network Structures via Local Sampling2022

    • Author(s)
      Li Chao, Zeng Junhua, Tao Zerui, Zhao Qibin
    • Journal Title

      Proceedings of the 39th International Conference on Machine Learning

      Volume: -

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Imbalanced low-rank tensor completion via latent matrix factorization2022

    • Author(s)
      Qiu Yuning、Zhou Guoxu、Zeng Junhua、Zhao Qibin、Xie Shengli
    • Journal Title

      Neural Networks

      Volume: 155 Pages: 369-382

    • DOI

      10.1016/j.neunet.2022.08.023

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] SPD domain-specific batch normalization to crack interpretable unsupervised domain adaptation in EEG2022

    • Author(s)
      Reinamar J. Kobler, Jun-ichiro Hirayama, Qibin Zhao, Motoaki Kawanabe
    • Journal Title

      36th Conference on Neural Information Processing Systems (NeurIPS 2022)

      Volume: -

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] MMT: Multi-Way Multi-Modal Transformer for Multimodal Learning2022

    • Author(s)
      Jiajia Tang, Kang Li, Ming Hou, Xuanyu Jin, Wanzeng Kong, Yu Ding, Qibin Zhao
    • Journal Title

      Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence (IJCAI-22)

      Volume: -

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Consistent MDT-Tucker: A Hankel Structure Constrained Tucker Decomposition in Delay Embedded Space2022

    • Author(s)
      Yamamoto Ryuki、Hontani Hidekata、Imakura Akira、Yokota Tatsuya
    • Journal Title

      Proceedings of 2022 APSIPA Annual Summit and Conference

      Volume: - Pages: 137-142

    • DOI

      10.23919/apsipaasc55919.2022.9980035

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Bayesian Tensor Completion and Decomposition with Automatic CP Rank Determination Using MGP Shrinkage Prior2022

    • Author(s)
      Takayama Hiromu、Zhao Qibin、Hontani Hidekata、Yokota Tatsuya
    • Journal Title

      SN Computer Science

      Volume: 3 Issue: 3 Pages: 1-17

    • DOI

      10.1007/s42979-022-01119-8

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Manifold Modeling in Embedded Space: An Interpretable Alternative to Deep Image Prior2022

    • Author(s)
      Yokota Tatsuya、Hontani Hidekata、Zhao Qibin、Cichocki Andrzej
    • Journal Title

      IEEE Transactions on Neural Networks and Learning Systems

      Volume: 33 Issue: 3 Pages: 1022-1036

    • DOI

      10.1109/tnnls.2020.3037923

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Tensor Completion via Fully-Connected Tensor Network Decomposition with Regularized Factors2022

    • Author(s)
      Zheng Yu-Bang、Huang Ting-Zhu、Zhao Xi-Le、Zhao Qibin
    • Journal Title

      Journal of Scientific Computing

      Volume: 92 Issue: 1

    • DOI

      10.1007/s10915-022-01841-8

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Learning from Incomplete Features by Simultaneous Training of Neural Networks and Sparse Coding2021

    • Author(s)
      Caiafa Cesar F.、Wang Ziyao、Sole-Casals Jordi、Zhao Qibin
    • Journal Title

      IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops

      Volume: 2021 Pages: 2621-2630

    • DOI

      10.1109/cvprw53098.2021.00296

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Multi-distorted Image Restoration with Tensor 1 × 1 Convolutional Layer2021

    • Author(s)
      Huang Zihao、Li Chao、Duan Feng、Zhao Qibin
    • Journal Title

      International Joint Conference on Neural Networks (IJCNN)

      Volume: 2021 Pages: 2161-4407

    • DOI

      10.1109/ijcnn52387.2021.9533737

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] On the Memory Mechanism of Tensor-Power Recurrent Models2021

    • Author(s)
      Qiu Hejia, Li Chao, Weng Ying, Sun Zhun, He Xingyu, Zhao Qibin
    • Journal Title

      Proceedings of the 24th International Conference on Artifi- cial Intelligence and Statistics (AISTATS)

      Volume: 130 Pages: 3682-3690

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] zHybrid Convolutional Neural Network for Localization of Epileptic Focus Based on iEEG2021

    • Author(s)
      Linfeng Sui, Xuyang Zhao, Qibin Zhao, Toshihisa Tanaka and Jianting Cao
    • Journal Title

      Neural Plasticity

      Volume: Article ID 6644365 Pages: 1-9

    • DOI

      10.1155/2021/6644365

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Hide Chopin in the Music: Efficient Information Steganography Via Random Shuffling2021

    • Author(s)
      Sun Zhun、Li Chao、Zhao Qibin
    • Journal Title

      Proceeding of ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

      Volume: 1 Pages: 2370-2374

    • DOI

      10.1109/icassp39728.2021.9413357

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] CTFN: Hierarchical Learning for Multimodal Sentiment Analysis Using Coupled-Translation Fusion Network2021

    • Author(s)
      Tang Jiajia、Li Kang、Jin Xuanyu、Cichocki Andrzej、Zhao Qibin、Kong Wanzeng
    • Journal Title

      Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing

      Volume: 2021 Pages: 5301-5311

    • DOI

      10.18653/v1/2021.acl-long.412

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Tensor Recovery via L-Spectral k-Support Norm2021

    • Author(s)
      Wang Andong、Zhou Guoxu、Jin Zhong、Zhao Qibin
    • Journal Title

      IEEE Journal of Selected Topics in Signal Processing

      Volume: 15 Issue: 3 Pages: 522-534

    • DOI

      10.1109/jstsp.2021.3058763

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Tensor Decomposition Via Core Tensor Networks2021

    • Author(s)
      ZHANG Jianfu、TAO ZERUI、ZHANG LIQING、ZHAO QIBIN
    • Journal Title

      IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

      Volume: 2021 Pages: 2130-2134

    • DOI

      10.1109/icassp39728.2021.9413637

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Multi-feature Fusion for Epileptic Focus Localization Based on Tensor Representation2021

    • Author(s)
      Xuyang Zhao; Jordi Sole-Casals; Qibin Zhao; Jianting Cao; Toshihisa Tanaka
    • Journal Title

      Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)

      Volume: 2021 Pages: 1323-1327

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Fully-Connected Tensor Network Decomposition and Its Application to Higher-Order Tensor Completion2021

    • Author(s)
      Zheng Yu-Bang、Huang Ting-Zhu、Zhao Xi-Le、Zhao Qibin、Jiang Tai-Xiang
    • Journal Title

      Proceedings of the AAAI Conference on Artificial Intelligence

      Volume: 35 Issue: 12 Pages: 11071-11078

    • DOI

      10.1609/aaai.v35i12.17321

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Bayesian Latent Factor Model for Higher-order Data2021

    • Author(s)
      Zerui Tao, Xuyang Zhao, Toshihisa Tanaka, Qibin Zhao
    • Journal Title

      Proceedings of The 13th Asian Conference on Machine Learning

      Volume: 157 Pages: 1285-1300

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Non-local Meets Global: An Integrated Paradigm for Hyperspectral Image Restoration2020

    • Author(s)
      He Wei、Yao Quanming、Li Chao、Yokoya Naoto、Zhao Qibin、Zhang Hongyan、Zhang Liangpei
    • Journal Title

      IEEE Transactions on Pattern Analysis and Machine Intelligence

      Volume: 1 Pages: 1-1

    • DOI

      10.1109/tpami.2020.3027563

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Low Tensor-Ring Rank Completion by Parallel Matrix Factorization2020

    • Author(s)
      Yu Jinshi、Zhou Guoxu、Li Chao、Zhao Qibin、Xie Shengli
    • Journal Title

      IEEE Transactions on Neural Networks and Learning Systems

      Volume: 1 Issue: 7 Pages: 1-14

    • DOI

      10.1109/tnnls.2020.3009210

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] TPFN: Applying Outer Product along Time to Multimodal Sentiment Analysis Fusion on Incomplete Data2020

    • Author(s)
      Binghua Li, Chao Li, Feng Duan, Ning Zheng, Qibin Zhao
    • Journal Title

      ECCV 2020

      Volume: 1 Pages: 431-447

    • Related Report
      2020 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Presentation] Permutation Search of Tensor Network Structures via Local Sampling2022

    • Author(s)
      Chao Li
    • Organizer
      International Conference on Machine Learning
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] SPD domain-specific batch normalization to crack interpretable unsupervised domain adaptation in EEG2022

    • Author(s)
      Reinmar J. Kobler
    • Organizer
      36th Conference on Neural Information Processing Systems (NeurIPS 2022)
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] MMT: Multi-Way Multi-Modal Transformer for Multimodal Learning2022

    • Author(s)
      Jiajia Tang
    • Organizer
      Thirty-First International Joint Conference on Artificial Intelligence (IJCAI-22)
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Tensor Networks in Machine Learning: Recent Advances and Frontiers2021

    • Author(s)
      Qibin Zhao
    • Organizer
      SIAM Conference on Applied Linear Algebra (LA21)
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research / Invited
  • [Presentation] TPFN: Applying Outer Product along Time to Multimodal Sentiment Analysis Fusion on Incomplete Data2020

    • Author(s)
      Binghua Li
    • Organizer
      ECCV 2020
    • Related Report
      2020 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Tensor Networks in Machine Learning: Recent Advances and Frontiers2020

    • Author(s)
      Qibin Zhao
    • Organizer
      12th Asian Conference on Machine Learning (ACML 2020)
    • Related Report
      2020 Annual Research Report
    • Invited
  • [Presentation] Tensor Network Representations in Machine Learning2020

    • Author(s)
      Qibin Zhao
    • Organizer
      11th IFIP International Conference on Intelligent Information Processing
    • Related Report
      2020 Annual Research Report
    • Invited

URL: 

Published: 2020-04-28   Modified: 2025-01-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi