• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Task Fusion Learning in Deep Learning

Research Project

Project/Area Number 22K19808
Research Category

Grant-in-Aid for Challenging Research (Exploratory)

Allocation TypeMulti-year Fund
Review Section Medium-sized Section 61:Human informatics and related fields
Research InstitutionThe University of Electro-Communications

Principal Investigator

Yanai Keiji  電気通信大学, 大学院情報理工学研究科, 教授 (20301179)

Project Period (FY) 2022-06-30 – 2024-03-31
Project Status Completed (Fiscal Year 2023)
Budget Amount *help
¥6,370,000 (Direct Cost: ¥4,900,000、Indirect Cost: ¥1,470,000)
Fiscal Year 2023: ¥3,250,000 (Direct Cost: ¥2,500,000、Indirect Cost: ¥750,000)
Fiscal Year 2022: ¥3,120,000 (Direct Cost: ¥2,400,000、Indirect Cost: ¥720,000)
Keywords深層学習 / 継続学習 / 大規模モデル / 視覚言語モデル / 連続学習 / Vision Transformer / Stable Diffusion / Zero-shot segmentation / マルチタスク学習
Outline of Research at the Start

本研究では,単一のCNN (Convolutional Neural Network, 畳み込みネットワーク)に複数のタスクのための機能を同時に学習させ,学習した独立の機能の組み合わせ・重ね合わせによって,事前に学習した個別の単一機能とは異なる,新しい機能を実現し,Neural Networkがより人間の脳に近い汎用的な能力を持っていることを実証することを目的とする.また,複数機能の組み合わせによる実用的な応用についても検討する.

Outline of Final Research Achievements

In this study, we started research to demonstrate that Neural Networks have general-purpose capabilities that are more similar to those of the human brain by having a single neural network learn functions for multiple tasks simultaneously, and by combining and superimposing the learned independent functions to realize new functions that are different from the individual single functions that were learned beforehand. The following three specific research projects has beed studied. (1) Superimposition of image transformation tasks using conditional signals. (2) Continuous learning of Vision Transformer(ViT).
(3) Stable Diffusion for region segmentation of arbitrary words without learning: Using a large-scale trained image generation model, we extract regions corresponding to words without additional learning.

Academic Significance and Societal Importance of the Research Achievements

本研究によって,ニューラルネットワークには,異なる機能の同時学習に関してより柔軟に対応できる能力が備わっていることが示された.また,10億スケールの大規模画像言語ペアデータで学習されたテキストからの画像生成モデルには,テキストと視覚概念をピクセルレベルで対応付ける能力が備わっていることが示され,追加の学習なしで多様なタスクに活用できる可能性が大いに高まったと言える.今後は,この学習なしの能力を多様なタスクに対して実証し,それらの複合的な処理も学習無しで実現することを追求することで,大規模視覚言語モデルの応用可能性を大いに広げることが可能となる.

Report

(3 results)
  • 2023 Annual Research Report   Final Research Report ( PDF )
  • 2022 Research-status Report
  • Research Products

    (26 results)

All 2024 2023 2022 2021

All Journal Article (4 results) (of which Int'l Joint Research: 1 results,  Peer Reviewed: 4 results,  Open Access: 4 results) Presentation (22 results) (of which Int'l Joint Research: 19 results,  Invited: 1 results)

  • [Journal Article] Multi-Style Shape Matching GAN for Text Images2024

    • Author(s)
      YUAN Honghui、YANAI Keiji
    • Journal Title

      IEICE Transactions on Information and Systems

      Volume: E107.D Issue: 4 Pages: 505-514

    • DOI

      10.1587/transinf.2023IHP0010

    • ISSN
      0916-8532, 1745-1361
    • Year and Date
      2024-04-01
    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Patent image retrieval using transformer-based deep metric learning2023

    • Author(s)
      Higuchi Kotaro、Yanai Keiji
    • Journal Title

      World Patent Information

      Volume: 74 Pages: 102217-102217

    • DOI

      10.1016/j.wpi.2023.102217

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Material Translation Based on Neural Style Transfer with Ideal Style Image Retrieval2022

    • Author(s)
      Benitez-Garcia Gibran、Takahashi Hiroki、Yanai Keiji
    • Journal Title

      Sensors

      Volume: 22 Issue: 19 Pages: 7317-7317

    • DOI

      10.3390/s22197317

    • Related Report
      2022 Research-status Report
    • Peer Reviewed / Open Access
  • [Journal Article] FASSD-Net: Fast and Accurate Real-Time Semantic Segmentation for Embedded Systems2021

    • Author(s)
      Rosas-Arias Leonel、Benitez-Garcia Gibran、Portillo-Portillo Jose、Olivares-Mercado Jesus、Sanchez-Perez Gabriel、Yanai Keiji
    • Journal Title

      IEEE Transactions on Intelligent Transportation Systems

      Volume: - Issue: 9 Pages: 1-12

    • DOI

      10.1109/tits.2021.3127553

    • Related Report
      2022 Research-status Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Presentation] Training-free Region Prediction with Stable Diffusion,2024

    • Author(s)
      Yuma Honbu and Keiji Yanai
    • Organizer
      International Conference on MultimeMedia Modeling (MMM)
    • Related Report
      2023 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Training-Free Region Prediction with Stable Diffusion.2024

    • Author(s)
      Yuma Honbu and Keiji Yanai
    • Organizer
      International Multimedia Modeling Conference (MMM)
    • Related Report
      2023 Annual Research Report
  • [Presentation] 生成AI技術の基礎と展望2024

    • Author(s)
      柳井 啓司
    • Organizer
      SID日本支部 講演会 「生成AIとメタバースが開く社会に向けたディスプレイ技術と課題」
    • Related Report
      2023 Annual Research Report
    • Invited
  • [Presentation] Mask-based Food Image Synthesis with Cross-Modal Recipe Embeddings2023

    • Author(s)
      Zhongtao Chen, Yuma Hombu and Keiji Yanai
    • Organizer
      ACM Multimedia Asia
    • Related Report
      2023 Annual Research Report
    • Int'l Joint Research
  • [Presentation] VQ-VDM: Video Diffusion Models with 3D VQGAN2023

    • Author(s)
      Ryota Kaji and Keiji Yanai
    • Organizer
      ACM Multimedia Asia
    • Related Report
      2023 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Contextual Associated Triplet Queries for Panoptic Scene Graph Generation2023

    • Author(s)
      Jingbin Xu, Junwen Chen and Keiji Yanai
    • Organizer
      ACM Multimedia Asia
    • Related Report
      2023 Annual Research Report
    • Int'l Joint Research
  • [Presentation] QAHOI: Query-Based Anchors for Human-Object Interaction Detection2023

    • Author(s)
      Junwen Chen and Keiji Yanai
    • Organizer
      International Conference on Machine Vision Applications (MVA)
    • Related Report
      2023 Annual Research Report
    • Int'l Joint Research
  • [Presentation] CalorieCam360: Simultaneous Eating Action Recognition of Multiple People Using Omnidirectional Camera2023

    • Author(s)
      Kento Terauchi and Keiji Yanai
    • Organizer
      ACM International Conference on Multimedia Retrieval (ICMR)
    • Related Report
      2023 Annual Research Report
    • Int'l Joint Research
  • [Presentation] HowToEat: Exploring Human Object Interaction and Eating Action in Eating Scenarios2023

    • Author(s)
      Yingcheng Wang, Junwen Chen and Keiji Yanai
    • Organizer
      ACMMM Workshop on Multimedia Assisted Dietary Management (MADIMA)
    • Related Report
      2023 Annual Research Report
    • Int'l Joint Research
  • [Presentation] StableSeg: Stable Diffusionによるゼロショット領域分割2023

    • Author(s)
      本部 勇真, 山口 廉斗, 柳井 啓司
    • Organizer
      画像の認識・理解シンポジウム (MIRU)
    • Related Report
      2023 Annual Research Report
  • [Presentation] Patent Image RetrievalUsing Cross-entropy-based Metric Learning2023

    • Author(s)
      Kotaro Higuchi,Yuma Honbu,Keiji Yanai
    • Organizer
      Proc.of International Workshop on Frontiers of Computer Vision (IW-FCV),
    • Related Report
      2022 Research-status Report
    • Int'l Joint Research
  • [Presentation] Virtual Try-On Considering Temporal Consistency for Videoconferencing.2023

    • Author(s)
      Daiki Shimizu,Keiji Yanai
    • Organizer
      Proc. of the International Multimedia Modeling Conference (MMM)
    • Related Report
      2022 Research-status Report
    • Int'l Joint Research
  • [Presentation] Transformer-Based Cross-Modal Recipe Embeddings with Large Batch Training.2023

    • Author(s)
      Jing Yang,Junwen Chen,Keiji Yanai
    • Organizer
      Proc. of the International Multimedia Modeling Conference (MMM)
    • Related Report
      2022 Research-status Report
    • Int'l Joint Research
  • [Presentation] Zero-shot Font Style Transfer with a Differentiable Renderer2022

    • Author(s)
      Kota Izumi,Keiji Yanai
    • Organizer
      Proc. of ACM Multimedia Asia
    • Related Report
      2022 Research-status Report
    • Int'l Joint Research
  • [Presentation] Parallel Queries for Human-Object Interaction Detection2022

    • Author(s)
      Junwen Chen,Keiji Yanai
    • Organizer
      Proc. of ACM Multimedia Asia
    • Related Report
      2022 Research-status Report
    • Int'l Joint Research
  • [Presentation] SetMealAsYouLike: Sketch-based Set Meal Image Synthesis with Plate Annotations2022

    • Author(s)
      Yuma Honbu,Keiji Yanai
    • Organizer
      Proc. of ACMMM Workshop on Multimedia Assisted Dietary Management (MADIMA)
    • Related Report
      2022 Research-status Report
    • Int'l Joint Research
  • [Presentation] DepthGrillCam: A Mobile Application for Real-time Eating Action Recording Using RGB-D Images2022

    • Author(s)
      Kento Adachi,Keiji Yanai
    • Organizer
      Proc. of ACMMM Workshop on Multimedia Assisted Dietary Management (MADIMA)
    • Related Report
      2022 Research-status Report
    • Int'l Joint Research
  • [Presentation] Text-based Image Editing for Food Images with CLIP2022

    • Author(s)
      Kohei Yamamoto,Keiji Yanai
    • Organizer
      Proc. of ACMMM Workshop on Multimedia Assisted Dietary Management (MADIMA)
    • Related Report
      2022 Research-status Report
    • Int'l Joint Research
  • [Presentation] Real Scale 3D Reconstruction of a Dish and a Plate using Implicit Function and a Single RGB-D Image2022

    • Author(s)
      Shu Naritomi,Keiji Yanai
    • Organizer
      Proc. of ACMMM Workshop on Multimedia Assisted Dietary Management (MADIMA)
    • Related Report
      2022 Research-status Report
    • Int'l Joint Research
  • [Presentation] Continual Learning in Vision Transformer2022

    • Author(s)
      Mana Takeda,Keiji Yanai
    • Organizer
      Proc.of IEEE International Conference on Image Processing (ICIP)
    • Related Report
      2022 Research-status Report
    • Int'l Joint Research
  • [Presentation] StyleGAN-based CLIP-guided Image Shape Manipulation2022

    • Author(s)
      Yuchen Qian,Kohei Yamamoto,Keiji Yanai
    • Organizer
      Proc.of International Conference on Content-based Multimedia Indexing (CBMI)
    • Related Report
      2022 Research-status Report
    • Int'l Joint Research
  • [Presentation] Unseen Food Segmentation2022

    • Author(s)
      Yuma Honbu,Keiji Yanai
    • Organizer
      Proc.of ACM International Conference on Multimedia Retrieval (ICMR)
    • Related Report
      2022 Research-status Report
    • Int'l Joint Research

URL: 

Published: 2022-07-05   Modified: 2025-01-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi