• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

任意ツールでのデザイン支援のためのGANを用いたサンプル模倣フレームワーク

Research Project

Project/Area Number 22KJ0353
Project/Area Number (Other) 21J20353 (2021-2022)
Research Category

Grant-in-Aid for JSPS Fellows

Allocation TypeMulti-year Fund (2023)
Single-year Grants (2021-2022)
Section国内
Review Section Basic Section 61020:Human interface and interaction-related
Research InstitutionUniversity of Tsukuba

Principal Investigator

矢倉 大夢  筑波大学, 理工情報生命学術院, 特別研究員(DC1)

Project Period (FY) 2023-03-08 – 2024-03-31
Project Status Completed (Fiscal Year 2023)
Budget Amount *help
¥1,300,000 (Direct Cost: ¥1,300,000)
Fiscal Year 2023: ¥400,000 (Direct Cost: ¥400,000)
Fiscal Year 2022: ¥400,000 (Direct Cost: ¥400,000)
Fiscal Year 2021: ¥500,000 (Direct Cost: ¥500,000)
Keywords創造性支援 / 生産性支援 / 機械学習 / ヒューマンコンピュータインタラクション / 創作支援 / 深層学習
Outline of Research at the Start

Generative Adversarial Networks (GANs)に代表される深層学習モデルを、エンドユーザが自由に活かせるようなインタラクションフレームワークの提示を目指す。またこれらを通して、人間による探索や新しいものの創造に寄与する形での、深層学習モデルの応用についての方法論の議論を深めることを狙う。

Outline of Annual Research Achievements

本研究では様々な機械学習モデルを、一般のユーザがより直観的にかつ創造的に使えるようになることを目指し、新たなシステムの開発やフレームワークの提示を行ってきた。当初はGANを中心にした技術開発を元に論文発表を行っていたが、大規模生成モデル等の新たな機械学習モデルも広がりつつあることを踏まえ、そうしたモデルも含めた形でのデザイン支援研究を行った。例えば、テキストから音楽を生成できるモデルについて、初心者ユーザは音楽に関する語彙に乏しく、適切に使いこなすことが難しいという課題に対し、大規模言語モデルを組み合わせながら幅広く生成結果を探索し、新たな語彙に触れることを可能にするインタラクションシステムを開発した。また、ユーザの(物理的な)挙動についてスマートウォッチから取得できるデータと機械学習モデルを組み合わせて、ユーザの作業を支援するシステムを開発という場合に、システムにどういったインタラクションを実装すべきかのデザインを支援するフレームワークをカーネギーメロン大学の研究室と共同で開発した。これらの内容は、ISMIR 2024への採択及びProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologiesへの採録につながった。
また、3年に渡る研究を総合する形で、得られた成果を幅広い機械学習モデルを一般のユーザが使うためのガイドラインとして整理した。ユーザのアウトプットを模倣するというアイデアを拡張しながら、その上でユーザのデザイン空間の探索をどう支援するかという点が重要だということを、過去の成果を踏まえながら主要な主張として明らかにすることができた。

Report

(3 results)
  • 2023 Annual Research Report
  • 2022 Annual Research Report
  • 2021 Annual Research Report
  • Research Products

    (13 results)

All 2023 2022 2021 Other

All Int'l Joint Research (1 results) Journal Article (8 results) (of which Int'l Joint Research: 4 results,  Peer Reviewed: 8 results,  Open Access: 8 results) Presentation (4 results) (of which Int'l Joint Research: 3 results,  Invited: 1 results)

  • [Int'l Joint Research] Carnegie Mellon University/Case Western Reserve University/University Hospitals of Cleveland(米国)

    • Related Report
      2023 Annual Research Report
  • [Journal Article] CatAlyst: Domain-Extensible Intervention for Preventing Task Procrastination Using Large Generative Models2023

    • Author(s)
      Arakawa Riku、Yakura Hiromu、Goto Masataka
    • Journal Title

      Proceedings of the 2023 ACM CHI Conference on Human Factors in Computing Systems

      Volume: 157 Pages: 1-19

    • DOI

      10.1145/3544548.3581133

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] PrISM-Tracker: A Framework for Multimodal Procedure Tracking using Wearable Sensors and State Transition Information with User-Driven Handling of Errors and Uncertainty2023

    • Author(s)
      Arakawa Riku、Yakura Hiromu、Mollyn Vimal、Nie Suzanne、Russell Emma、DeMeo Dustin P.、Reddy Haarika A.、Maytin Alexander K.、Carroll Bryan T.、Lehman Jill Fain、Goel Mayank
    • Journal Title

      Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

      Volume: 6 Issue: 4 Pages: 1-27

    • DOI

      10.1145/3569504

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] IteraTTA: An Interface for Exploring Both Text Prompts and Audio Priors in Generating Music with Text-to-Audio Models2023

    • Author(s)
      Yakura Hiromu、Goto Masataka
    • Journal Title

      Proceedings of the 24th International Society for Music Information Retrieval Conference

      Volume: 14 Pages: 129-137

    • Related Report
      2023 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] An automated system recommending background music to listen to while working2022

    • Author(s)
      Yakura Hiromu、Nakano Tomoyasu、Goto Masataka
    • Journal Title

      User Modeling and User-Adapted Interaction

      Volume: 32 Issue: 3 Pages: 355-388

    • DOI

      10.1007/s11257-022-09325-y

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] Self-Supervised Contrastive Learning for Singing Voices2022

    • Author(s)
      Hiromu Yakura, Kento Watanabe, Masataka Goto
    • Journal Title

      IEEE/ACM Transactions on Audio, Speech, and Language Processing

      Volume: 30 Pages: 1614-1623

    • DOI

      10.1109/taslp.2022.3169627

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access
  • [Journal Article] VocabEncounter: NMT-powered Vocabulary Learning by Presenting Computer-Generated Usages of Foreign Words into Users’ Daily Lives2022

    • Author(s)
      Arakawa Riku、Yakura Hiromu、Kobayashi Sosuke
    • Journal Title

      Proceedings of the 2022 ACM CHI Conference on Human Factors in Computing Systems

      Volume: 6 Pages: 1-21

    • DOI

      10.1145/3491102.3501839

    • Related Report
      2022 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] BeParrot: Efficient Interface for Transcribing Unclear Speech via Respeaking2022

    • Author(s)
      Riku Arakawa, Hiromu Yakura (equal contribution), and Masataka Goto
    • Journal Title

      Proceedings of the 27th ACM International Conference on Intelligent User Interface

      Volume: 1 Pages: 832-840

    • DOI

      10.1145/3490099.3511164

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Tool- and Domain-Agnostic Parameterization of Style Transfer Effects Leveraging Pretrained Perceptual Metrics2021

    • Author(s)
      Hiromu Yakura, Yuki Koyama, and Masataka Goto
    • Journal Title

      Proceedings of the 30th International Joint Conference on Artificial Intelligence

      Volume: 1 Pages: 1208-1216

    • DOI

      10.24963/ijcai.2021/167

    • Related Report
      2021 Annual Research Report
    • Peer Reviewed / Open Access
  • [Presentation] Failure-Resistant Intelligent Interaction for Reliable Human-AI Collaboration2023

    • Author(s)
      Yakura Hiromu
    • Organizer
      Doctorial Consortium of the 2023 AAAI Conference on Artificial Intelligence
    • Related Report
      2023 Annual Research Report
    • Int'l Joint Research
  • [Presentation] A Generative Framework for Designing Interactions to Overcome the Gaps between Humans and Imperfect AIs Instead of Improving the Accuracy of the AIs2023

    • Author(s)
      Yakura Hiromu
    • Organizer
      Doctorial Consortium of the 2023 CHI Conference on Human Factors in Computing Systems
    • Related Report
      2023 Annual Research Report
    • Int'l Joint Research
  • [Presentation] AI for human assessment: What do professional assessors need?2023

    • Author(s)
      Arakawa Riku、Yakura Hiromu
    • Organizer
      The 2023 CHI Conference on Human Factors in Computing Systems
    • Related Report
      2023 Annual Research Report
    • Int'l Joint Research
  • [Presentation] CatAlyst: Domain-Extensible Intervention for Preventing Task Procrastination Using Large Generative Models2023

    • Author(s)
      矢倉 大夢、荒川 陸、後藤 真孝
    • Organizer
      第31回インタラクティブシステムとソフトウェアに関するワークショップ
    • Related Report
      2023 Annual Research Report
    • Invited

URL: 

Published: 2021-05-27   Modified: 2024-12-25  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi