• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Understanding Concrete and Abstract Representations in Art

Research Project

Project/Area Number 20K19822
Research Category

Grant-in-Aid for Early-Career Scientists

Allocation TypeMulti-year Fund
Review Section Basic Section 61010:Perceptual information processing-related
Research InstitutionOsaka University

Principal Investigator

GARCIA・DOCAMPO NOA  大阪大学, データビリティフロンティア機構, 特任助教(常勤) (80870005)

Project Period (FY) 2020-04-01 – 2023-03-31
Project Status Granted (Fiscal Year 2021)
Budget Amount *help
¥2,860,000 (Direct Cost: ¥2,200,000、Indirect Cost: ¥660,000)
Fiscal Year 2021: ¥650,000 (Direct Cost: ¥500,000、Indirect Cost: ¥150,000)
Fiscal Year 2020: ¥2,210,000 (Direct Cost: ¥1,700,000、Indirect Cost: ¥510,000)
Keywordsautomatic art analysis / art description / artwork recognition / visual recognition / image captioning / art analysis / visual understanding / deep neural networks / transfer learning / question answering
Outline of Research at the Start

This research studies how machines can understand and interpret art as we humans do. When we look at a painting, we "read" it by recognizing its depicted objects, its emotions, its style, etc. By using machine learning tools, we will replicate this behavior and provide new interpretations of art.

Outline of Annual Research Achievements

In this project, we have investigated low and high-level visual representations for art images. Low-level representations contain specific and detailed information about the style and textures of the image. This is helpful to identify a specific pieace of art from a large catalog. On the other hand, high-level representations contain more contextual information, which is can be used to explain certain aspects of the artwork such as the year of creation, the author, ot the topic.

Current Status of Research Progress
Current Status of Research Progress

3: Progress in research has been slightly delayed.

Reason

Due to the coronavirus pandemic, the project has been slightly delayed until the end of 2022.

Strategy for Future Research Activity

The plan for the next months research is to finish the project and investigate how visual representations from paintings can capture human emotions.

Report

(2 results)
  • 2021 Research-status Report
  • 2020 Research-status Report

Research Products

(24 results)

All 2021 2020 Other

All Int'l Joint Research (10 results) Presentation (10 results) (of which Int'l Joint Research: 8 results,  Invited: 2 results) Remarks (4 results)

  • [Int'l Joint Research] Czech Technical University in Prague(チェコ)

    • Related Report
      2021 Research-status Report
  • [Int'l Joint Research] University of Amsterdam(オランダ)

    • Related Report
      2021 Research-status Report
  • [Int'l Joint Research] Columbia University(米国)

    • Related Report
      2021 Research-status Report
  • [Int'l Joint Research] University Rennes(フランス)

    • Related Report
      2021 Research-status Report
  • [Int'l Joint Research] Istituto Italiano di Tecnologia(イタリア)

    • Related Report
      2021 Research-status Report
  • [Int'l Joint Research] Chinese Academy of Sciences(中国)

    • Related Report
      2020 Research-status Report
  • [Int'l Joint Research] Carnegie Mellon University/Columbia University(米国)

    • Related Report
      2020 Research-status Report
  • [Int'l Joint Research] University of Bamberg(ドイツ)

    • Related Report
      2020 Research-status Report
  • [Int'l Joint Research] Czech Technical University in Prague(チェコ)

    • Related Report
      2020 Research-status Report
  • [Int'l Joint Research] University of Amsterdam(オランダ)

    • Related Report
      2020 Research-status Report
  • [Presentation] Explain Me the Painting: Multi-Topic Knowledgeable Art Description Generation2021

    • Author(s)
      Zechen Bai, Yuta Nakashima, Noa Garcia
    • Organizer
      IEEE/CVF International Conference on Computer Vision 2021
    • Related Report
      2021 Research-status Report
    • Int'l Joint Research
  • [Presentation] Instance-level Recognition for Artworks: The MET Dataset2021

    • Author(s)
      Nikolaos-Antonios Ypsilantis, Noa Garcia, Guangxing Han, Sarah Ibrahimi, Nanne Van Noord, Giorgos Tolias
    • Organizer
      Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track 2021
    • Related Report
      2021 Research-status Report
    • Int'l Joint Research
  • [Presentation] Transferring Domain-Agnostic Knowledge in Video Question Answering2021

    • Author(s)
      Tianran Wu, Noa Garcia, Mayu Otani, Chenhui Chu, Yuta Nakashima, Haruo Takemura
    • Organizer
      British Machine Vision Conference 2021
    • Related Report
      2021 Research-status Report
    • Int'l Joint Research
  • [Presentation] GCNBoost: Artwork Classification by Label Propagation through a Knowledge Graph2021

    • Author(s)
      Cheikh Brahim El Vaigh, Noa Garcia, Benjamin Renoust, Chenhui Chu, Yuta Nakashima, Hajime Nagahara
    • Organizer
      ACM International Conference in Multimedia Retrieval 2021
    • Related Report
      2021 Research-status Report
    • Int'l Joint Research
  • [Presentation] Visual Question Answering with Textual Representations2021

    • Author(s)
      Yusuke Hirota, Noa Garcia, Mayu Otani, Chenhui Chu, Yuta Nakashima, Ittetsu Taniguchi, Takao Onoye
    • Organizer
      Workshop on Closing the Loop Between Vision and Language, IEEE/CVF International Conference on Computer Vision 2021
    • Related Report
      2021 Research-status Report
    • Int'l Joint Research
  • [Presentation] Understanding Fine-Art Paintings through Visual and Language Representations2021

    • Author(s)
      Noa Garcia
    • Organizer
      CAI+CAI Workshop, Natural Language Processing 2021
    • Related Report
      2021 Research-status Report
    • Invited
  • [Presentation] Understanding Fine-Art Paintings through Visual and Language Representations2021

    • Author(s)
      Noa Garcia
    • Organizer
      CAI+CAI co-held at Natural Language Processing 2021 Conference (言語処理学会年次大会2021)
    • Related Report
      2020 Research-status Report
    • Invited
  • [Presentation] Knowledge-Based Video Question Answering with Unsupervised Scene Descriptions2020

    • Author(s)
      Noa Garcia, Yuta Nakashima
    • Organizer
      European Conference on Computer Vision
    • Related Report
      2020 Research-status Report
    • Int'l Joint Research
  • [Presentation] A Dataset and Baselines for Visual Question Answering on Art2020

    • Author(s)
      Noa Garcia, Chentao Ye, Zihua Liu, Qingtao Hu, Mayu Otani, Chenhui Chu, Yuta Nakashima and Teruko Mitamura
    • Organizer
      VISART workshop at European Conference on Computer Vision 2020
    • Related Report
      2020 Research-status Report
    • Int'l Joint Research
  • [Presentation] Demographic Influences on Contemporary Art with Unsupervised Style Embeddings2020

    • Author(s)
      Nikolai Huckle, Noa Garcia and Yuta Nakashima
    • Organizer
      VISART workshop at European Conference on Computer Vision 2020
    • Related Report
      2020 Research-status Report
    • Int'l Joint Research
  • [Remarks] Art Description Generation

    • URL

      https://sites.google.com/view/art-description-generation

    • Related Report
      2021 Research-status Report
  • [Remarks] The Met Dataset

    • URL

      http://cmp.felk.cvut.cz/met/

    • Related Report
      2021 Research-status Report
  • [Remarks] Visual Question Answering on Art

    • URL

      https://github.com/noagarcia/ArtVQA

    • Related Report
      2021 Research-status Report
  • [Remarks] contempArt

    • URL

      https://contempart.org/

    • Related Report
      2021 Research-status Report

URL: 

Published: 2020-04-28   Modified: 2022-12-28  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi