• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to project page

2023 Fiscal Year Annual Research Report

Knowledge-Base-Grounded Language Models

Research Project

Project/Area Number 21K17814
Research InstitutionInstitute of Physical and Chemical Research

Principal Investigator

HEINZERLING BENJAMIN  国立研究開発法人理化学研究所, 革新知能統合研究センター, 副チームリーダー (50846491)

Project Period (FY) 2021-04-01 – 2024-03-31
Keywordslanguage model / knowledge base / world knowledge / numeric properties / knowledge representation / interpretability
Outline of Annual Research Achievements

In 2023, we developed an interpretation method for analyzing how well LMs represent a specific kind of structured knowledge, namely an numeric properties such as a person's year of birth or a city's population.
This method allows directly analyzing and manipulating the internal state of LMs in order to control its behavior when generating output involving numeric properties.
In the broader context of interpretability, transparency, and explainability of LMs, this method contributes to a improved understanding of how LMs encode structured knowledge.

  • Research Products

    (5 results)

All 2024 2023

All Journal Article (5 results) (of which Int'l Joint Research: 4 results,  Peer Reviewed: 4 results,  Open Access: 5 results)

  • [Journal Article] Monotonic Representation of Numeric Properties in Language Models2024

    • Author(s)
      Heinzerling Benjamin, Inui Kentaro
    • Journal Title

      preprint

      Volume: - Pages: -

    • DOI

      10.48550/arXiv.2403.10381

    • Open Access
  • [Journal Article] Examining the effect of whitening on static and contextualized word embeddings2023

    • Author(s)
      Sasaki Shota、Heinzerling Benjamin、Suzuki Jun、Inui Kentaro
    • Journal Title

      Information Processing & Management

      Volume: 60 Pages: 103272~103272

    • DOI

      10.1016/j.ipm.2023.103272

    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Prompting for explanations improves Adversarial NLI. Is this true? {Yes} it is {true} because {it weakens superficial cues}2023

    • Author(s)
      Kavumba Pride、Brassard Ana、Heinzerling Benjamin、Inui Kentaro
    • Journal Title

      : Findings of the Association for Computational Linguistics: EACL 2023

      Volume: 1 Pages: 2165-2180

    • DOI

      10.18653/v1/2023.findings-eacl.162

    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Can LMs Store and Retrieve 1-to-N Relational Knowledge?2023

    • Author(s)
      Nagasawa Haruki、Heinzerling Benjamin、Kokuta Kazuma、Inui Kentaro
    • Journal Title

      Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)

      Volume: 4 Pages: 130-138

    • DOI

      10.18653/v1/2023.acl-srw.22

    • Peer Reviewed / Open Access / Int'l Joint Research
  • [Journal Article] Test-time Augmentation for Factual Probing2023

    • Author(s)
      Kamoda Go、Heinzerling Benjamin、Sakaguchi Keisuke、Inui Kentaro
    • Journal Title

      Findings of the Association for Computational Linguistics: EMNLP 2023

      Volume: 1 Pages: 3650-3661

    • DOI

      10.18653/v1/2023.findings-emnlp.236

    • Peer Reviewed / Open Access / Int'l Joint Research

URL: 

Published: 2024-12-25  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi