• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Deep metric learning through local attention and generative loss

Research Project

Project/Area Number 21K17761
Research Category

Grant-in-Aid for Early-Career Scientists

Allocation TypeMulti-year Fund
Review Section Basic Section 61010:Perceptual information processing-related
Research InstitutionTokyo University of Agriculture and Technology

Principal Investigator

NGUYENTUAN CUONG  東京農工大学, 工学(系)研究科(研究院), 特任助教 (10814246)

Project Period (FY) 2021-04-01 – 2023-03-31
Project Status Discontinued (Fiscal Year 2022)
Budget Amount *help
¥4,420,000 (Direct Cost: ¥3,400,000、Indirect Cost: ¥1,020,000)
Fiscal Year 2023: ¥780,000 (Direct Cost: ¥600,000、Indirect Cost: ¥180,000)
Fiscal Year 2022: ¥780,000 (Direct Cost: ¥600,000、Indirect Cost: ¥180,000)
Fiscal Year 2021: ¥2,860,000 (Direct Cost: ¥2,200,000、Indirect Cost: ¥660,000)
KeywordsSimilarity / Transformer / Graph Neural Networks / Minimum Spanning Tree / 距離学習 / Seq2Seq / 手書き数式 / クラスタリング / deep metric learning / clustering / mathematical expression / handwriting
Outline of Research at the Start

Learning structural representation of data samples through their sub-structures and the relations between these sub-structures is efficient since the sub-structures and relations are shared among the data samples. As supervised learning models requires annotations of these sub-structures and relations, which is costly and unfeasible for some type of data, automatic learning of these sub-structures and relations are necessary. We propose a deep metric learning method based on attention generative model for automatically focus on learning the discriminative local features as the sub-structures.

Outline of Annual Research Achievements

今年はオンライン手書き数式の類似度とGraph手法で手書き数式認識の研究をした。オンライン手書き数式の類似度では、数式パターンとLatexの列の類似度を推定する手法をである。手書き数式答案の採点や手書き数式検索などの応用がある。この類似度を計算するため、シンボル・位置関係の列の経由で、オンライン手書き数式をRecurrent Neural Networksに入力し、シンボル・位置関係列を出力し、また、LatexからTransformerSeq2Seqを入力し、シンボル・位置関係を出力する。その二つのシンボル・位置関係列をTemporal Classification Lossで、類似度を推定ができた。Graph手法で手書き数式認識では、オンライン手書き数式からRecurrent Neural Networksを入力し、シンボル・位置関係列を出力し、シンボル全体の位置関係を計算することができる。このGraphを数式の標準(Symbol Label Graph)を生成するため、Minimum Spanning Treeアルゴリズムを研究した。Graph手法は手書き数式認識だけではなく、数式の構造まで認識を行い、また、手書き数式の書き順を依存なしで認識ができた。

Report

(2 results)
  • 2022 Annual Research Report
  • 2021 Research-status Report
  • Research Products

    (10 results)

All 2022 2021

All Journal Article (1 results) (of which Peer Reviewed: 1 results) Presentation (9 results) (of which Int'l Joint Research: 7 results)

  • [Journal Article] Syntactic data generation for handwritten mathematical expression recognition2022

    • Author(s)
      [2]Thanh-Nghia Truong, Cuong Tuan Nguyen, and Masaki Nakagawa
    • Journal Title

      Pattern Recognition Letters

      Volume: Vol. 153 Pages: 83-91

    • DOI

      10.1016/j.patrec.2021.12.002

    • Related Report
      2021 Research-status Report
    • Peer Reviewed
  • [Presentation] Handwriting Recognition and Automatic Scoring for Descriptive Answers in Japanese Language Tests2022

    • Author(s)
      Hung Tuan Nguyen, Cuong Tuan Nguyen, Haruki Oka, Tsunenori Ishioka, Masaki Nakagawa
    • Organizer
      18th International Conference on Frontiers in Handwriting Recognition, ICFHR 2020
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Fully Automated Short Answer Scoring of the Trial Tests for Common Entrance Examinations for Japanese University2022

    • Author(s)
      Haruki Oka, Hung Tuan Nguyen, Cuong Tuan Nguyen, Masaki Nakagawa, Tsunenori Ishioka
    • Organizer
      23rd International Conference on Artificial Intelligence in Education, AIED 2022
    • Related Report
      2022 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Learning Symbol Relation Tree for Online Handwritten Mathematical Expression Recognition2021

    • Author(s)
      Truong Thanh-Nghia、Nguyen Hung Tuan、Nguyen Cuong Tuan、Nakagawa Masaki
    • Organizer
      Asian Conference on Pattern Recognition, ACPR2021
    • Related Report
      2021 Research-status Report
    • Int'l Joint Research
  • [Presentation] Temporal Classification Constraint for Improving Handwritten Mathematical Expression Recognition2021

    • Author(s)
      Cuong Tuan Nguyen, Hung Tuan Nguyen, Kei Morizumi & Masaki Nakagawa
    • Organizer
      International Conference on Document Analysis and Recognition, ICDAR 2021 Workshops
    • Related Report
      2021 Research-status Report
    • Int'l Joint Research
  • [Presentation] A Transformer-Based Math Language Model for Handwritten Math Expression Recognition2021

    • Author(s)
      Huy Quang Ung, Cuong Tuan Nguyen, Hung Tuan Nguyen, Thanh-Nghia Truong & Masaki Nakagawa
    • Organizer
      International Conference on Document Analysis and Recognition, ICDAR 2021 Workshops
    • Related Report
      2021 Research-status Report
    • Int'l Joint Research
  • [Presentation] GSSF: A Generative Sequence Similarity Function Based on a Seq2Seq Model for Clustering Online Handwritten Mathematical Answers2021

    • Author(s)
      Huy Quang Ung, Cuong Tuan Nguyen, Hung Tuan Nguyen & Masaki Nakagawa
    • Organizer
      International Conference on Document Analysis and Recognition, ICDAR 2021
    • Related Report
      2021 Research-status Report
    • Int'l Joint Research
  • [Presentation] Global Context for Improving Recognition of Online Handwritten Mathematical Expressions2021

    • Author(s)
      Cuong Tuan Nguyen, Thanh-Nghia Truong, Hung Tuan Nguyen & Masaki Nakagawa
    • Organizer
      International Conference on Document Analysis and Recognition, ICDAR 2021
    • Related Report
      2021 Research-status Report
    • Int'l Joint Research
  • [Presentation] Context free grammar with tree structure analysis for online handwritten mathematical expression recognition2021

    • Author(s)
      Naruki Kitashima, Cuong Tuan Nguyen, Ikuko Shimizu, Masaki Nakagawa
    • Organizer
      IEICE Technical Report
    • Related Report
      2021 Research-status Report
  • [Presentation] Visual Constraints for Generating Multi-domain Offline Handwritten Mathematical Expressions2021

    • Author(s)
      Huy Quang Ung, Hung Tuan Nguyen, Cuong Tuan Nguyen, Masaki Nakagawa
    • Organizer
      IEICE Technical Report
    • Related Report
      2021 Research-status Report

URL: 

Published: 2021-04-28   Modified: 2023-12-25  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi