• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Object Recognition by Deep Neural Network with Knowledge Graph Embedding - Proposal for Semantic Object Projection -

Research Project

Project/Area Number 17K00236
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeMulti-year Fund
Section一般
Research Field Perceptual information processing
Research InstitutionKobe University

Principal Investigator

ARIKI Yasuo  神戸大学, 都市安全研究センター, 名誉教授 (10135519)

Project Period (FY) 2017-04-01 – 2020-03-31
Project Status Completed (Fiscal Year 2019)
Budget Amount *help
¥4,550,000 (Direct Cost: ¥3,500,000、Indirect Cost: ¥1,050,000)
Fiscal Year 2019: ¥1,170,000 (Direct Cost: ¥900,000、Indirect Cost: ¥270,000)
Fiscal Year 2018: ¥1,560,000 (Direct Cost: ¥1,200,000、Indirect Cost: ¥360,000)
Fiscal Year 2017: ¥1,820,000 (Direct Cost: ¥1,400,000、Indirect Cost: ¥420,000)
Keywordsゼロショット学習 / 意味的特徴 / 画像特徴 / 知識グラフ / 深層学習 / 意味ベクトル / 対話システム / ImageNetデータセット / 見出し単語 / テキスト文書 / 意味表現 / パターン認識 / 一般物体認識 / シンボルグラウンディング
Outline of Final Research Achievements

In this study, zero-shot deep learning was investigated, which can learn both of image features and the related knowledges simultaneously, and then improve the recognition accuracy of known objects as well as unknown objects. As the results, knowledge graph was proven to be superior to words and texts in terms of the description ability and the knowledge graph of the complete Wordnet improved the recognition rate by 45% compared with state of the arts. In addition, as the application of zero-shot deep learning, we studied about dialogue systems which can incorporate external knowledge.

Academic Significance and Societal Importance of the Research Achievements

物体画像だけから特徴を抽出し認識するという方法には限界があり、「物体に関する知識も同時に学習させることにより、物体認識精度の向上、未知物体の識別と学習(ゼロショット学習)が可能になる」という考えに基づき、シンボル表現された知識と画像信号とを融合して学習を進める方法を明らかにした。また、この考えに基づき、発話文と知識を融合することで、知的な対話を行う方法を明らかにした。

Report

(4 results)
  • 2019 Annual Research Report   Final Research Report ( PDF )
  • 2018 Research-status Report
  • 2017 Research-status Report
  • Research Products

    (36 results)

All 2019 2018 2017 Other

All Journal Article (1 results) (of which Peer Reviewed: 1 results,  Open Access: 1 results) Presentation (34 results) (of which Int'l Joint Research: 14 results) Remarks (1 results)

  • [Journal Article] Semantic embeddings of generic objects for zero-shot learning2019

    • Author(s)
      Tristan Hascoet, Yasuo Ariki and Tetsuya Takiguchi
    • Journal Title

      EURASIP Journal on Image and Video Processing

      Volume: ー Issue: 1 Pages: 1-14

    • DOI

      10.1186/s13640-018-0371-x

    • NAID

      120006552895

    • Related Report
      2018 Research-status Report
    • Peer Reviewed / Open Access
  • [Presentation] On Zero-Shot Recognition of Genetic Objects2019

    • Author(s)
      Tristan Hascoet, Yasuo Ariki, Tetsuya Takiguchi
    • Organizer
      International Conference on Computer Vision and Pattern Recognition (CVPR)
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Generation of Objections Using Topic and Claim Information in Debate Dialogue System2019

    • Author(s)
      Kazuaki Furumai, Tetsuya Takiguchi, Yasuo Ariki
    • Organizer
      International Workshop on Spoken Dialog System Technology (IWSDS)
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] ロバストな対話システムのための発話スタイルに非依存なエンコード・デコード手法の提案2019

    • Author(s)
      古舞千暁, 滝口哲也, 有木康雄
    • Organizer
      第10回対話システムシンポジウム
    • Related Report
      2019 Annual Research Report
  • [Presentation] 発話スタイルに非依存な文章表現を用いたスタイルロバストなSeq2seqモデルの提案2019

    • Author(s)
      古舞千暁, 滝口哲也, 有木康雄
    • Organizer
      言語処理学会第26回年次大会発表論文集
    • Related Report
      2019 Annual Research Report
  • [Presentation] WordNetを用いた雑談対話システムの汎化性能の向上2019

    • Author(s)
      麻生大聖, 高島遼一, 滝口哲也, 有木康雄
    • Organizer
      電子情報通信学会技術研究報告
    • Related Report
      2019 Annual Research Report
  • [Presentation] 外部知識を用いた雑談対話システムの汎化性能向上の検討2019

    • Author(s)
      麻生大聖, 高島遼一, 滝口哲也, 有木康雄
    • Organizer
      日本音響学会2019年秋季研究発表会講演論文集
    • Related Report
      2019 Annual Research Report
  • [Presentation] On Zero-Shot Recognition of Genetic Objects2019

    • Author(s)
      Tristan Hascoet, Yasuo Ariki and Tetsuya Takiguchi
    • Organizer
      International Conference on Computer Vision and Pattern Recognition (CVPR)
    • Related Report
      2018 Research-status Report
    • Int'l Joint Research
  • [Presentation] Generation of Objections Using Topic and Claim Information in Debate Dialogue System2019

    • Author(s)
      Kazuaki Furumai, Tetsuya Takiguchi and Yasuo Ariki
    • Organizer
      International Workshop on Spoken Dialog System Technology (IWSDS)
    • Related Report
      2018 Research-status Report
    • Int'l Joint Research
  • [Presentation] ゼロショット学習を用いた一般物体セグメンテーション2019

    • Author(s)
      谷田 啓一, Tristan Hascoet, 滝口 哲也, 有木 康雄
    • Organizer
      情報処理学会第81回全国大会
    • Related Report
      2018 Research-status Report
  • [Presentation] 議論システムにおける言語モデルを用いた賛成/反対意見の自動生成手法の検討2019

    • Author(s)
      古舞 千暁, 滝口 哲也, 有木 康雄
    • Organizer
      日本音響学会2019年春季研究発表会
    • Related Report
      2018 Research-status Report
  • [Presentation] ユーザーの発話意図理解に基づくインタビュー発話の生成2019

    • Author(s)
      松好祐紀, 滝口哲也, 有木康雄, 瀧和男, 片桐恵子
    • Organizer
      言語処理学会第26回年次大会
    • Related Report
      2018 Research-status Report
  • [Presentation] Debate Dialog for News Question Answering System ‘NetTv’ -Debate Based on Claim and Reason Estimation-2018

    • Author(s)
      Rikito Marumoto, Katsuyuki Tanaka, Tetsuya Takiguchi, Yasuo Ariki
    • Organizer
      International Workshop on Spoken Dialog System Technology (IWSDS)
    • Related Report
      2018 Research-status Report
    • Int'l Joint Research
  • [Presentation] Chat Response Generation Based on Semantic Prediction Using Distributed Representations of Words2018

    • Author(s)
      Kazuaki Furumai, Tetsuya Takiguchi, Yasuo Ariki
    • Organizer
      International Workshop on Spoken Dialog System Technology (IWSDS)
    • Related Report
      2018 Research-status Report
    • Int'l Joint Research
  • [Presentation] User’s Intention Understanding in Question-Answering System Using Attention-based LSTM2018

    • Author(s)
      Yuki Matsuyoshi, Tetsuya Takiguchi, Yasuo Ariki
    • Organizer
      APSIPA
    • Related Report
      2018 Research-status Report
    • Int'l Joint Research
  • [Presentation] An investigation of semantic representations of visual classes for zero-shot learning2018

    • Author(s)
      Tristan Hascoet, Yasuo Ariki, Tetsuya Takiguchi
    • Organizer
      第21回画像の認識・理解シンポジウム
    • Related Report
      2018 Research-status Report
  • [Presentation] 議論システムにおける賛成/反対意見の生成手法の検討2018

    • Author(s)
      古舞 千暁, 有木 康雄, 滝口 哲也
    • Organizer
      人工知能学会 言語・音声理解と対話処理研究会
    • Related Report
      2018 Research-status Report
  • [Presentation] 議論システムにおける賛成/反対意見の生成のための発話のベクトル化手法の検討2018

    • Author(s)
      古舞 千暁,滝口 哲也, 有木 康雄
    • Organizer
      日本音響学会2018年秋季研究発表会
    • Related Report
      2018 Research-status Report
  • [Presentation] Attention-based LSTMを用いた意図理解とキーワード抽出の統合による質問応答システム2018

    • Author(s)
      松好祐紀, 滝口 哲也, 有木康雄
    • Organizer
      電子情報通信学会音声研究会
    • Related Report
      2018 Research-status Report
  • [Presentation] ユーザーの発話意図理解に基づくインタビュー発話の 生成に向けて2018

    • Author(s)
      松好祐紀, 滝口 哲也, 有木康雄
    • Organizer
      人工知能学会 言語・音声理解と対話処理研究会
    • Related Report
      2018 Research-status Report
  • [Presentation] Zero-shot learning using dictionary definitions2018

    • Author(s)
      Tristan Hascoet, Yasuo Ariki, Tetsuya Takiguchi
    • Organizer
      IWFCV-2018
    • Related Report
      2017 Research-status Report
    • Int'l Joint Research
  • [Presentation] Satellite Image Semantic Segmentation Using Fully Convolutional Network2018

    • Author(s)
      Atsushi Yoshihara, Tetsuya Takiguchi, Yasuo Ariki
    • Organizer
      IWFCV-2018
    • Related Report
      2017 Research-status Report
    • Int'l Joint Research
  • [Presentation] Estimation of Object Function Using Visual Attention2018

    • Author(s)
      Ryunosuke Azuma, Tetsuya Takiguchi, Yasuo Ariki
    • Organizer
      IWFCV-2018
    • Related Report
      2017 Research-status Report
    • Int'l Joint Research
  • [Presentation] Debate Dialog for News Question Answering System “NetTv”: Debate Based on Claim and Reason Estimation2018

    • Author(s)
      Rikito Marumoto, Katsuyuki Tanaka, Tetsuya Takiguchi, Yasuo Ariki
    • Organizer
      IWSDS2018
    • Related Report
      2017 Research-status Report
    • Int'l Joint Research
  • [Presentation] Chat Response Generation Based on Semantic Prediction Using Distributed Representations of Words2018

    • Author(s)
      Kazuaki Furumai, Tetsuya Takiguchi and Yasuo Ariki
    • Organizer
      IWSDS2018
    • Related Report
      2017 Research-status Report
    • Int'l Joint Research
  • [Presentation] Visual grounding of word embeddings: application to zero-shot image classification2018

    • Author(s)
      ハスコエト トリスタン, 有木 康雄, 滝口 哲也
    • Organizer
      情報処理学会、コンピュータビジョンとイメージメディア研究会(CVIM)
    • Related Report
      2017 Research-status Report
  • [Presentation] ニュース情報検索システム「NetTv」のための議論対話システムー賛否判定と根拠推定に基づく議論ー2018

    • Author(s)
      丸本 理貴人,田中 克幸,滝口 哲也,有木 康雄
    • Organizer
      日本音響学会2018年春季研究発表会
    • Related Report
      2017 Research-status Report
  • [Presentation] 単語の分散表現を用いた意味予測に基づく雑談応答生成2018

    • Author(s)
      古舞 千暁,有木 康雄,滝口 哲也
    • Organizer
      日本音響学会2018年春季研究発表会講演論文集
    • Related Report
      2017 Research-status Report
  • [Presentation] Attention-based LSTMを用いた音声質問応答システムにおけるユーザーの質問意図理解2018

    • Author(s)
      松好 祐紀,滝口 哲也,有木 康雄
    • Organizer
      日本音響学会2018年春季研究発表会講演論文集
    • Related Report
      2017 Research-status Report
  • [Presentation] Semantic Web and Zero-Shot Learning of Large Scale Visual Classes2017

    • Author(s)
      Tristan Hascoet, Yasuo Ariki, Tetsuya Takiguchi
    • Organizer
      First International Workshop on Symbolic-Neural Learning (SNL-2017)
    • Related Report
      2017 Research-status Report
    • Int'l Joint Research
  • [Presentation] Investigation of the correlations between CNN visual features and word embeddings2017

    • Author(s)
      Tristan Hascoet, Yasuo Ariki, Tetsuya Takiguchi
    • Organizer
      2nd Workshop on Closing the Loop Between Vision and Language, ICCV
    • Related Report
      2017 Research-status Report
    • Int'l Joint Research
  • [Presentation] 災害応急対策支援を目的とした衛星画像の被覆分類精度向上について2017

    • Author(s)
      吉原 篤, 滝口 哲也, 有木 康雄
    • Organizer
      第20回画像の認識・理解シンポジウム
    • Related Report
      2017 Research-status Report
  • [Presentation] ニュース情報検索システム「NetTv」における議論対話システム実現のためのユーザ主張・根拠の推定2017

    • Author(s)
      丸本理貴人,田中克幸, 滝口 哲也, 有木 康雄
    • Organizer
      人工知能学会 言語・音声理解と対話処理研究会
    • Related Report
      2017 Research-status Report
  • [Presentation] ユーザー支援を目的とした音声質問応答システム2017

    • Author(s)
      松好 祐紀,滝口 哲也,有木 康雄
    • Organizer
      日本音響学会2017年秋季研究発表会講演論文集
    • Related Report
      2017 Research-status Report
  • [Presentation] 人の理解や習熟をサポートする音声質問応答システム2017

    • Author(s)
      松好 祐紀, 滝口 哲也, 有木 康雄
    • Organizer
      人工知能学会 言語・音声理解と対話処理研究会
    • Related Report
      2017 Research-status Report
  • [Remarks] 神戸大学 システム情報学研究科 情報科学専攻 メディア情報講座

    • URL

      http://www.me.cs.scitec.kobe-u.ac.jp/~takigu/publications.html

    • Related Report
      2019 Annual Research Report 2018 Research-status Report 2017 Research-status Report

URL: 

Published: 2017-04-28   Modified: 2021-02-19  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi