• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Cooperative Understanding of Speeches and Images Using Multiple Recognizer and Its Application to Multimodal Dialogue System

Research Project

Project/Area Number 21500143
Research Category

Grant-in-Aid for Scientific Research (C)

Allocation TypeSingle-year Grants
Section一般
Research Field Intelligent informatics
Research InstitutionKyushu Institute of Technology

Principal Investigator

ENDO Tsutomu  九州工業大学, 大学院・情報工学研究院, 教授 (10112294)

Co-Investigator(Kenkyū-buntansha) SHIMADA Kazutaka  九州工業大学, 大学院情報工学研究院, 助教 (50346863)
Project Period (FY) 2009 – 2011
Project Status Completed (Fiscal Year 2011)
Budget Amount *help
¥2,730,000 (Direct Cost: ¥2,100,000、Indirect Cost: ¥630,000)
Fiscal Year 2011: ¥650,000 (Direct Cost: ¥500,000、Indirect Cost: ¥150,000)
Fiscal Year 2010: ¥780,000 (Direct Cost: ¥600,000、Indirect Cost: ¥180,000)
Fiscal Year 2009: ¥1,300,000 (Direct Cost: ¥1,000,000、Indirect Cost: ¥300,000)
Keywords自然言語処理 / マルチモーダルインタフェース / 音声理解 / ジェスチャ認識 / マルチモーダル / 文脈処理 / 複合認識器 / 人物識別 / 頭上画像 / マルチモーダルインターフェース / 情報統合 / 複数認識器
Research Abstract

We proposed a wide variety of methods to integrate several ap-proaches and features for multimodal dialogue systems. We developed a Web based image retrieval system using linguistic and image features first. We also realized a multiple speech recognizer with hierarchical relations. For hand posture recognition, we combined online and offline machine learning techniques. We introduced context features and top-view images to person identification.

Report

(4 results)
  • 2011 Annual Research Report   Final Research Report ( PDF )
  • 2010 Annual Research Report
  • 2009 Annual Research Report
  • Research Products

    (29 results)

All 2012 2011 2010 2009 Other

All Journal Article (2 results) (of which Peer Reviewed: 2 results) Presentation (24 results) Remarks (3 results)

  • [Journal Article] Recurrent Neural Network Classifier for Three Layer Conceptual Network and Performance2010

    • Author(s)
      Md. Khalilur Lhaman and Tsutomu Endo
    • Journal Title

      JOURNAL OF COMPUTERS

      Volume: Vol.5, No.1 Pages: 40-48

    • Related Report
      2011 Final Research Report
    • Peer Reviewed
  • [Journal Article] Recurrent Neural Network Classifier for Three Layer Conceptual Network and Performance2010

    • Author(s)
      Md.Khalilur Rhaman, Tsutomu Endo
    • Journal Title

      JOURNAL OF COMPUTERS Vol.5

      Pages: 40-48

    • Related Report
      2009 Annual Research Report
    • Peer Reviewed
  • [Presentation] 複数人談話における言語情報と非言語情報を利用した盛り上がり判定2012

    • Author(s)
      横山貴彦, 嶋田和孝, 遠藤勉
    • Organizer
      言語処理学会第18回年次大会(NLP2012)
    • Place of Presentation
      広島市立大学
    • Year and Date
      2012-03-14
    • Related Report
      2011 Final Research Report
  • [Presentation] 複数人談話における言語情報と非言語情報を利用した盛り上がり判定2012

    • Author(s)
      横山貴彦, 嶋田和孝, 遠藤勉
    • Organizer
      言語処理学会第18回年次大会(NLP2012)
    • Place of Presentation
      広島県・広島市立大学
    • Year and Date
      2012-03-14
    • Related Report
      2011 Annual Research Report
  • [Presentation] A Combined Method Based on SVM and Online Learning with HOG for Hand Shape Recognition2011

    • Author(s)
      Kazutaka Shimada, Ryosuke Muto and Tsutomu Endo
    • Organizer
      The 2nd Interna-tional Workshop on Advanced Computa-tional Intelligence and Intelligent Informatics(IWACIII2011)
    • Place of Presentation
      蘇州大学(中国)
    • Year and Date
      2011-11-21
    • Related Report
      2011 Final Research Report
  • [Presentation] A Person Identification Method Using a Top-view Head Image from an Overhead Camera2011

    • Author(s)
      Ryota Nakatani, Daichi Kouno, Kazutaka Shimada and Tsutomu Endo
    • Organizer
      The 2nd International Workshop on Advanced Computational Intelligence and Intel-ligent Informatics(IWACIII2011)
    • Place of Presentation
      蘇州大学(中国)
    • Year and Date
      2011-11-21
    • Related Report
      2011 Final Research Report
  • [Presentation] A Combined Method Based on SVM and Online Learning with HOG for Hand Shape Recognition2011

    • Author(s)
      Kazutaka Shimada, Ryosuke Muto, Tsutomu Endo
    • Organizer
      2nd International Workshop on Advanced Computational Intelligence and Intelligent Informatics (IWACIII2011)
    • Place of Presentation
      中国・蘇州
    • Year and Date
      2011-11-21
    • Related Report
      2011 Annual Research Report
  • [Presentation] A Person Identification Method Using a Top-view Head Image from an Overhead Camera2011

    • Author(s)
      Ryota Nakatani, Daichi Kouno, Kazutaka Shimada, Tsutomu Endo
    • Organizer
      2nd International Workshop on Advanced Computational Intelligence and Intelligent Informatics (IWACIII2011)
    • Place of Presentation
      中国・蘇州
    • Year and Date
      2011-11-21
    • Related Report
      2011 Annual Research Report
  • [Presentation] A person identification method using facial, clothing and time feature2011

    • Author(s)
      Kazuaki Komatsu, Kazutaka Shimada and Tsutomu Endo
    • Organizer
      The 2nd International Workshop on Advanced Computational Intelligence and Intelligent Informatics(IWACIII2011)
    • Place of Presentation
      蘇州大学(中国)
    • Year and Date
      2011-11-20
    • Related Report
      2011 Final Research Report
  • [Presentation] A person identification method using facial, clothing and time feature2011

    • Author(s)
      Kazuaki Komatsu, Kazutaka Shimada, Tsntomu Endo
    • Organizer
      2nd International Workshop on Advanced Computational Intelligence and Intelligent Informatics (IWACIII2011)
    • Place of Presentation
      中国・蘇州
    • Year and Date
      2011-11-20
    • Related Report
      2011 Annual Research Report
  • [Presentation] 深度情報を利用した頭上方向画像からの付属品情報の抽出2011

    • Author(s)
      香野大地, 中谷良太, 嶋田和孝, 遠藤勉
    • Organizer
      第19回電子情報通信学会九州支部学生会
    • Place of Presentation
      佐賀県・佐賀大学
    • Year and Date
      2011-09-28
    • Related Report
      2011 Annual Research Report
  • [Presentation] 頭上方向から撮影されたカメラ画像を利用した人物識別2011

    • Author(s)
      中谷良太, 香野大地, 嶋田和孝, 遠藤勉
    • Organizer
      第14回画像の認識理解シンポジウムMIRU2011
    • Place of Presentation
      石川県・金沢文化ホール
    • Year and Date
      2011-07-21
    • Related Report
      2011 Annual Research Report
  • [Presentation] コンテキスト情報を利用した人物識別の改良と機械学習の適用2011

    • Author(s)
      小松和朗, 嶋田和孝, 遠藤勉
    • Organizer
      第14回画像の認識理解シンポジウムMIRU2011
    • Place of Presentation
      石川県・金沢文化ホール
    • Year and Date
      2011-07-21
    • Related Report
      2011 Annual Research Report
  • [Presentation] 衣服情報を利用した人物識別における輝度勾配特徴量の適用2010

    • Author(s)
      小松和朗, 嶋田和孝, 遠藤勉
    • Organizer
      第18回電子情報通信学会九州支部学生会,D-44,2010
    • Place of Presentation
      福岡・福岡工業大学
    • Year and Date
      2010-09-24
    • Related Report
      2010 Annual Research Report
  • [Presentation] 頭上方向からのカメラ画像を用いた人物識別2010

    • Author(s)
      中谷良太, 香野大地, 嶋田和孝, 遠藤勉
    • Organizer
      第18回電子情報通信学会九州支部学生会,D-44,2010
    • Place of Presentation
      福岡・福岡工業大学
    • Year and Date
      2010-09-24
    • Related Report
      2010 Annual Research Report
  • [Presentation] 頭上方向からのカメラ画像を用いた付属品抽出2010

    • Author(s)
      香野大地, 中谷良太, 嶋田和孝, 遠藤勉
    • Organizer
      第18回電子情報通信学会九州支部学生会,D-44,2010
    • Place of Presentation
      福岡・福岡工業大学
    • Year and Date
      2010-09-24
    • Related Report
      2010 Annual Research Report
  • [Presentation] A Hierarchical Multiple Recognizer for Robust Speech Under-standing2010

    • Author(s)
      Takahiko Yokoyama, Kazutaka Shimada and Tsutomu Endo
    • Organizer
      The Pacific Rim International Conference on Artificial Intelligence(PRICAI 2010)
    • Place of Presentation
      ノボテルホテル大邱(韓国)
    • Year and Date
      2010-08-31
    • Related Report
      2011 Final Research Report
  • [Presentation] A Hierarchical Multiple Recognizer for Robust Speech Understanding2010

    • Author(s)
      Takahiko Yokoyama, Kazutaka Shimada, Tsutomu Endo
    • Organizer
      Proceedings of The Pacific Rim International Conference on Artificial Intelligence(PRICAI)2010
    • Place of Presentation
      韓国・大邱
    • Year and Date
      2010-08-31
    • Related Report
      2010 Annual Research Report
  • [Presentation] SVMと逐次学習を併用したHOG特徴による手形状認識手法2010

    • Author(s)
      武藤亮介, 嶋田和孝, 遠藤勉
    • Organizer
      電子情報通信学会, パターン認識・メディア理解研究会(PRMU)
    • Place of Presentation
      鹿児島大学
    • Year and Date
      2010-03-16
    • Related Report
      2009 Annual Research Report
  • [Presentation] 階層化された複数の音声認識器を選択的に利用する音声理解手法2010

    • Author(s)
      横山貴彦, 嶋田和孝, 遠藤勉
    • Organizer
      情報処理学会創立50周年記念第72回全国大会
    • Place of Presentation
      東京大学
    • Year and Date
      2010-03-10
    • Related Report
      2009 Annual Research Report
  • [Presentation] 複数の音声認識器とマーカを用いたマルチモーダルインターフェース2010

    • Author(s)
      武藤亮介, 嶋田和孝, 遠藤勉
    • Organizer
      情報処理学会創立50周年記念第72回全国大会
    • Place of Presentation
      東京大学
    • Year and Date
      2010-03-09
    • Related Report
      2009 Annual Research Report
  • [Presentation] Web image retrieval for abstract queries using text and image information2009

    • Author(s)
      Kazutaka Shimada, Suguru Ishikawa and Tsutomu Endo
    • Organizer
      The Fifth Asia Information Retrieval Symposium(AIRS 2009)
    • Place of Presentation
      北海道大学
    • Year and Date
      2009-10-22
    • Related Report
      2011 Final Research Report
  • [Presentation] Web image retrieval for abstract queries using text and image information2009

    • Author(s)
      Kazutaka Shimada, Suguru Ishikawa, Tsutomu Endo
    • Organizer
      Proceedings of The Fifth Asia Information Retrieval Symposium(AIRS 2009)
    • Place of Presentation
      北海道大学
    • Year and Date
      2009-10-22
    • Related Report
      2009 Annual Research Report
  • [Presentation] 複数の認識器を選択的に利用する音声理解手法のマルチモーダルインタフェースへの適用2009

    • Author(s)
      横山貴彦, 嶋田和孝, 遠藤勉
    • Organizer
      第17回電子情報通信学会九州支部学生会
    • Place of Presentation
      九州工業大学
    • Year and Date
      2009-09-29
    • Related Report
      2009 Annual Research Report
  • [Presentation] Effective construction and expansion of a sentiment corpus using an existing corpus and evaluative criteria estimation2009

    • Author(s)
      Ryosuke Tadano, Kazutaka Shimada and Tsutomu Endo
    • Organizer
      The 11th Confer-ence of the Pacific Association for Computational Linguistics(PACLING2009)
    • Place of Presentation
      北海道大学
    • Year and Date
      2009-09-03
    • Related Report
      2011 Final Research Report
  • [Presentation] Effective construction and expansion of a sentiment corpus using anexisting corpus and evaluative criteria estimation2009

    • Author(s)
      Ryosuke Tadano, Kazutaka Shimada, Tsutomu Endo
    • Organizer
      Proceedings of the 11th Conference of the Pacific Association for Computational Linguistics
    • Place of Presentation
      北海道大学
    • Year and Date
      2009-09-03
    • Related Report
      2009 Annual Research Report
  • [Remarks]

    • URL

      http://www.pluto.ai.kyutech.ac.jp/plt/endo-lab/index.html

    • Related Report
      2011 Final Research Report
  • [Remarks]

    • URL

      http://www.pluto.ai.kyutech.ac.jp/plt/endo-lab/index.html

    • Related Report
      2011 Annual Research Report
  • [Remarks]

    • URL

      http://www.pluto.ai.kyutech.ac.jp/plt/endo-lab/index.html

    • Related Report
      2010 Annual Research Report

URL: 

Published: 2009-04-01   Modified: 2016-04-21  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi