• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Development of a multimodal discussion map and a assessment support system based on discussion understanding

Research Project

Project/Area Number 17H01840
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeSingle-year Grants
Section一般
Research Field Learning support system
Research InstitutionKyushu Institute of Technology

Principal Investigator

Shimada Kazutaka  九州工業大学, 大学院情報工学研究院, 教授 (50346863)

Co-Investigator(Kenkyū-buntansha) 齊藤 剛史  九州工業大学, 大学院情報工学研究院, 准教授 (10379654)
Project Period (FY) 2017-04-01 – 2020-03-31
Project Status Completed (Fiscal Year 2019)
Budget Amount *help
¥15,600,000 (Direct Cost: ¥12,000,000、Indirect Cost: ¥3,600,000)
Fiscal Year 2019: ¥4,160,000 (Direct Cost: ¥3,200,000、Indirect Cost: ¥960,000)
Fiscal Year 2018: ¥5,980,000 (Direct Cost: ¥4,600,000、Indirect Cost: ¥1,380,000)
Fiscal Year 2017: ¥5,460,000 (Direct Cost: ¥4,200,000、Indirect Cost: ¥1,260,000)
Keywordsマルチモーダル / 言語処理 / 画像処理 / 自然言語処理
Outline of Final Research Achievements

In collaborative work, people need to discuss several topics for decision-making on a meeting, namely multi-party conversation. It is a very important task in intelligent systems to support consensus-building in conversations with multiple participants. The goal of this study is to construct a system that cooperatively supports consensus-building and management of conversation for high-quality discussion, namely Digital Facilitator.
We investigated the Digital Facilitator model from macro and micro perspectives. We introduced not only verbal features but also non-verbal features from video, namely multimodal features. In addition, we investigated other participants' behavior for predicting fruitless arguments in a conversation.

Academic Significance and Societal Importance of the Research Achievements

本研究では,コーパス(言語資源)の構築を進めてきた.これまでの自然言語処理の研究結果を踏まえ,各発話の役割を人手で割り当てるなどの作業をした.このコーパスは無償で利用できるように申請者のWebページで公開している.このコーパスは類似した研究を進める他の研究者にとっても役に立つであろう.
計算機による円滑な議論支援のために,実際の対話でリーダシップを取った参加者の行動を機械学習によってモデル化し,議論を通したリーダの振る舞いやリーダとして発言すべきタイミングの検出モデルなどを構築した.また言語的な情報だけではなく,音声や画像を利用したマルチモーダル情報理解をした点も大きな成果である.

Report

(4 results)
  • 2019 Annual Research Report   Final Research Report ( PDF )
  • 2018 Annual Research Report
  • 2017 Annual Research Report
  • Research Products

    (28 results)

All 2020 2019 2018 2017 Other

All Journal Article (1 results) (of which Peer Reviewed: 1 results,  Open Access: 1 results) Presentation (23 results) (of which Int'l Joint Research: 11 results) Remarks (3 results) Patent(Industrial Property Rights) (1 results)

  • [Journal Article] スポーツ要約生成におけるテンプレート型手法とニューラル型手法の提案と比較2018

    • Author(s)
      田川 裕輝 and 嶋田 和孝
    • Journal Title

      自然言語処理

      Volume: 25 Pages: 357-393

    • NAID

      130007531035

    • Related Report
      2018 Annual Research Report
    • Peer Reviewed / Open Access
  • [Presentation] 複数人対話における話題を蒸し返す話者の推定2020

    • Author(s)
      米満 駿甫, 嶋田 和孝
    • Organizer
      火の国シンポジウム2020, C5-3, 2020
    • Related Report
      2019 Annual Research Report
  • [Presentation] マルチモーダル情報を考慮した議論の取りまとめ役推定2020

    • Author(s)
      本多 幸希, 塩田 宰, 嶋田 和孝, 齊藤 剛史
    • Organizer
      電子情報通信学会 言語理解とコミュニケーション研究会, NLC2019-41, pp. 27-32, 2020
    • Related Report
      2019 Annual Research Report
  • [Presentation] 説得力分析に向けた複数人ディベートコーパスの構築2020

    • Author(s)
      塩田 宰, 嶋田 和孝
    • Organizer
      電子情報通信学会 言語理解とコミュニケーション研究会, NLC2019-36, pp. 1-6, 2020.
    • Related Report
      2019 Annual Research Report
  • [Presentation] Timing Prediction of Facilitating Utterance in Multi-party Conversation2019

    • Author(s)
      Tomonobu Sembokuya and Kazutaka Shimada
    • Organizer
      Proceedings of PACLING 2019
    • Related Report
      2019 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 複数人対話における取りまとめ発話のタイミング予測2019

    • Author(s)
      仙北谷 知将, 嶋田 和孝
    • Organizer
      火の国情報シンポジウム2019
    • Related Report
      2018 Annual Research Report
  • [Presentation] Tatara: 支援機能を持ったアノテーションツールの構築2019

    • Author(s)
      山村 崇, 嶋田 和孝, 吉川 和, 岩倉 友哉
    • Organizer
      言語処理学会第25回年次大会(NLP2019)
    • Related Report
      2018 Annual Research Report
  • [Presentation] Image-based Attention Level Estimation of Interaction Scene by Head Pose and Gaze Information2018

    • Author(s)
      Rinko Komiya, Takeshi Saitohy and Kazutaka Shimada
    • Organizer
      the 17th IEEE/ACIS International Conference on Computer and Information Science (ICIS 2018)
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Annotation and Analysis of Extractive Summaries for the Kyutech Corpus2018

    • Author(s)
      Takashi Yamamura and Kazutaka Shimada
    • Organizer
      the 11th edition of the Language Resources and Evaluation Conference
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Discussion map with an assistant function for decision-making: a tool supporting consensus-building2018

    • Author(s)
      Ryunosuke Kirikihira and Kazutaka Shimada
    • Organizer
      the tenth International Conference on Collaboration Technologies (CollabTech 2018)
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Analysis of facilitators' behaviors in multi-party conversations for constructing a digital facilitator system2018

    • Author(s)
      Tsukasa Shiota, Takashi Yamamura and Kazutaka Shimada
    • Organizer
      the tenth International Conference on Collaboration Technologies (CollabTech 2018)
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Dialogue Act Annotation and Identification in a Japanese Multi-party Conversation Corpus2018

    • Author(s)
      Takashi Yamamura, Masato Hino and Kazutaka Shimada
    • Organizer
      Asia-Pacific Corpus Linguistics Conferenc (APCLC2018)
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Construction of a Liar Corpus and Detection of Lying Situations2018

    • Author(s)
      Shohei Takabatake, Kazutaka Shimada and Takeshi Saitoh
    • Organizer
      SCIS-ISIS 2018
    • Related Report
      2018 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Kyutech コーパスにおけるアノテーションツールの試作2018

    • Author(s)
      山村 崇, 嶋田 和孝
    • Organizer
      情報処理学会 九州支部 若手の会セミナー
    • Related Report
      2018 Annual Research Report
  • [Presentation] Sarcasm detection Using Features Based on Indicator and Roles2018

    • Author(s)
      Satoshi Hiai and Kazutaka Shimada
    • Organizer
      the 3rd International Conference on Soft Computing and Data Mining 2018 (SCDM 2018)
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] 複数人議論における取りまとめ役の分類と特徴の分析2018

    • Author(s)
      塩田 宰, 山村 崇, 嶋田 和孝
    • Organizer
      言語処理学会第24回年次大会(NLP2018)
    • Related Report
      2017 Annual Research Report
  • [Presentation] 視線情報を用いた対話シーンにおける注目度推定2018

    • Author(s)
      小宮 凜子,齊藤 剛史,嶋田 和孝
    • Organizer
      火の国情報シンポジウム2018
    • Related Report
      2017 Annual Research Report
  • [Presentation] Simple and sophisticated inning summary generation based on encoder-decoder model and transfer learning2017

    • Author(s)
      Yuuki Tagawa and Kazutaka Shimada
    • Organizer
      the 21st International Conference on Asian Language Processing (IALP)
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Image-based Attention Estimation for Interaction Scene2017

    • Author(s)
      Rinko Komiya, Takeshi Saitoh, and Kazutaka Shimada
    • Organizer
      International Conference on Information and Communication Technology Robotics (ICT-ROBOT2017)
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Head Pose Estimation Using Convolutional Neural Network2017

    • Author(s)
      Seungsu Lee and Takeshi Saitoh
    • Organizer
      7th iCatse International Conference on IT Convergence and Security (ICITCS2017)
    • Related Report
      2017 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Kyutechコーパスを対象にした対話行為推定2017

    • Author(s)
      日野 優登, 山村 崇, 嶋田 和孝
    • Organizer
      第4回自然言語処理シンポジウム,
    • Related Report
      2017 Annual Research Report
  • [Presentation] 嘘つきコーパスの構築と分析2017

    • Author(s)
      高畠 匠平, 嶋田 和孝
    • Organizer
      HCGシンポジウム2017,
    • Related Report
      2017 Annual Research Report
  • [Presentation] 複数人対話における議論の取りまとめ役の推定2017

    • Author(s)
      塩田 宰, 山村 崇, 嶋田 和孝
    • Organizer
      第25回電子情報通信学会九州支部 学生会講演会
    • Related Report
      2017 Annual Research Report
  • [Presentation] 画像処理技術を用いた対話シーンにおける注目度推定2017

    • Author(s)
      小宮 凜子,齊藤 剛史,嶋田 和孝
    • Organizer
      第16回情報科学技術フォーラム(FIT2017)
    • Related Report
      2017 Annual Research Report
  • [Remarks] 九州工業大学・嶋田研究室・公開資源

    • URL

      http://www.pluto.ai.kyutech.ac.jp/~shimada/resources.html

    • Related Report
      2019 Annual Research Report
  • [Remarks] 嶋田研究室:ツールや資源

    • URL

      http://www.pluto.ai.kyutech.ac.jp/~shimada/resources.html

    • Related Report
      2018 Annual Research Report
  • [Remarks] Kyutechコーパス

    • URL

      http://www.pluto.ai.kyutech.ac.jp/~shimada/resources.html

    • Related Report
      2017 Annual Research Report
  • [Patent(Industrial Property Rights)] 読唇装置及び読唇方法2019

    • Inventor(s)
      齊藤剛史
    • Industrial Property Rights Holder
      齊藤剛史
    • Industrial Property Rights Type
      特許
    • Industrial Property Number
      2019-213234
    • Filing Date
      2019
    • Related Report
      2019 Annual Research Report

URL: 

Published: 2017-04-28   Modified: 2021-02-19  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi