• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to previous page

Learning conversation agents for long-vision response generation in multi-round communications

Research Project

Project/Area Number 19K20345
Research Category

Grant-in-Aid for Early-Career Scientists

Allocation TypeMulti-year Fund
Review Section Basic Section 61030:Intelligent informatics-related
Research InstitutionThe University of Tokushima

Principal Investigator

KANG Xin  徳島大学, 大学院社会産業理工学研究部(理工学域), 助教 (80777350)

Project Period (FY) 2019-04-01 – 2022-03-31
Project Status Completed (Fiscal Year 2021)
Budget Amount *help
¥4,160,000 (Direct Cost: ¥3,200,000、Indirect Cost: ¥960,000)
Fiscal Year 2021: ¥1,040,000 (Direct Cost: ¥800,000、Indirect Cost: ¥240,000)
Fiscal Year 2020: ¥1,040,000 (Direct Cost: ¥800,000、Indirect Cost: ¥240,000)
Fiscal Year 2019: ¥2,080,000 (Direct Cost: ¥1,600,000、Indirect Cost: ¥480,000)
Keywords対話生成 / 対話完成程度 / 対話完成効率 / 対話ユーザー満足度 / long-vision conversation / dialogue topic tagging / neuro-symbolic AI / DialogQuality Evaluation / response generation / language ability / self-concept identifying / strategic conversation
Outline of Research at the Start

The purpose of this research plan is to learn conversation agents from scratch for generating long-vision responses. Three strategic communication games will be designed to provide an evolving environment for the conversation agents to progressively learn the conversation abilities through deep reinforcement learning. Specifically, the conversation agents will learn the language ability through Game I, the self-concept identification ability though Game II, and the strategic conversation ability through Game III, to finally make sequential decisions reasonably in a multi-round communication.

Outline of Final Research Achievements

Dialogue system has been an important research field in intelligent Human-Computer Interface (HCI) studies and has been widely applied as an intelligent service in smart products and self-driving systems. Traditional dialogue system consists of four independent modules, which are the utterance understanding module, the dialogue-state updating module, the dialogue policy managing module, and the utterance generating module. Such system is difficult to tune because independent modules cannot broadcast the learning error between each other and turns to accumulate prediction errors through the pipeline. In this project, the applicant proposed a topic knowledge driven end-to-end neural dialogue system. Specifically, the applicant proposed an automatic dialogue topic extraction method, an automatic dialogue quality evaluation method, and a generative adversarial network for high quality dialogue generation, and a knowledge-based nurse-patient dialogue system.

Academic Significance and Societal Importance of the Research Achievements

モジュール連結に基づく対話システムにより,本研究はモジュール機能をエンドツーエンド深層学習に統合して,話題知識抽出,対話分品質評価と敵対的生成学習を用いて,マルチターン対話のタスク完成程度,完成効率とユーザー満足度を向上し,対話を用いた知能ヒューマンマシンインターフェイス技術を構築する上に,スマート製品におけるユーザー体験や学習コストの改善や医療介護支援ロボット開発の推進を期待されている.

Report

(4 results)
  • 2021 Annual Research Report   Final Research Report ( PDF )
  • 2020 Research-status Report
  • 2019 Research-status Report
  • Research Products

    (49 results)

All 2022 2021 2020 2019

All Journal Article (6 results) (of which Int'l Joint Research: 2 results,  Peer Reviewed: 6 results,  Open Access: 2 results) Presentation (43 results) (of which Int'l Joint Research: 32 results)

  • [Journal Article] The AI Development Through Transforming Tacit Knowledge to Explicit Knowledge of Nurses' Dialogue for Patients with Dementia2021

    • Author(s)
      Hirokazu Ito, Kazuyuki Matsumoto, Xin Kang, Tetsuya Tanioka, Yuko Yasuhara, Rozzano De Castro Locsin and Fuji Ren
    • Journal Title

      International Journal of Advanced Intelligence (IJAI)

      Volume: 12 Pages: 11-21

    • Related Report
      2021 Annual Research Report 2020 Research-status Report
    • Peer Reviewed
  • [Journal Article] Active learning with complementary sampling for instructing class-biased multi-label text emotion classification2020

    • Author(s)
      Kang Xin、Shi Xuefeng、Wu Yunong、Ren Fuji
    • Journal Title

      IEEE Transactions on Affective Computing

      Volume: - Issue: 1 Pages: 1-1

    • DOI

      10.1109/taffc.2020.3038401

    • Related Report
      2020 Research-status Report
    • Peer Reviewed / Int'l Joint Research
  • [Journal Article] Vowel priority lip matching scheme and similarity evaluation model based on humanoid robot Ren-Xin2020

    • Author(s)
      Liu Zheng、Kang Xin、Nishide Shun、Ren Fuji
    • Journal Title

      Journal of Ambient Intelligence and Humanized Computing

      Volume: - Issue: 11 Pages: 1-12

    • DOI

      10.1007/s12652-020-02175-9

    • Related Report
      2020 Research-status Report
    • Peer Reviewed / Open Access
  • [Journal Article] DFF-ResNet: An insect pest recognition model based on residual networks2020

    • Author(s)
      Liu Wenjie、Wu Guoqing、Ren Fuji、Kang Xin
    • Journal Title

      Big Data Mining and Analytics

      Volume: 3 Issue: 4 Pages: 300-310

    • DOI

      10.26599/bdma.2020.9020021

    • Related Report
      2020 Research-status Report
    • Peer Reviewed / Open Access
  • [Journal Article] Surface defect detection of steel strips based on classification priority YOLOv3-dense network2020

    • Author(s)
      Zhang Jiaqiao、Kang Xin、Ni Hongjun、Ren Fuji
    • Journal Title

      Ironmaking & Steelmaking

      Volume: - Issue: 5 Pages: 1-12

    • DOI

      10.1080/03019233.2020.1816806

    • Related Report
      2020 Research-status Report
    • Peer Reviewed
  • [Journal Article] Toward action comprehension for searching: Mining actionable intents in query entities2019

    • Author(s)
      Kang Xin、Wu Yunong、Ren Fuji
    • Journal Title

      Journal of the Association for Information Science and Technology

      Volume: 71 Issue: 2 Pages: 143-157

    • DOI

      10.1002/asi.24220

    • Related Report
      2019 Research-status Report
    • Peer Reviewed / Int'l Joint Research
  • [Presentation] Creating a Japanese dialogue corpus with multi-level topic analysis2022

    • Author(s)
      Yuma Komoto, Xin Kang and Fuji Ren
    • Organizer
      2022 4th International Conference on Natural Language Processing
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Improvement of Japanese Text Emotion Analysis by Active Learning Using Transformers Language Model2022

    • Author(s)
      Tatsuya Ikegami, Xin Kang and Fuji Ren
    • Organizer
      2022 14th International Conference on Computer Research and Development (ICCRD)
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] TUA1 at the NTCIR-16 DialEval-2 Task2022

    • Author(s)
      Fei Ding, Xin Kang, Yunong Wu, Fuji Ren
    • Organizer
      Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Prediction and Generation of Multiple Complex Drawing Figures From Partial Drawing Sequences2021

    • Author(s)
      Kubono Yusuke, Shun Nishide, Xin Kang and Fuji Ren
    • Organizer
      2021 IEEE 13th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment, and Management (HNICEM)
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] Classification of Steel Strip Surface Defects Based on Optimized ResNet182021

    • Author(s)
      Hao Zhuangzhuang, Fuji Ren, Xin Kang, Ni Hongjun, Lv Shuaishuai and Wang Hui
    • Organizer
      2021 IEEE International Conference on Agents (ICA)
    • Related Report
      2021 Annual Research Report
    • Int'l Joint Research
  • [Presentation] TUA1 at the NTCIR-15 DialEval-1 Task2020

    • Author(s)
      Xin Kang, Yunong Wu and Fuji Ren
    • Organizer
      Proceedings of the 15th NTCIR Conference on Evaluation of Information Access Technologies
    • Related Report
      2020 Research-status Report
    • Int'l Joint Research
  • [Presentation] Gait recognition in real environment using gait energy image generated by Mask R-CNN2020

    • Author(s)
      Shota Inui, Fuji Ren, Shun Nishide and Xin Kang
    • Organizer
      2020 IEEE International Conference on Mechatronics and Automation (ICMA)
    • Related Report
      2020 Research-status Report
    • Int'l Joint Research
  • [Presentation] Construction of emergency medical support system based on understanding dialect intention2020

    • Author(s)
      Kyoya Abe, Fuji Ren, Shun Nishide and Xin Kang
    • Organizer
      The 15th International Conference on Natural Language Processing and Knowledge Engineering
    • Related Report
      2020 Research-status Report
    • Int'l Joint Research
  • [Presentation] A fusion model for multi-label emotion classification based on BERT and topic clustering2020

    • Author(s)
      Fei Ding, Xin Kang, Shun Nishide, Guan Zhijin and Fuji Ren
    • Organizer
      In International Symposium on Artificial Intelligence and Robotics 2020
    • Related Report
      2020 Research-status Report
    • Int'l Joint Research
  • [Presentation] Speech emotion recognition based on data enhancement in time-frequency domain2020

    • Author(s)
      LI Qianqian, Fuji Ren, Shen Xiaoyan and Xin Kang
    • Organizer
      International Symposium on Artificial Intelligence and Robotics 2020
    • Related Report
      2020 Research-status Report
    • Int'l Joint Research
  • [Presentation] Collaborative filtering recommendation algorithm based on bisecting K-means clustering2020

    • Author(s)
      Liu Jia, Xin Kang, Shun Nishide and Fuji Ren
    • Organizer
      International Symposium on Artificial Intelligence and Robotics 2020
    • Related Report
      2020 Research-status Report
    • Int'l Joint Research
  • [Presentation] Examination to develop the artificial intelligence through transforming tacit knowledge of nurses' dialogue for patients with dementia towards explicit knowledge2020

    • Author(s)
      Hirokazu Ito, Kazuyuki Matsumoto, Xin Kang, Tetsuya Tanioka, Yuko Yasuhara, Rozzano De Castro Locsin and Fuji Ren
    • Organizer
      The 15th International Conference on Natural Language Processing and Knowledge Engineering (NLP-KE20)
    • Related Report
      2020 Research-status Report
    • Int'l Joint Research
  • [Presentation] ロボット教師における特定分野のQAシステムの構築2020

    • Author(s)
      門田 尚之, 任 福継, 西出 俊, 康 シン
    • Organizer
      情報処理学会全国大会
    • Related Report
      2020 Research-status Report
  • [Presentation] ロボット教師における特定人物の音声の自動生成2020

    • Author(s)
      寺尾 渉吾, 任 福継, 西出 俊, 康 シン
    • Organizer
      情報処理学会全国大会
    • Related Report
      2020 Research-status Report
  • [Presentation] Tweet日本語感情コーパスに対するBERTの効果の検証2020

    • Author(s)
      池上 達也, 任 福継, 西出 俊, 康 シン
    • Organizer
      情報処理学会全国大会
    • Related Report
      2020 Research-status Report
  • [Presentation] Environment recognition using robot camera2020

    • Author(s)
      Hiroaki Sumida, Fuji Ren, Shun Nishide, and Xin Kang
    • Organizer
      The 5th IEEE International Conference on Big Data Analytics (IEEE ICBDA 2020)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] A Chinese-Japanese Vowel Priority Lip Matching Scheme Based on Humanoid Robot Ren-Xin2020

    • Author(s)
      Zheng Liu, Xin Kang, Shun Nishide and Fuji Ren
    • Organizer
      The 5th International Symposium on Artificial Intelligence and Robotics 2020 (ISAIR 2020)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] A Fusion Model for Multi-label Emotion Classification Based on BERT and Topic Clustering2020

    • Author(s)
      Fei Ding, Xin Kang, Shun Nishide, Zhijin Guan and Fuji Ren
    • Organizer
      The 5th International Symposium on Artificial Intelligence and Robotics 2020 (ISAIR 2020)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] 神経力学モデルを用いたプリミティブの階層的学習2020

    • Author(s)
      野本 楓晟, 安田 伊慶, 西出 俊, 康 シン
    • Organizer
      情報処理学会全国大会
    • Related Report
      2019 Research-status Report
  • [Presentation] リカレントニューラルネットワークを用いた未学習のコードからの楽譜生成2020

    • Author(s)
      真砂 光志, 西出 俊, 康 シン, 任 福継
    • Organizer
      情報処理学会全国大会
    • Related Report
      2019 Research-status Report
  • [Presentation] 深層モデルを用いた画像系列の特徴量自己組織化と映像生成2020

    • Author(s)
      川野 真子, 西出 俊, 康 シン, 任 福継
    • Organizer
      情報処理学会全国大会
    • Related Report
      2019 Research-status Report
  • [Presentation] ペンの持ち上げを考慮した神経力学モデルによる描画図形予測と生成2020

    • Author(s)
      久保野 雄亮, 西出 俊, 康 シン, 任 福継
    • Organizer
      情報処理学会全国大会
    • Related Report
      2019 Research-status Report
  • [Presentation] ロボット教師における特定分野のQAシステムの構築2020

    • Author(s)
      門田 尚之, 康 シン, 西出 俊, 任 福継
    • Organizer
      電子情報通信学会総合大会
    • Related Report
      2019 Research-status Report
  • [Presentation] ロボット教師における特定人物の音声の自動生成2020

    • Author(s)
      寺尾 渉吾, 康 シン, 西出 俊, 任 福継
    • Organizer
      電子情報通信学会総合大会
    • Related Report
      2019 Research-status Report
  • [Presentation] 単語の感情属性を活かしたロボット教師の表情生成2020

    • Author(s)
      房登 淳平, 康 シン, 西出 俊, 任 福継
    • Organizer
      電子情報通信学会総合大会
    • Related Report
      2019 Research-status Report
  • [Presentation] ロボット教師による特定分野の記述式問題の判定について2020

    • Author(s)
      山崎 直登, 康 シン, 西出 俊, 任 福継
    • Organizer
      電子情報通信学会総合大会
    • Related Report
      2019 Research-status Report
  • [Presentation] Research on the Effect of Different Speech Segment Lengths on Speech Emotion Recognition Based on LSTM2019

    • Author(s)
      Zheng Liu, Fuji Ren, and Xin Kang
    • Organizer
      The 9th International Workshop on Computer Science and Engineering (WCSE 2019)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] Traffic Sign Recognition Based on Up-sampling Convolution2019

    • Author(s)
      Yitian Lu, Ping Jiang, Shun Nishide, Xin Kang, and Fuji Ren
    • Organizer
      The 9th International Workshop on Computer Science and Engineering (WCSE 2019)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] Feature Fusion Based on Neural Image Captioning with Spatial Attention2019

    • Author(s)
      Qingqing Lu, Xiaomei Zhang, Xin Kang, and Fuji Ren
    • Organizer
      The 9th International Workshop on Computer Science and Engineering (WCSE 2019)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] Position-Free Hand Gesture Recognition Using Single Shot MultiBox Detector Based Neural Network2019

    • Author(s)
      Jingwei Tang, Xingtian Yao, Xin Kang, Shun Nishide, and Fuji Ren
    • Organizer
      2019 IEEE International Conference on Mechatronics and Automation (ICMA)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] Text Detection for Natural Scene based on MobileNet V2 and U-Net2019

    • Author(s)
      Fu, Kangwei, Ling Sun, Xin Kang, and Fuji Ren
    • Organizer
      2019 IEEE International Conference on Mechatronics and Automation (ICMA)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] Face and Gender Recognition System Based on Convolutional Neural networks2019

    • Author(s)
      Zhou, Yuxiang, Hongjun Ni, Fuji Ren, and Xin Kang
    • Organizer
      2019 IEEE International Conference on Mechatronics and Automation (ICMA)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] Building Label-Balanced Emotion Corpus Based On Active Learning for Text Emotion Classification2019

    • Author(s)
      Xuefeng Shi, Xin Kang, Ping Liao, and Fuji Ren
    • Organizer
      The 4th International Symposium on Artificial Intelligence and Robotics (ISAIR2019)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] Capture Important Information for Reading andReasoning by A Two-Pass Attention Mechanism in Dynamic Memory Network2019

    • Author(s)
      Zheng Zhou, Xin Kang, Shun Nishide, and Fuji Ren
    • Organizer
      The 6th IEEE International Conference on Cloud Computing and Intelligence Systems (CCIS2019)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] Text multi-label sentiment analysis based on Bi-LSTM2019

    • Author(s)
      Junlin Hu, Xin Kang, Shun Nishide, and Fuji Ren
    • Organizer
      The 6th IEEE International Conference on Cloud Computing and Intelligence Systems (CCIS2019)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] ExploringUncertain Samples through Active Learning To Enhance Text Emotion Classification2019

    • Author(s)
      Rongyu Dou, Nishide Shun, Fuji Ren, and Xin Kang
    • Organizer
      The 6th IEEE International Conference on Cloud Computing and Intelligence Systems (CCIS2019)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] Object detection based on SSD-ResNet2019

    • Author(s)
      Xin Lu, Xin Kang, Shun Nishide, and Fuji Ren
    • Organizer
      The 6th IEEE International Conference on Cloud Computing and Intelligence Systems (CCIS2019)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] Character action recognition based on deep convolutional neural network and action sequence2019

    • Author(s)
      Jiaen Li, Fuji Ren, Shun Nishide, and Xin Kang
    • Organizer
      The 6th IEEE International Conference on Cloud Computing and Intelligence Systems (CCIS2019)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] Constructing a Fundamental DevelopmentalDrawing Learning Model Using an Arm Robot2019

    • Author(s)
      Ryosuke Yasumura, Shun Nishide, Xin Kang, and Fuji Ren
    • Organizer
      The 6th IEEE International Conference on Cloud Computing and Intelligence Systems (CCIS2019)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] Investigating voice features for Speech emotion recognition based on four kinds of machine learning methods2019

    • Author(s)
      Haiyan Chen, Zheng Liu, Xin Kang, Shun Nishide, and Fuji Ren
    • Organizer
      The 6th IEEE International Conference on Cloud Computing and Intelligence Systems (CCIS2019)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] TUA1 at the NTCIR-14 STC-3 Task2019

    • Author(s)
      Zhou, Yangyang, Zheng Liu, Xin Kang, Yunong Wu, and Fuji Ren
    • Organizer
      The 14th NTCIR Conference
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] Tua1 at imageclef 2019 vqa-med: A classification and generation model based on transfer learning2019

    • Author(s)
      Zhou, Yangyang, Xin Kang, and Fuji Ren
    • Organizer
      Conference and Labs of the Evaluation Form (CLEF) 2019
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research
  • [Presentation] TUA1 at the TREC 2019: Deep Learning Track2019

    • Author(s)
      Yun Gao, Haitao Yu, Xin Kang, and Fuji Ren
    • Organizer
      The 28th Text Retrieval Conference (TREC 2019)
    • Related Report
      2019 Research-status Report
    • Int'l Joint Research

URL: 

Published: 2019-04-18   Modified: 2023-01-30  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi