• Search Research Projects
  • Search Researchers
  • How to Use
  1. Back to project page

1996 Fiscal Year Final Research Report Summary

A study on multi-modal man-machine interface through spontaneous speech

Research Project

Project/Area Number 06452401
Research Category

Grant-in-Aid for Scientific Research (B)

Allocation TypeSingle-year Grants
Section一般
Research Field Intelligent informatics
Research InstitutionToyahashi University of Thechnology

Principal Investigator

NAKAGAWA Seiichi  Toyohashi University of Technology, Faculty of Engineering, Professor, 工学部, 教授 (20115893)

Co-Investigator(Kenkyū-buntansha) YAMAMOTO Mikio  University of Tsukuba Intitle of Information Sciences and Electronics, Assistant, 第3学群, 講師 (40210562)
MINEMATSU Nobuaki  Toyohashi University of Technology, Faculty of Engineering, Research Assistant, 工学部, 助手 (90273333)
Project Period (FY) 1994 – 1996
KeywordsSpoken dialog / speech recognition / natural language understanding / multi-model inteface / spontaneous speech
Research Abstract

We developed a malti-modal dialogue system that is composed of 4 parts : input by speech recognizer and touch screen, graphical user interface, natural language interpreter, and response generater.
Our speech recognizer intergrates the acoustic process with linguistic process directly without the phrase or word lattice. Furthermore, the recognizer processes interjections and restarts based on an unknown word processing technique.
A context free grammar is made to be able to accept sentences with omitted post-positions and inversion of word in order to recognize spontaneous speech.
Although our spontaneous speech recognizer outputs some errors caused by misrecognition (substitution errors), out of vacabulary (unknown words) and out of grammar (illegal utterances), the language interpreter can understand the meaning of errorful/illegal utterances.
The input by touch screen is used to designate the location of map on the display or to select the desired item form the menu which consists of the set of items responded by a speech synthesizer. We use boht the display output (map and menu) and speech systhesis for the response. User can use the postioning / selecting input and speech input at the same time. On man-machine communication, user wants to know his or machine situation what information he gets from the dialogue or how machine interprets / understands his utterances, as well as the speech recognition result. Therefore our system displays the history of dialogue. This function helps to eliminate the user uneasiness. Experimental evalution showed that our interpretation mechanism was suitable for understanding the recognition result of spontaneous speech. And we found that the multi-modal interface with spontaneous speech and touch screen was user-friendly.

  • Research Products

    (12 results)

All Other

All Publications (12 results)

  • [Publications] 山本幹雄: "メニューによりガイドされた文節単位による音声対話システム" 情報処理学会論文誌. 37・4. 461-469 (1996)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] 山本幹雄: "人間の理解手法を用いたロバストな音声対話システム" 情報処理学会論文誌. 37・4. 471-481 (1996)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] Min Zhou: "Succeding word prediction for speech recognition based on stochastic language model" Trans. IEICE Inf. & Syst.E79-D・4. 333-341 (1996)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] 中川聖一: "セグメント統計量を用いた隠れマルコフモデルによる音声認識" 電子情報通信学会論文誌. 79-DII(12). 2032-2038 (1996)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] 中川聖一: "音声対話システムの構成法とユーザ発話の関係" 電子情報通信学会論文誌. 79-DII・12. 2139-2145 (1996)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] 伊藤敏彦: "マルチモーダルインターフェースと協調的応答を備えた観光案内対話システムの評価" 情報処理学会シンポジウム、インタラクション '97. 135-142 (1997)

    • Description
      「研究成果報告書概要(和文)」より
  • [Publications] Atsuhiko Kai and Seiichi Nakagawa: "Investigation on unknown word processig and strategies for spontaneous speech understanding" Proc.4th European Conf.on Speech Communication and Thchnology. 2095-2098 (1995)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] Mikio Yamamoto, Mitsunori Takagi and Seiichi Nakagawa: "A menu-guided spoken dialog system by phrase utterance" Trans.Information Processig Society of Japan. Vol.37, No.4 (in Japanese). 461-469 (1996)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] Mikio Yamamoto, Toshihiko Itoh, Masaru Hidano and Seiichi Nakagawa: "A robust spoken dialogue system based on understanding mechanism of human being" Trans.Information Processig Society of Japan. Vol.37, No.4 (in Japanese). 471-481 (1996)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] Min.Zhou and Seiichi Nakagawa: "Succeeding word prediction for speech recogniton based on stochastic language model" Trans.IEICE Inf & Sys. Vol.E 79-D,No.4. 333-341 (1996)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] Akihiro Denda, Toshihiko Itoh and Seiichi Nakagawa: "A robust dialogue system with spontaneous speech and touch screen" Proc.Int.Conf.Multimodal Interface - 96'. 144-151 (1996)

    • Description
      「研究成果報告書概要(欧文)」より
  • [Publications] Seiichi Nakagawa and Seiji Yamamoto: "Relationship between the structure of spoken dialog systems and user utterances" Trans.IEICE. Vol.79-D II , No.12 (in Japanese). 2139-2145 (1996)

    • Description
      「研究成果報告書概要(欧文)」より

URL: 

Published: 1999-03-09  

Information User Guide FAQ News Terms of Use Attribution of KAKENHI

Powered by NII kakenhi