1996 Fiscal Year Final Research Report Summary
A study on multi-modal man-machine interface through spontaneous speech
| Project/Area Number |
06452401
|
|---|
| Research Category |
Grant-in-Aid for Scientific Research (B)
|
| Allocation Type | Single-year Grants |
|---|
| Section | 一般 |
|---|
| Research Field |
Intelligent informatics
|
|---|
| Research Institution | Toyahashi University of Thechnology |
|---|
Principal Investigator |
NAKAGAWA Seiichi Toyohashi University of Technology, Faculty of Engineering, Professor, 工学部, 教授 (20115893)
|
|---|
| Co-Investigator(Kenkyū-buntansha) |
YAMAMOTO Mikio University of Tsukuba Intitle of Information Sciences and Electronics, Assistant, 第3学群, 講師 (40210562)
MINEMATSU Nobuaki Toyohashi University of Technology, Faculty of Engineering, Research Assistant, 工学部, 助手 (90273333)
|
|---|
| Project Period (FY) |
1994 – 1996
|
| Keywords | Spoken dialog / speech recognition / natural language understanding / multi-model inteface / spontaneous speech |
|---|
| Research Abstract |
We developed a malti-modal dialogue system that is composed of 4 parts : input by speech recognizer and touch screen, graphical user interface, natural language interpreter, and response generater.
Our speech recognizer intergrates the acoustic process with linguistic process directly without the phrase or word lattice. Furthermore, the recognizer processes interjections and restarts based on an unknown word processing technique.
A context free grammar is made to be able to accept sentences with omitted post-positions and inversion of word in order to recognize spontaneous speech.
Although our spontaneous speech recognizer outputs some errors caused by misrecognition (substitution errors), out of vacabulary (unknown words) and out of grammar (illegal utterances), the language interpreter can understand the meaning of errorful/illegal utterances.
The input by touch screen is used to designate the location of map on the display or to select the desired item form the menu which consists of the set of items responded by a speech synthesizer. We use boht the display output (map and menu) and speech systhesis for the response. User can use the postioning / selecting input and speech input at the same time. On man-machine communication, user wants to know his or machine situation what information he gets from the dialogue or how machine interprets / understands his utterances, as well as the speech recognition result. Therefore our system displays the history of dialogue. This function helps to eliminate the user uneasiness. Experimental evalution showed that our interpretation mechanism was suitable for understanding the recognition result of spontaneous speech. And we found that the multi-modal interface with spontaneous speech and touch screen was user-friendly.
|
