2004 Fiscal Year Final Research Report Summary
Verbal/nonverbal bidirectional remote dialog system performed by amicable agent to man
| Project/Area Number |
14380183
|
|---|
| Research Category |
Grant-in-Aid for Scientific Research (B)
|
| Allocation Type | Single-year Grants |
|---|
| Section | 一般 |
|---|
| Research Field |
情報システム学(含情報図書館学)
|
|---|
| Research Institution | Kyusyu Institute of Technology |
|---|
Principal Investigator |
ABE Norihiro Kyusyu Institute of Technology, Faculty of Computer Science and System Engineering, Professor, 情報工学部, 教授 (00029571)
|
|---|
| Co-Investigator(Kenkyū-buntansha) |
YAGI Tetsuya Osaka University, Graduate School of Engineering, Professor, 大学院・工学研究科, 教授 (50183976)
TAKI Hirokazu Wakayama University, Faculty of System Engineering, Professor, システム工学部, 教授 (10304180)
TANAKA Kazuaki Kyusyu Institute of Technology, Faculty of Computer Science and System Engineering, Lecture, 情報工学部, 講師 (70253565)
|
|---|
| Project Period (FY) |
2002 – 2004
|
| Keywords | virtual environment / avatar / utterance and gesture recognition / pointing action / concurrent processing / silicon parallel vision chip / utterance and gesture generation / man machine interface |
|---|
| Research Abstract |
The aim of this research is to make it possible for users to use functions and contents machineries offer without suffering from operation of complicated machineries by talking verbal and non-verbal information including gesture/a hand gesture exchanged between persons as interface between a system and person instead of mouse and keyboard which are the current mainstream. When we are talking with other people, we usually share the dialog environment. And in such a case not only language but also gesture/hand gesture and facial expressions are utilized to convey contents which we want to express each other.
In this research, to realize a dialog in real time between a virtual agent (avatar) and a user in the virtual environment such as Internet shopping, parallel computation including linguistic and visual information processing, voice and graphic generation was realized using silicon parallel vision chips developed by Prof. Yagi of Osaka University and SCRAM NET making instant transmissi
… More
on of information between computers possible.
A user's hand can be captured with a silicon parallel vision chip as his hand stops at the moment when he is going to point his forefinger at a target object he wants to specify. Then it is possible to detect instantly whether a user is pointing his finger at the target object or not by only capturing the region including only a hand with a CCD camera.
On the other hand, in voice recognition the demonstrative pronoun such as 'that' or 'this' is detected at the first phase of voice recognition process, then the time when it is uttered is compared with that when the pointing action is conducted. By comparing them through SCRAM NET, synchronized recognition between utterance and action is attained and the object specified with a pointing action is identified. User's utterance can be analyzed by substituting the object name for the demonstrative pronoun in the uttered sentence.
The answer corresponding to user's demand is expressed using utterance and behavior by the avatar. The ambiguity in identification of a target object specified with a pointing action is solved by interrupt operation by a user ; when an avatar made a mistake in identifying the target object, the avatar's behavior the can be revised by user's operation which means interruption, prohibition or denial. Less
|
Research Products
(60 results)
