| Project/Area Number |
14380183
|
|---|
| Research Category |
Grant-in-Aid for Scientific Research (B)
|
| Allocation Type | Single-year Grants |
|---|
| Section | 一般 |
|---|
| Research Field |
情報システム学(含情報図書館学)
|
|---|
| Research Institution | Kyusyu Institute of Technology |
|---|
Principal Investigator |
ABE Norihiro Kyusyu Institute of Technology, Faculty of Computer Science and System Engineering, Professor, 情報工学部, 教授 (00029571)
|
|---|
| Co-Investigator(Kenkyū-buntansha) |
YAGI Tetsuya Osaka University, Graduate School of Engineering, Professor, 大学院・工学研究科, 教授 (50183976)
TAKI Hirokazu Wakayama University, Faculty of System Engineering, Professor, システム工学部, 教授 (10304180)
TANAKA Kazuaki Kyusyu Institute of Technology, Faculty of Computer Science and System Engineering, Lecture, 情報工学部, 講師 (70253565)
|
|---|
| Project Period (FY) |
2002 – 2004
|
| Project Status |
Completed (Fiscal Year 2004)
|
| Budget Amount *help |
¥10,200,000 (Direct Cost: ¥10,200,000)
Fiscal Year 2004: ¥2,100,000 (Direct Cost: ¥2,100,000)
Fiscal Year 2003: ¥2,400,000 (Direct Cost: ¥2,400,000)
Fiscal Year 2002: ¥5,700,000 (Direct Cost: ¥5,700,000)
|
|---|
| Keywords | virtual environment / avatar / utterance and gesture recognition / pointing action / concurrent processing / silicon parallel vision chip / utterance and gesture generation / man machine interface / 言語・非言語動作 / 指さし動作確認 / 音声割り込み / 割り込み動作 / 立体視 / 仮想現実感 / 副詞対応動作の認識 / 再帰目標動作の実現 / 干渉検出 / Vortex / 音声割り込み処理 / 発話と行動の動機 / 手の形状解析 / 発話の解析 / 発話と動作の同期 / スクラムネット / 網膜チップカメラ / 相槌動作と音声の同期合成 |
|---|
| Research Abstract |
The aim of this research is to make it possible for users to use functions and contents machineries offer without suffering from operation of complicated machineries by talking verbal and non-verbal information including gesture/a hand gesture exchanged between persons as interface between a system and person instead of mouse and keyboard which are the current mainstream. When we are talking with other people, we usually share the dialog environment. And in such a case not only language but also gesture/hand gesture and facial expressions are utilized to convey contents which we want to express each other.
In this research, to realize a dialog in real time between a virtual agent (avatar) and a user in the virtual environment such as Internet shopping, parallel computation including linguistic and visual information processing, voice and graphic generation was realized using silicon parallel vision chips developed by Prof. Yagi of Osaka University and SCRAM NET making instant transmissi
… More
on of information between computers possible.
A user's hand can be captured with a silicon parallel vision chip as his hand stops at the moment when he is going to point his forefinger at a target object he wants to specify. Then it is possible to detect instantly whether a user is pointing his finger at the target object or not by only capturing the region including only a hand with a CCD camera.
On the other hand, in voice recognition the demonstrative pronoun such as 'that' or 'this' is detected at the first phase of voice recognition process, then the time when it is uttered is compared with that when the pointing action is conducted. By comparing them through SCRAM NET, synchronized recognition between utterance and action is attained and the object specified with a pointing action is identified. User's utterance can be analyzed by substituting the object name for the demonstrative pronoun in the uttered sentence.
The answer corresponding to user's demand is expressed using utterance and behavior by the avatar. The ambiguity in identification of a target object specified with a pointing action is solved by interrupt operation by a user ; when an avatar made a mistake in identifying the target object, the avatar's behavior the can be revised by user's operation which means interruption, prohibition or denial. Less
|
