| Co-Investigator(Kenkyū-buntansha) |
TANAKA Kazuaki Kyushu Institute of Technology, Faculty of Computer Science and System Engineering, Assistant Professor, 情報工学部, 助手 (70253565)
TAKI Hirokazu Wakayama University, Faculty of System Engineering, Professor, システム工学部, 教授 (10304180)
|
|---|
| Research Abstract |
The following goals of the research are successfully achieved.
(1) The generating of virtual space in which the visual sensation is synchronized with the auditory and haptic sensation
(2) Synchronization of the action and utterance by a digital human (avatar) acting in virtual space, and the completion of a verbal / non-verbal communication system between a person and an avatar,
Regarding (1), in order to realize the real time collision detection between a virtual object moving at high speed and a static virtual object, the collision detection is implemented that uses the oct tree representation of a virtual object. The system is implemented that synchronizes the collision noise with the reaction force occurring with collision and that outputs the force to a force feedback device called PHANToM. As a concrete example, an experimental system that has the drawing, the sound occurred when a drum or cymbals are hit with a stick and haptic sensation synchronized is successfully realized. The e
… More
stimation is done for ten examinees using a questionnaire asking the degree of the synchronization among the three kinds of senses. As a result, there is the little difference between the position at which a drumstick hits a drum and the place where the reaction force is returned with a PHANToM, realization of a synchronization among the visual, auditory and haptic sense was confirmed.
(2) In a dialog between persons, it is understood that the expression to show the agreement or understanding for the utterance or gesture of a partner, and the modification of the utterance or behavior to cope with the interruption from a partner are indispensable to the mutual dialog promotion. Considering these facts, the following three components are introduced into the system: the simultaneous recognition of the utterance and behavior of a people, the simultaneous generation of action and utterance by an avatar, and the immediate processing mechanism against the interruption during the utterance or behavior. When a people wearing a magnetic sensor and data glove tells "put this and this here" to the avatar pointing the objects, the avatar can successfully nod looking at the object pointed and complete the task ordered.
Consequently the expected aim of the research was achieved. Less
|
