A Study of Dynamic Audiovisual Integration based on High Speed Vision
| Project/Area Number |
16360205
|
|---|
| Research Category |
Grant-in-Aid for Scientific Research (B)
|
| Allocation Type | Single-year Grants |
|---|
| Section | 一般 |
|---|
| Research Field |
Measurement engineering
|
|---|
| Research Institution | HIROSHIMA UNIVERSITY |
|---|
Principal Investigator |
ISHII Idaku Hiroshima University, Graduate School Associate Of Engineering, Professor, 大学院・工学研究科, 助教授 (40282686)
|
|---|
| Co-Investigator(Kenkyū-buntansha) |
KANEKO Makoto Hiroshima University, Graduate School Associate Of Engineering, Professor, 大学院・工学研究科, 教授 (70224607)
HIGASHIMORI Mitsuru Hiroshima University, Graduate School Associate Of Engineering, Assistant, 大学院・工学研究科, 助手 (30346522)
|
|---|
| Project Period (FY) |
2004 – 2005
|
| Project Status |
Completed (Fiscal Year 2005)
|
| Budget Amount *help |
¥12,800,000 (Direct Cost: ¥12,800,000)
Fiscal Year 2005: ¥6,900,000 (Direct Cost: ¥6,900,000)
Fiscal Year 2004: ¥5,900,000 (Direct Cost: ¥5,900,000)
|
|---|
| Keywords | High Speed Vision / Visual Auditorization / Vibration Dynamics / Sound Synthesis / Audiovisual Integration / Pixelwise Source Detection / Doppler Shift / Source Measurement |
|---|
| Research Abstract |
In this research, we constructed a theory for dynamic audiovisual integration which can integrate both vision sensor information and auditory sensor information at signal level in human audible frequency domain (20〜20kHz) by matching sampling and signal processing rates of vision sensor and auditory sensor. We also showed the effectiveness of dynamic audiovisual integration based on the vibrated dynamics which is contained in both visual information and auditory information commonly.
Concretely, we studied the following issues ; A)VA (Visual Auditorizaition) signal processing, in which visual information is converted into sound information at real-time, B)Pixel-wise Sound Source Detection by extracting the common dynamics between visual information and auditory information, C)Sound source distance / velocity measurement based on the difference between sound speed and light speed. These issues were implemented on the high speed vision system whose frame rate is in kHz order, and we verified the fundamental function of dynamic audiovisual integration.
|
Report
(3 results)
Research Products
(38 results)
