Three dimensional cocktail party effect processor using audio-visual sensor fusion technique

1999 Fiscal Year Final Research Report Summary

• Project/Area Number: 10650405
• Research Category: Grant-in-Aid for Scientific Research (C)
• Allocation Type: Single-year Grants
• Section: 一般
• Research Field: Measurement engineering
• Research Institution: The University of Electro-Communications
• Principal Investigator: TAKAHASHI Kota The University of Electro-Communications, Associate Professor, 電気通信学部, 助教授 (10188005)
• Co-Investigator(Kenkyū-buntansha): 高橋 弘太 電気通信大学, 電気通信学部, 助教授 (10188005)
• Project Period (FY): 1998 – 1999
• Keywords: image processing / parallel processing / FPGA

Research Abstract

A psychological term "cocktail party effect" means biological special ability to extract an important signal from other ambient noises. In this technological research, the cocktail party effect is focused on in order to construct a new type of sensing system. Three dimensional array of logic elements for sensor fusion, i.e., three dimensional sensor fusion system which integrate audio signals and sequential images on time-space coordinates, is proposed in the research. Summaries of the research are as follows.
A prototype system with a three dimensional array of 16 x 16 x 8 fusion cells has been constructed. Sixteen 60,000 gate FPGAs are used in the prototype system. Results of experiments using this system demonstrated the advantages of the concept of the new 3D system.
The BPF-ICA technique was also developed in this year, which was used as a preliminary step of sound source localizations. Using this algorithm, improvements in the source localization were demonstrated experimentally, especially in the case of 3D sound source location.
As the consequence of these results, the advantages of the concept of three dimensional logic elements for sensor fusion on time-space coordinates were shown.