2007 Fiscal Year Final Research Report Summary
A Motion-Analysis VLSI Image Sensor System Extracting the Meaning of Action From Moving Images
| Project/Area Number |
17206030
|
|---|
| Research Category |
Grant-in-Aid for Scientific Research (A)
|
| Allocation Type | Single-year Grants |
|---|
| Section | 一般 |
|---|
| Research Field |
Electronic materials/Electric materials
|
|---|
| Research Institution | The University of Tokyo |
|---|
Principal Investigator |
SHIBATA Tadashi The University of Tokyo, Graduate School of Engineering, Department of Electrical Engineering, Professor (00187402)
|
|---|
| Co-Investigator(Kenkyū-buntansha) |
MITA Yoshio Graduate School of Engineering, Department of Electrical Engineering, Associate Professor (40323472)
|
|---|
| Project Period (FY) |
2005 – 2007
|
| Keywords | brain processor / CMOS image sensor / motion field / directional edge detection / image recognition / gesture recognition / object tracking / ego-motion perception |
|---|
| Research Abstract |
Human perception is based on the automatic retrieval of past experience in the brain that is most relevant to the current event happening in front. Based on this postulate, a human-like intelligent VLSI system has been developed in our group mimicking the processing in the mind. The objective of this research is to further extend the capability for the system from the understanding of still images to the recognition of motions and actions in moving images. In understanding the motion, generating motion fields from moving images and representing them by feature vectors are of primary importance. However, they are computationally very demanding, and therefore, specialized VLSI hardware development is mandatory to achieve a real time performance of the system. An analog CMOS image sensor capable of detecting normal optical flow at 500fps has been developed employing a new time-domain hardware computation algorithm. A more accurate motion field generation based on the block matching algorithm has also been developed. By using the directional edge information and their histogram matching, a digital motion-field-generation processor has been built and evaluated by measurements. It shows more than 1000 times faster a performance than the software processing using 2.8GHzCPU at such a low clock frequency as 20MHz. Based on the motion fields thus generated, a time sequence of PPMD (Projected Principal Motion Distribution) vectors are formed and the action is recognized using Hidden Markov Models. Another motion representation called "Motion History Vector" has been developed by spatio-temporally integrating the PPMD vector sequence, which has been successfully applied to primitive gesture recognition by simple template matching. Application of the system to ego motion perception and object tracking are also demonstrated. (274 words)
|
