Smart Eyes: Attending and Recognizing Instances of Salient Events (SEARISE)

Funding Agency: CEC

Funding period: 2008-2011

Contact Person

Silvio P. Sabatini
Department of Biophysical and Electronic Engineering
University of Genoa
Via Opera Pia, 11a
I-16145 Genova, ITALY
tel:  (+39) 10 353 2092
fax:  (+39) 10 353 2777
e-mail: silvio.sabatini AT unige.it

Participants

Coordinator:
	     Fraunhofer Gesellschaft, Institure for Applied Information Technology			(DE)
Partners:   
             DIBE - University of Genoa			(DE)
             University of Ulm				(IT)
	     INRIA - Odyssee Lab			(FR)
	     University of Tuebingen			(DE)
	     TrackMen Ltd.				(DE)
             LTU Arena					(DE)

Abstract

The SEARISE project has been developing a trinocular active cognitive visual system, the Smart-Eyes, for detection, tracking and categorization of salient events and behaviours. The system will have human-like capability to learn from and self-adjust to ever changing visual input; fixate at salient events and follow their motion; perform visual categorization of salient events based on environmental context and a set of policy rules. The hardware part of Smart-Eyes consists of a fixed camera for global overview of the scenery complemented by two active stereo cameras, - the binocular cameras, which should automatically fixate at the salient events replicating the functionality of saccadic motions of human eyes. The system should perform multi-scale analysis by zooming individual parts of attended events, which might either uncover object's identity or display its salient actions in details. The software part of Smart-Eyes implements a computational cognitive model of visual processing replicating major principles and computational strategies found in the mammalian visual cortex. The Smart-Eyes system will naturally combine an engineering paradigm for coordinated eye-like movements of the binocular cameras with an innovative computational theory of visual cortex. The system is supposed to respond intelligently to events happening in its field of view by continuously switching its attention to those objects or object parts exhibiting most salient actions. Following the above concept SEARISE project pursues three major objectives: 1. Achieve comparable to human level of performance in identification, rough categorization, fixation and pursuit of salient events in complex videos for surveillance applications. 2. Develop the Smart Eyes prototype. 3. Prove the Smart Eyes functionality in real-life environment for observation the behaviour pattern of crowded gathering of people in a large distance scenario, and for monitoring of small groups of people including the behaviour of single individuals in a close range scenario. To achieve these objectives SEARISE has been developing a bunch of biologically-plausible algorithms modelling principal mechanisms of visual processing by the visual cortex.