Surveillance System and Method for Tracking and Identifying Objects in Environments

Abstract

A method and system tracks objects using a surveillance database storing events acquired by a set of sensors and sequences of images acquired by a set of cameras. Sequences of temporally and spatially adjacent events sensed by the set of sensors are linked to form a set of tracklets and stored in the database. Each tracklet has endpoints being either a track-start, track-join, tracklet-merge or tracklet-end node. A subset of sensors is selected, and a subset of tracklets associated with the subset of sensors is identified. A single starting tracklet is selected. All sequences of tracklets temporally and spatially adjacent to the starting tracklet are aggregated to construct a tracklet graph. The track-join nodes and the track-split nodes are disambiguated and eliminated from the track graph to determine a track of the object in the environment.

1 . A computer implemented method for tracking objects using a surveillance database, the surveillance database storing events acquired by a set of sensors and sequences of images acquired by a set of cameras, each event and image having an associated location and time, the method comprising the steps of:
 linking sequences of temporally and spatially adjacent events sensed by the set of sensors to form a set of tracklets, each tracklet beginning with a track-start node, a track-join node or a tracklet-split node and ending with a track-end node, the tracklet-join node or the tracklet-split node, the tracklet-join nodes occurring where multiple preceding tracklets merge to a single successor tracklet and the track-split nodes occurring where a single preceding tracklet diverges to multiple successor tracklets;   selecting a subset of sensors;   identifying a subset of tracklets associated with the subset of sensors   selecting a single tracklet from the subset of tracklet as a starting tracklet;   aggregating all tracklets temporally and spatially adjacent to the starting tracklet to construct a tracklet graph; and   disambiguating and eliminating the track-join nodes and the track-split nodes from the tracklet graph to determine a track of an object in the environment.   
     
     
         2 . The method of  claim 1 , in which the disambiguating further comprising:
 displaying available images temporally and spatially related to the events of the tracklet graph to identify the object.   
     
     
         3 . The method of  claim 1 , in which the sensors are infra-red motion sensors, and the cameras are movable. 
     
     
         4 . The method of  claim 1 , in which the sensors using wireless transmitters for transmitting the events. 
     
     
         5 . The method of  claim 1 , further comprising:
 retrieving the sequences of images only when events are detected by sensors in a view of a particular camera.   
     
     
         6 . The method of  claim 5 , further comprising:
 directing the particular camera at a general vicinity of the particular sensor when a particular event is sensed.   
     
     
         7 . The method of  claim 1 , in which the aggregating is performed according to temporal and spatial constraints. 
     
     
         8 . The method of  claim 8 , in which the temporal and spatial constraints are selected by a user. 
     
     
         9 . The method of  claim 8 , in which the temporal and spatial constraints are learned over time. 
     
     
         10 . The method of  claim 1 , further comprising:
 drawing the track on a floor plan of the environment.   
     
     
         11 . The method of  claim 1 , further comprising:
 associating particular sequences of images with the tracklets.   
     
     
         12 . The method of  claim 11 , further comprising:
 collecting the particular sequences of images associated with the track as video evidence related to the track and object.   
     
     
         13 . The method of  claim 1 , further comprising:
 identifying sensors with cameras at any given time.   
     
     
         14 . The method of  claim 1 , further comprising:
 identifying particular events visible in the sequences of images at any given time.   
     
     
         15 . The method of  claim 14 , further comprising:
 reducing the video evidence to only images corresponding to visible sensor activations.   
     
     
         16 . The method of  claim 1 , in which the linking step is performed periodically and the set of tracklets are pre-stored in the surveillance database. 
     
     
         17 . A system for tracking objects using a surveillance database, the surveillance database storing events acquired by a set of sensors and sequences of images acquired by a set of cameras, each event and image having an associated location and time, the system comprising:
 means for linking sequences of temporally and spatially adjacent events sensed by the set of sensors to form a set of tracklets, each tracklet beginning with a track-start node, a track-join node or a tracklet-split node and ending with a track-end node, the tracklet-join node or the tracklet-split node, the tracklet-join nodes occurring where multiple preceding tracklets merge to a single successor tracklet and the track-split nodes occurring where a single preceding tracklet diverges to multiple successor tracklets;   means for selecting a staring tracklet;   a user interface selecting a subset of sensors;   means for aggregating all tracklets temporally and spatially adjacent to the starting tracklet to construct a tracklet graph; and   means for disambiguating and eliminating the track-join nodes and the track-split nodes from the tracklet graph to determine a track of an object in the environment.   
     
     
         18 . The system of  claim 17 , in which the disambiguating further comprises:
 means for displaying available images temporally and spatially related to the events of the tracklet graph to identify the object.   
     
     
         19 . The system of  claim 18 , in which the sensors are infra-red motion sensors, and the cameras are movable.