System and method for multi-agent event detection and recognition

Abstract

A method and system for creating a histogram of oriented occurrences (HO2) is disclosed. A plurality of entities in at least one image are detected and tracked. One of the plurality of entities is designated as a reference entity. A local 2-dimensional ground plane coordinate system centered on and oriented with respect to the reference entity is defined. The 2-dimensional ground plane is partitioned into a plurality of non-overlapping bins, the bins forming a histogram, a bin tracking a number of occurrences of an entity class. An occurrence of at least one other entity of the plurality of entities located in the at least one image may be associated with one of the plurality of non-overlapping bins. A number of occurrences of entities of at least one entity class in at least one bin may be into a vector to define an HO2 feature.

1 . A non-transitory computer-implemented method comprising:
 detecting and tracking one or more entities in one or more images;   designating one or more of the one or more entities as a reference entity;   defining a coordinate system oriented correspondingly with the reference entity;   partitioning a space defined by the coordinate system into a plurality of bins; and   counting, for each bin, a number of occurrences of an entity class.   
     
     
         2 . The method of  claim 1  wherein the coordinate system is defined as two or more dimensions for a ground plane and the partitioning the space partitions the ground plane. 
     
     
         3 . The method of  claim 1  wherein the counting creates a histogram of oriented occurrences (HO2) where occurrences comprise at least one or more attributes of motion of the entity. 
     
     
         4 . The method of  claim 1 , further comprising the step of associating an occurrence of at least one other entity of the plurality of entities located in the one or more images with one of the plurality of non-overlapping bins. 
     
     
         5 . The method of  claim 1  further comprising automatically detecting, based on the number of occurrences, one or more events associated with the reference entity. 
     
     
         6 . The method of  claim 1 , further comprising associating an occurrence of at least one other entity of the plurality of entities located in the at least one image with one of the plurality of non-overlapping bins. 
     
     
         7 . The method of  claim 2  wherein the partitioned ground plane moves when the reference entity moves. 
     
     
         8 . The method of  claim 1 , wherein the partitioning comprises employing a parts-based partition, wherein the parts-based partition measures a distance to the reference entity as the shortest distance to a point on the boundary of the reference entity and the bins are defined based on parts of the reference entity. 
     
     
         9 . The method of  claim 3 , further comprising loading a number of occurrences of entities of at least one entity class in at least one of the bins into a vector to define an HO2 feature. 
     
     
         10 . The method of  claim 9 , further comprising the steps of:
 computing HO2 features for an entity of interest from the plurality of entities over a sliding window of time to form a time sequence; and   clustering the time sequence using a clustering algorithm.   
     
     
         11 . A non-transitory computer-implemented method for recognizing an event, comprising:
 detecting and tracking a first set of entities in a first plurality of images;   geo-referencing the first set of entities to a defined coordinate system;   designating each member of the first set of entities as a positive or negative sample with respect to the event;   counting a number of occurrences for each member of the first set of entities within a related region of the coordinate system;   building a classifier for the event based on the occurrence counts for the positive and negative samples; and   using the classifier to determine whether the event occurs in a second plurality of images.   
     
     
         12 . The method of  claim 11  wherein the using the classifier further comprises:
 detecting and tracking a second set of entities in a second plurality of images; and 
 counting a number of occurrences for each member of the second set of entities within a related region of the coordinate system; and 
 applying the classifier for the event based on the occurrence counts for the second set of entities. 
 
     
     
         13 . The method of  claim 11  wherein the counting creates a histogram of oriented occurrences (HO2) feature for each member of the first set of entities, the building a classifier is based on the HO2 feature, and the occurrences comprise at least one or more attributes of motion of the entity. 
     
     
         14 . The method of  claim 13 , wherein the classifier is a support vector machine (SVM). 
     
     
         15 . The method of  claim 14 , wherein the SVM is built using the HO2 features of participants as positive samples and non-participants as negative samples: 
     
     
         16 . The method of  claim 15 , wherein the clustering algorithm comprises constructing a hierarchical cluster tree using χ2 distance and a using nearest neighbor strategy. 
     
     
         17 . The method of  claim 16 , wherein the distance between two clusters is the smallest distance between objects in the two clusters. 
     
     
         18 . The method of  claim 17 , wherein the clustering algorithm further comprises constructing clusters recursively from the root to leaves of the hierarchical cluster tree based on one of an inconsistence measure and the maximum number of clusters. 
     
     
         19 . The method of  claim 18 , further comprising loading a number of occurrences of entities of at least one entity class in at least one of the bins into a vector to define an HO2 features. 
     
     
         20 . The method of  claim 19 , further comprising the steps of:
 computing HO2 features for an entity of interest from the plurality of entities over a sliding window of time to form a time sequence; and   clustering the time sequence using a clustering algorithm.