Computer-vision based security system using a depth camera

Abstract

A method for securing an environment. The method includes obtaining a two-dimensional (2D) representation of a three-dimensional (3D) environment. The 2D representation includes a 2D frame of pixels encoding depth values of the 3D environment. The method further includes identifying a set of foreground pixels in the 2D representation, defining a foreground object based on the set of foreground pixels. The method also includes classifying the foreground object, and taking an action based on the classification of the foreground object.

1 .- 8 . (canceled) 
     
     
         9 . A method for securing an environment, comprising:
 receiving a two-dimensional (2D) representation of a three-dimensional (3D) environment, wherein the 2D representation is a 2D frame of pixels encoding depth values of the 3D environment, wherein the 2D representation comprises a foreground object, and wherein a background has been removed from the 2D representation;   classifying the foreground object; and   taking an action based on the classification of the foreground object.   
     
     
         10 . The method of  claim 9 , wherein classifying the foreground object comprises using a camera-specific classifier. 
     
     
         11 . The method of  claim 10 , further comprising:
 prior to classifying the foreground object using the camera-specific classifier, training the camera-specific classifier using data samples that are specific to a field of view of a camera with which the camera-specific classifier is associated and data samples that do not include the field of view of the camera.   
     
     
         12 . The method of  claim 9 , wherein classifying the foreground object comprises:
 associating the foreground object with a category; and   classifying the foreground object as one selected from a group consisting of a threat and a non-threat based, at least in part, on the category.   
     
     
         13 . The method of  claim 9 , wherein classifying the foreground object comprises:
 making a first determination, by a classifier, that the classification of the foreground object is unknown;   based on the first determination, sending a request to a human operator to classify the foreground object; and   receiving a classification of the object from the human operator.   
     
     
         14 . The method of  claim 13 , further comprising:
 updating the classifier based on the classification by the human operator.   
     
     
         15 . The method of  claim 9 , wherein classifying the foreground object comprises:
 sending at least the foreground object to a plurality of portable devices;   receiving a response from at least two of the plurality of portable devices; and   determining the classification of the foreground object based on the responses from the at least two of the plurality of portable devices.   
     
     
         16 . A method for securing an environment, comprising:
 receiving a two-dimensional (2D) representation of a three-dimensional (3D) environment, wherein the 2D representation is a 2D frame of pixels encoding depth values of the 3D environment;   identifying a plurality of foreground pixels in the 2D representation;   defining a foreground object based on the plurality foreground pixels;   classifying the foreground object; and   taking an action based on the classification of the foreground object.   
     
     
         17 .- 21 . (canceled) 
     
     
         22 . The method of  claim 9 , wherein classifying the foreground object is performed based on at least one feature selected from a group consisting of a geometry of a bounding box associated with the foreground object, a shape of the foreground object, and a motion descriptor of the foreground object. 
     
     
         23 . The method of  claim 9 , wherein classifying the foreground object is performed on a set of subsequent frames that comprise the foreground object. 
     
     
         24 . A non-transitory computer readable medium (CRM) comprising instructions that enable a system to:
 receive a two-dimensional (2D) representation of a three-dimensional (3D) environment, wherein the 2D representation is a 2D frame of pixels encoding depth values of the 3D environment, wherein the 2D representation comprises a foreground object, and wherein a background has been removed from the 2D representation;   classify the foreground object; and   take an action based on the classification of the foreground object.   
     
     
         25 . The non-transitory CRM of  claim 24 , wherein classifying the foreground object comprises using a camera-specific classifier. 
     
     
         26 . The non-transitory CRM of  claim 25 , further comprising instructions that enable the system to:
 prior to classifying the foreground object using the camera-specific classifier, train the camera-specific classifier using data samples that are specific to a field of view of a camera with which the camera-specific classifier is associated and data samples that do not include the field of view of the camera.   
     
     
         27 . The non-transitory CRM of  claim 24 , wherein the instructions for classifying the foreground object comprise functionality to:
 associate the foreground object with a category; and   classify the foreground object as one selected from a group consisting of a threat and a non-threat based, at least in part, on the category.   
     
     
         28 . The non-transitory CRM of  claim 24 , wherein the instructions for classifying the foreground object comprise functionality to:
 make a first determination, by a classifier, that the classification of the foreground object is unknown;   based on the first determination, send a request to a human operator to classify the foreground object; and   receive a classification of the object from the human operator.   
     
     
         29 . The non-transitory CRM of  claim 28 , further comprising instructions that enable the system to:
 update the classifier based on the classification by the human operator.   
     
     
         30 . The non-transitory CRM of  claim 24 , wherein the instructions for classifying the foreground object comprise functionality to:
 send at least the foreground object to a plurality of portable devices;   receive a response from at least two of the plurality of portable devices; and   determine the classification of the foreground object based on the responses from the at least two of the plurality of portable devices.   
     
     
         31 . The non-transitory CRM of  claim 24 , wherein classifying the foreground object is performed based on at least one feature selected from a group consisting of a geometry of a bounding box associated with the foreground object, a shape of the foreground object, and a motion descriptor of the foreground object. 
     
     
         32 . The non-transitory CRM of  claim 24 , wherein classifying the foreground object is performed on a set of subsequent frames that comprise the foreground object. 
     
     
         33 . A non-transitory computer readable medium comprising instructions that enable a system to:
 receive a two-dimensional (2D) representation of a three-dimensional (3D) environment, wherein the 2D representation is a 2D frame of pixels encoding depth values of the 3D environment;   identify a plurality of foreground pixels in the 2D representation;   define a foreground object based on the plurality foreground pixels;   classify the foreground object; and   take an action based on the classification of the foreground object.