Secure and Private Tracking Across Multiple Cameras

Abstract

A technique for multi-camera object tracking is disclosed that preserves privacy of imagery from each camera or group of cameras. This technique uses secure multi-party computation to compute a distance metric across data from multiple cameras without revealing any information to operators of the cameras except whether or not an object was observed by both cameras. This is achieved by a distance metric learning technique that reduces the computing complexity of secure computation while maintaining object identification accuracy.

1 . A system comprising:
 one or more processing elements;   an object detection module communicatively coupled to or integrated with the one or more processing elements and configured to detect a first object when the first object is in view of a first video camera associated with a first site;   a feature vector module communicatively coupled to or integrated with the one or more processing elements and configured to extract a first feature vector from the first object;   a preprocessing module communicatively coupled to or integrated with the one or more processing elements and configured to perform local processing of the first feature vector by normalizing the first feature vector and quantizing the first feature vector into an integer value to form a first secure input;   a garbled circuit generator communicatively coupled to or integrated with the one or more processing elements and configured to generate a Boolean circuit for a jointly-computed portion of a vector distance function, wherein inputs to the Boolean circuit comprise the first secure input from the first site and a second secure input from a second site, the second secure input generated by processing a second feature vector generated from a second object that was in view of a second video camera associated with the second site;   a match query module communicatively coupled to or integrated with the one or more processing elements and configured to send a query to the second site to request participation in solving the jointly-computed portion of the vector distance function; and   a joint computation module communicatively coupled to or integrated with the one or more processing elements and configured to solve the jointly-computed portion of the vector distance function with the second site to determine if the first object is the same as the second object.   
     
     
         2 . The system of  claim 1 , wherein the first and second feature vectors comprise at least one of color histograms or texture histograms. 
     
     
         3 . The system of  claim 1 , wherein the Boolean circuit does not include multiplication circuits. 
     
     
         4 . The system of  claim 1 , further comprising a database communicatively coupled to or integrated with the one or more processing elements and configured to store the first feature vector for later comparison. 
     
     
         5 . The system of  claim 1 , further comprising an alert module communicatively coupled to or integrated with the one or more processing elements and configured to generate a real-time alert if the first object is the same as the second object. 
     
     
         6 . The system of  claim 5 , wherein at least one of the feature vector module, the preprocessing module, the garbled circuit generator, the match query module, or the joint computation module is implemented in a hardware logic component. 
     
     
         7 . One or more computer-readable media having computer-executable instructions which, when executed by one or more processing elements, cause a computing system to:
 obtain a first feature vector of a first object viewed by first video camera;   issue a secure match evaluation request to a peer computing system;   jointly compute at least a portion of a distance function over a first input derived from the first feature vector and a second input derived by the peer computing system from a second feature vector of a second object viewed by a second video camera, without disclosing either feature vector to another computing system; and   characterize, based at least in part on an output of the distance function, the first object and the second object as the same or different.   
     
     
         8 . The media of  claim 7 , wherein the distance function comprises determining a histogram intersection function for the first input and the second input. 
     
     
         9 . The media of  claim 7 , wherein the first input comprises a transformation of the first feature vector to a lower dimensionality vector. 
     
     
         10 . The media of  claim 9 , wherein the transformation comprises Principal Component Analysis (PCA) and linear mapping. 
     
     
         11 . The media of  claim 7 , wherein the computer-executable instructions further cause the computing system to issue the secure match evaluation request when the first object matches an object of interest, enters the view of the first video camera, or leaves the view of the first video camera. 
     
     
         12 . The media of  claim 7 , wherein the computer-executable instructions further cause the computing system to send, as part of jointly computing the distance function, a garbled version of a Boolean circuit of at least a portion of the distance function to the peer computing system and wait for the peer computing system to compute an output to the Boolean circuit. 
     
     
         13 . The media of  claim 12 , wherein the Boolean circuit is a computationally efficient circuit consisting of comparison, minimum, and addition circuits. 
     
     
         14 . A method for jointly computing a distance metric between two vectors using secure two-party computing, the method comprising:
 obtaining a first feature vector derived from a first private input;   solving a local computation portion of a distance function, the local computation portion computed within a privacy boundary of a first entity that provided the first private input to determine a first secure input;   solving, jointly with a second entity that has also independently solved a same local computation portion of the distance function for a second feature vector from a second private input to determine a second secure input, a joint computation portion of the distance function that takes the first secure input and the second secure input as inputs;   comparing a computed distance determined by the distance function to a threshold distance; and   when the computed distance is smaller than the threshold distance, categorizing the first private input as matching the second private input.   
     
     
         15 . The method of  claim 14 , wherein the first and second private inputs both comprise one or more of video, still images, images of human faces, handwritten text, or biometric features. 
     
     
         16 . The method of  claim 14 , wherein the distance function comprises a Metric learning by Collapsing Classes (MCC) distance function transformed to a different vector space by a linear projection matrix and having a Euclidean distance function replaced with a histogram intersection function. 
     
     
         17 . The method of  claim 14 , wherein the joint computation portion of the distance function is constructed as a garbled circuit. 
     
     
         18 . The method of  claim 17 , wherein the garbled circuit comprises a Boolean circuit comprising comparison, minimum, and addition circuits and excluding multiplication circuits. 
     
     
         19 . The method of  claim 14 , wherein the first secure input is quantized into an integer value. 
     
     
         20 . The method of  claim 14 , wherein the joint solving comprising a single process using a transmission control protocol (TCP) port opened by the second entity exclusively for comparisons against the second feature vector.