Target localization utilizing wireless and camera sensor fusion

Abstract

According to some implementations, an estimate of a target's location can be calculated by correlating Wi-Fi and video location measurements. This spatio-temporal correlation combines the Wi-Fi and video measurements to determine an identity and location of an object. The accuracy of the video localization and the identity from the Wi-Fi network provide an accurate location of the Wi-Fi identified object.

What is claimed is: 
     
       1. A computer-implemented method comprising:
 obtaining, by a computer, image data associated with an object; 
 determining, by the computer, based on the image data, a first track associated with the object, wherein the first track is determined by localizing the object in the image data in two-dimensional space relative to a ground plane by identifying a pixel in an image of the image data where the object comes in contact with the ground plane and transforming the pixel coordinates through a ground plane homography to coordinates of a floor plan; 
 obtaining, by the computer, wireless signal data from a device, the device being associated with the object; 
 determining, by the computer, based on the wireless signal data, a second track associated with the device; 
 determining, by the computer, whether trajectories of the first track and the second track are correlated in time and space; and 
 identifying, by the computer, the object based on the correlation of the trajectories of the first track and the second track. 
 
     
     
       2. The computer-implemented method of  claim 1 , wherein the trajectories represent paths that the object takes through space over time. 
     
     
       3. The computer-implemented method of  claim 1 , wherein determining whether trajectories of the first track and the second track are correlated in time and space includes defining a similarity measure between the first track and the second track. 
     
     
       4. The computer-implemented method of  claim 3 , wherein defining the similarity measure between the first track and the second track includes utilizing at least one of L p  norms, time warping, longest common subsequence (LCSS), or deformable Markov as model templates. 
     
     
       5. The computer-implemented method of  claim 3 , wherein the similarity measure between the first track and the second track is updated at each time sample without having to store an entire history of the first track and the second track. 
     
     
       6. The computer-implemented method of  claim 1 , further comprising providing, by the computer, real-time indoor location tracking solutions based on the indentifying the object of the first track and the second track. 
     
     
       7. A system comprising:
 one or more processors; and 
 a non-transitory computer-readable medium including one or more sequences of instructions which, when executed by the one or more processors, cause the one or more processors to:
 obtain image data associated with an object; 
 determine, based on the image data, a first track associated with the object, wherein the first track is determined by localizing the object in the image data in two-dimensional space relative to a ground plane by identifying a pixel in an image of the image data where the object comes in contact with the ground plane and transforming the pixel coordinates through a ground plane homography to coordinates of a floor plan; 
 obtain wireless signal data from a device; 
 determine, based on the wireless signal data, a second track associated with the device; 
 determine whether trajectories of the first track and the second track are correlated in time and space; and 
 identify the object based on the correlation of the trajectories of the first track and the second track. 
 
 
     
     
       8. The system of  claim 7 , wherein the trajectories represent paths that the object takes through space over time. 
     
     
       9. The system of  claim 7 , wherein determining whether trajectories of the first track and the second track are correlated in time and space includes defining a similarity measure between the first track and the second track. 
     
     
       10. The system of  claim 9 , wherein defining the similarity measure between the first track and the second track includes utilizing at least one of L p  norms, time warping, longest common subsequence (LCSS), or deformable Markov as model templates. 
     
     
       11. The system of  claim 9 , wherein the similarity measure between the first track and the second track is updated at each time sample without having to store an entire history of the first track and the second track. 
     
     
       12. The system of  claim 7 , further comprising instructions to provide real-time indoor location tracking solutions based on the indentifying the object of the first track and the second track. 
     
     
       13. A non-transitory computer-readable medium including one or more sequences of instructions which, when executed by one or more processors, cause the one or more processors to:
 obtain image data associated with an object; 
 determine, based on the image data, a first track associated with the object, wherein the first track is determined by localizing the object in the image data in two-dimensional space relative to a ground plane by identifying a pixel in an image of the image data where the object comes in contact with the ground plane and transforming the pixel coordinates through a ground plane homography to coordinates of a floor plan; 
 obtaining wireless signal data from a device; 
 determine, based on the wireless signal data, a second track associated with the device; 
 determine whether trajectories of the first track and the second track are correlated in time and space; and 
 identify the object based on the correlation of the trajectories of the first track and the second track. 
 
     
     
       14. The non-transitory computer-readable medium of  claim 13 , wherein the trajectories represent paths that the object takes through space over time. 
     
     
       15. The non-transitory computer-readable medium of  claim 13 , wherein determining whether trajectories of the first track and the second track are correlated in time and space includes defining a similarity measure between the first track and the second track. 
     
     
       16. The non-transitory computer-readable medium of  claim 15 , wherein defining the similarity measure between the first track and the second track includes utilizing at least one of L p  norms, time warping, longest common subsequence (LCSS), or deformable Markov as model templates. 
     
     
       17. The non-transitory computer-readable medium of  claim 15 , wherein the similarity measure between the first track and the second track is updated at each time sample without having to store an entire history of the first track and the second track.