Visible-Light- and Time-of-Flight-Based Passive Tracking System

Abstract

A passive-tracking system is described herein. The system can include a visible-light sensor, a sound transducer, a thermal sensor, a time-of-flight (ToF) sensor, and a processor. The processor can receive visible-light frames from the visible-light sensor, sound frames from the sound transducer, thermal frames from the thermal sensor, and modulated-light frames from the ToF sensor. The processor, based on data of the visible-light and temperature frames, can also determine that an object is a living being and can provide an X and Y position of the object. The processor, based on data of the sound and positioning frames, can determine a Z position of the object. The X, Y, and Z positions may combine to form a three-dimensional (3D) position of the object. The processor can also passively track the object over time by selectively updating the 3D position of the object.

What is claimed is: 
     
         1 . A passive-tracking system for passively tracking a human, comprising:
 a visible-light sensor that is configured to generate a series of frames based at least in part on visible light that is reflected off the human;   a time-of-flight sensor that is configured to generate a series of frames based at least in part on light emitted by the time-of-flight sensor and reflected off the human; and   a processor that is communicatively coupled to the visible-light sensor and the time-of-flight sensor, wherein the processor is configured to:
 receive the frames from the visible-light sensor and from the time-of-flight sensor; 
 based at least in part on data of the frames from the visible-light sensor, provide an X position and a Y position of the human; 
 based at least in part on data of the frames from the time-of-flight sensor, provide a Z position of the human, wherein the X, Y, and Z positions combine to form a three-dimensional position of the human; and 
 passively track the human over time by selectively updating the three-dimensional position of the human. 
   
     
     
         2 . The passive-tracking system of  claim 1 , wherein the time-of-flight sensor is further configured to emit modulated light and wherein the light on which the frames are at least based in part on is modulated light emitted by the time-of-flight sensor and reflected off the human. 
     
     
         3 . The passive-tracking system of  claim 1 , wherein the processor is further configured to:
 determine the human is a human through identification of human-recognition features from the data of the frames from the visible-light sensor;   determine the human is a human through motion analysis of the data of the frames from the visible-light sensor; or   determine the human is a human through identification of human-recognition features from and motion analysis of the data of the frames from the visible-light sensor.   
     
     
         4 . The passive-tracking system of  claim 1 , wherein one of the frames from the visible-light sensor is a current frame and the processor is further configured to:
 compare the current frame to one or more reference frames; and   based on the comparison, identify one or more insignificant objects that do not warrant passive tracking.   
     
     
         5 . The passive-tracking system of  claim 4 , wherein the passive-tracking system is assigned to a monitoring area and wherein the processor is further configured to track the reference frame over time to account for changes in the monitoring area. 
     
     
         6 . The passive-tracking system of  claim 5 , wherein the processor is further configured to track the reference frame only when a novelty value for the monitoring area is below a predetermined activity threshold. 
     
     
         7 . The passive-tracking system of  claim 1 , wherein the processor is configured to provide the Z position of the human through a mapping of depth data from the data of the frames from the time-of-flight sensor against high-novelty blocks from the data of the frames of the visible-light sensor. 
     
     
         8 . The passive-tracking system of  claim 7 , wherein the processor is further configured to filter out or ignore low-novelty blocks from the data of the frames from the visible-light sensor as part of the mapping of the depth data against the high-novelty blocks. 
     
     
         9 . The passive-tracking system of  claim 1 , wherein the processor is further configured to identify one or more depth boundaries to distinguish different groups of blocks of the data of the frames from the visible-light sensor from one another. 
     
     
         10 . The passive-tracking system of  claim 1 , wherein the processor is further configured to:
 simultaneous to the passive tracking of the human, passively track a second human over time by selectively updating a three-dimensional position of the second human, wherein the human and the second human are positioned in a monitoring area; and   determine a passive count of the monitoring area that includes the human and the second human.   
     
     
         11 . A method of passively tracking a first human and a second human in a monitoring area, comprising:
 receiving frames that include data associated with visible light reflected off the first human and the second human in the monitoring area;   receiving frames that include data associated with emitted light reflected off the first human and the second human in the monitoring area;   based at least in part on the frames that include data associated with the reflected visible light, providing an X position and a Y position for both the first human and the second human;   based at least in part on the frames that include data associated with the reflected emitted light, providing a Z position for both the first human and the second human, wherein the Z position is a depth position;   combining the X, Y, and Z positions to provide a three-dimensional position for both the first human and the second human; and   passively tracking the first human and the second human by periodically updating the three-dimensional positions of the first human and the second human as the first human and the second human move in the monitoring area.   
     
     
         12 . The method of  claim 11 , wherein the X, Y, and Z positions of both the first human and the second human are adjusted X, Y, and Z positions converted from initial X, Y, and Z positions and wherein combining the X, Y, and Z positions to provide the three-dimensional position for both the first human and the second human comprises combining the adjusted X, Y, and Z positions to provide the three-dimensional position for both the first human and the second human. 
     
     
         13 . The method according to  claim 12 , wherein the adjusted X, Y, and Z positions are defined by adjusted X, Y, and Z axes, and the adjusted X, Y, and Z axes are defined by one or more reference sensors. 
     
     
         14 . The method of  claim 11 , further comprising identifying one or more depth boundaries associated with the first human and the second human to distinguish the first human from the second human. 
     
     
         15 . The method according to  claim 11 , further comprising:
 comparing the frames that include data associated with visible light reflected off the first human and the second human with one or more reference frames; and   tracking the reference frame to account for changing conditions in the monitoring area.   
     
     
         16 . The method according to  claim 11 , wherein the emitted light reflected off the first human and the second human has a wavelength that is within the near-infrared light range. 
     
     
         17 . A passive-tracking system for passively tracking multiple humans, comprising:
 a visible-light sensor that is configured to generate frames based at least in part on visible light that is reflected off a first human and a second human in a monitoring area at the same time;   a time-of-flight sensor that is configured to generate frames based at least in part on light emitted by the time-of-flight sensor and reflected off the first human and the second human in the monitoring area; and   a processor that is communicatively coupled to the visible-light sensor and the time-of-flight sensor, wherein the processor is configured to:
 receive the frames from the visible-light sensor and the frames from the time-of-flight sensor; 
 based on data of the frames from the visible-light sensor or data of the frames from the time-of-flight sensor, determine the first human and the second human are humans; 
 based on the data of the frames from the visible-light sensor, provide an X position and a Y position of both the first human and the second human; 
 based on the data of the frames from the time-of-flight sensor, provide a Z position of both the first human and the second human, wherein the X, Y, and Z positions provide a three-dimensional position of both the first human and the second human; and 
 passively track the first human and the second human in the monitoring area over time by periodically updating the three-dimensional positions of both the first human and the second human. 
   
     
     
         18 . The passive-tracking system of  claim 17 , wherein the light emitted by the time-of-flight sensor and reflected off the first human and the second human is modulated light. 
     
     
         19 . The passive-tracking system of  claim 17 , wherein the processor is further configured to determine the first human and the second human are humans by detecting human-recognition features from the data of the frames from the visible-light sensor. 
     
     
         20 . The passive-tracking system of  claim 17 , wherein the processor is further configured to:
 compare the data of the frames from the visible-light sensor to one or more reference frames to detect new objects in the frames from the visible-light sensor; and   track the reference frames over time to account for changes in the monitoring area.