Dynamic person queue analytics

Abstract

In one embodiment, a device identifies, from image data captured by one or more cameras of a physical location, a focal point of interest and people located within the physical location. The device forms a set of nodes whereby a given node represents one or more of the identified people located within the physical location. The device represents a person queue as an ordered list of nodes from the set of nodes and adds a particular one of the set of nodes to the list based on the particular node being within a predefined distance to the focal point of interest. The device adds one or more nodes to the list based on the added node being within an angle and distance range trailing a forward direction associated with at least one node in the list. The device provides an indication of the person queue to an interface.

What is claimed is: 
     
       1. A method comprising:
 identifying, by a device and from image data captured by one or more cameras of a physical location, a focal point of interest and people located within the physical location; 
 forming, by the device, a set of nodes, wherein a given node represents one or more of the identified people located within the physical location; 
 representing, by the device, a person queue as an ordered list of nodes from the set of nodes; 
 adding, by the device, a particular one of the set of nodes to the list based on the particular node being within a predefined distance to the focal point of interest; 
 adding, by the device, one or more nodes to the list based on the added node being within an angle and distance range trailing a forward direction associated with at least one node in the list, wherein the angle and the distance range are determined using a multimodal recurrent neural network; 
 regressing, by the device, the angle and distance range by using temporal information as input to the multimodal recurrent neural network to identify a queue pattern of the person queue comprising at least one of: a fan out pattern for the person queue, a fan in pattern for the person queue, a person queue that is parallel to the person queue, or a zig zag pattern of the person queue; and 
 providing, by the device, an indication of the person queue and the queue pattern to an interface. 
 
     
     
       2. The method as in  claim 1 , wherein forming the set of nodes comprises:
 performing, by the device, person re-identification across the image data from non-overlapping cameras to identify a subset of the people that traveled together at a point in time; and 
 grouping, by the device, the subset of the people as a single node based on the subset of the people traveling together at a point in time. 
 
     
     
       3. The method as in  claim 1 , wherein identifying the focal point of interest and people located within the physical location from the captured image data comprises:
 locating, by the device, the people within the image data based in part on wireless network signals. 
 
     
     
       4. The method as in  claim 1 , further comprising:
 identifying, by the device, a second focal point of interest within the physical location; and 
 determining, by the device, a second person queue that begins at the second focal point of interest. 
 
     
     
       5. The method as in  claim 1 , further comprising:
 determining, by the device, the forward direction associated with the at least one node in the list by:
 identifying shoulders of the one or more people represented by the at least one node; and 
 forming a two dimensional (2D) representation of the one or more people based on the identified shoulders of the one or more people; and 
 tracking the 2D representation of the one or more people over time. 
 
 
     
     
       6. The method as in  claim 1 , further comprising:
 tracking, by the device, movement of the nodes over time; and 
 estimating, by the device, a wait time metric for the person queue based on the tracked movement of the nodes over time, wherein the indication of the person queue provided to the interface comprises the estimated wait time. 
 
     
     
       7. The method as in  claim 1 , wherein at least one of the nodes added to the list represents a non-human object. 
     
     
       8. An apparatus, comprising:
 one or more network interfaces to communicate with a network; 
 a processor coupled to the network interfaces and configured to execute one or more processes; and 
 a memory configured to store a process executable by the processor, the process when executed configured to:
 identify, from image data captured by one or more cameras of a physical location, a focal point of interest and people located within the physical location; 
 form a set of nodes, wherein a given node represents one or more of the identified people located within the physical location; 
 represent a person queue as an ordered list of nodes from the set of nodes; 
 add a particular one of the set of nodes to the list based on the particular node being within a predefined distance to the focal point of interest; 
 add one or more nodes from the set to the list based on the added node being within an angle and distance range trailing a forward direction associated with at least one node in the list, wherein the angle and the distance range are determined using a multimodal recurrent neural network; 
 regress the angle and distance range by using temporal information as input to the multimodal recurrent neural network to identify a queue pattern of the person queue comprising at least one of: a fan out pattern for the person queue, a fan in pattern for the person queue, a person queue that is parallel to the person queue, or a zig zag pattern of the person queue; and 
 provide an indication of the person queue and the queue pattern to an interface. 
 
 
     
     
       9. The apparatus as in  claim 8 , wherein the apparatus forms the set of nodes by:
 performing person re-identification across the image data from non-overlapping cameras to identify a subset of the people that traveled together at a point in time; and 
 grouping the subset of the people as a single node based on the subset of the people traveling together at a point in time. 
 
     
     
       10. The apparatus as in  claim 8 , wherein the apparatus identifies the focal point of interest and people located within the physical location from the captured image data by:
 locating, by the device, the people within the image data based in part on wireless network signals. 
 
     
     
       11. The apparatus as in  claim 8 , wherein the process when executed is further configured to:
 identify a second focal point of interest within the physical location; and 
 determine a second person queue that begins at the second focal point of interest. 
 
     
     
       12. The apparatus as in  claim 8 , wherein the process when executed is further configured to:
 determine the forward direction associated with the at least one node in the list by:
 identifying shoulders of the one or more people represented by the at least one node; and 
 forming a two dimensional (2D) representation of the one or more people based on the identified shoulders of the one or more people; and 
 tracking the 2D representation of the one or more people over time. 
 
 
     
     
       13. The apparatus as in  claim 8 , wherein the process when executed is further configured to:
 tracking, by the device, movement of the nodes over time; and 
 estimating, by the device, a wait time metric for the person queue based on the tracked movement of the nodes over time, wherein the indication of the person queue provided to the interface comprises the estimated wait time. 
 
     
     
       14. The apparatus as in  claim 8 , wherein at least one of the nodes added to the list represents a non-human object. 
     
     
       15. The apparatus as in  claim 8 , wherein the apparatus comprises a fog computing node in communication with the one or more cameras via the network. 
     
     
       16. A tangible, non-transitory, computer-readable media having software encoded thereon, the software when executed by a device cause the device to perform a process comprising:
 identifying, by the device and from image data captured by one or more cameras of a physical location, a focal point of interest and people located within the physical location; 
 forming, by the device, a set of nodes, wherein a given node represents one or more of the identified people located within the physical location; 
 representing, by the device, a person queue as an ordered list of nodes from the set of nodes; 
 adding, by the device, a particular one of the set of nodes to the list based on the particular node being within a predefined distance to the focal point of interest; 
 adding, by the device, one or more nodes to the list based on the added node being within an angle and distance range trailing a forward direction associated with at least one node in the list wherein the angle and the distance range are determined using a multimodal recurrent neural network; 
 regressing, by the device, the angle and distance range by using temporal information as input to the multimodal recurrent neural network to identify a queue pattern of the person queue comprising at least one of: a fan out pattern for the person queue, a fan in pattern for the person queue, a person queue that is parallel to the person queue, or a zig zag pattern of the person queue; and 
 providing, by the device, an indication of the person queue and the queue pattern to an interface.