Streaming Network Monitoring Caching Infrastructure

Abstract

Systems and methods of network telemetry caching and distribution are provided. The system can receive network telemetry data and store it as a plurality of data nodes. The system can maintain a node pointer map and a node pointer queue. If the system receives an update to a data node having a corresponding node pointer not already present in the node pointer map, the system can add the node pointer to the node pointer queue and to the node pointer map with a count of zero. If the node pointer is already present in the node pointer map, the system can increment the node count for the node pointer in the node pointer map and not add the node pointer to the node pointer queue. The system can transmit data values and node counts to the client device for each node pointer in the node pointer queue.

What is claimed is: 
     
         1 . A collection and translation system comprising:
 a collector configured to establish a connection with a network device and receive telemetry data from the network device;   a translator configured to translate vendor-specific notifications into vendor-neutral notifications;   a cache;   a node map capable of storing vendor-neutral data previously translated by the translator;   a configuration library comprising configuration files indicating which vendor-specific paths should map to which vendor-neutral paths;   a normalized data model; and   a type information database configured to be utilized by the translator to perform type conversion for network telemetry data from vendor-specific formats to a vendor-neutral format.   
     
     
         2 . The system of  claim 1 , wherein the network telemetry data includes a data value and a data path that identifies the data. 
     
     
         3 . The system of  claim 2 , wherein the collector receives network telemetry data from the network device in the form of streaming update packets, unpacks the packets, deserializes relevant data, and passes the data on to the translator. 
     
     
         4 . The system of  claim 2 , wherein the node map can store the vendor-neutral data in an in-memory cache. 
     
     
         5 . The system of  claim 3 , wherein the configuration files can be encapsulated into human-editable configuration files. 
     
     
         6 . The system of  claim 2 , wherein the normalized data model can include data models based on a vendor neutral format. 
     
     
         7 . The system of  claim 6 , wherein the vendor neutral format is OpenConfig. 
     
     
         8 . A computing system, comprising:
 one or more processing devices;   one or more memories storing instructions, the instructions causing the one or more processing devices to:
 establish a connection with a network device and receive telemetry data from a network device; 
 translate vendor-specific notifications into vendor-neutral notifications; 
 access a node map capable of storing vendor-neutral data previously translated by a translator; 
 access a configuration library comprising configuration files indicating which vendor-specific paths should map to which vendor-neutral paths; 
 access a normalized data model; and 
 perform type conversion for network telemetry data from vendor-specific formats to a vendor-neutral format using a type information database configured to be utilized by the translator. 
   
     
     
         9 . The system of  claim 8 , wherein the network telemetry data includes a data value and a data path that identifies the data. 
     
     
         10 . The system of  claim 9 , wherein the network telemetry data is received from the network device in the form of streaming update packets. 
     
     
         11 . The computing system of  claim 10 , wherein the instructions cause the one or more processing devices to unpack the packets, deserialize relevant data, and pass the data on to a translator. 
     
     
         12 . The system of  claim 8 , wherein the configuration files can be encapsulated into human-editable configuration files. 
     
     
         13 . The system of  claim 9 , wherein the node map can store the vendor-neutral data in an in-memory cache. 
     
     
         14 . The system of  claim 9 , wherein the normalized data model can include data models based on a vendor neutral format. 
     
     
         15 . The system of  claim 14 , wherein the vendor neutral format is OpenConfig.