Discovering a computer network topology for an executing application

Abstract

There are disclosed devices, system and methods for continual automated discovering of a topology of a computer network of network objects for an application. A first network object selects portions of sets of data messages being sent over time by the application and related network objects; and collects network configuration and time dimension information, and timeseries information from the portions of messages. A second network object receives the information and uses it to determine topology information for the application over time, which includes unique keys of, types of objects of, types of relationships between pairs of, groupings of, time dimension data of and metrics data of the network objects. The topology information can be stored; and can be queried to create and display a graph representation of the topology information that changes over time.

It is claimed: 
     
       1. A method of mapping a network object topology of network objects of a computer communication network, the method comprising:
 selecting a remote network object that is part of a first macro-component of the network that includes a first set of the network objects, the network including a second macro-component that includes a second set of the network objects, the first macro-component communicating with the second macro-component; 
 gathering:
 flow log data from communication flow logs of the selected network object and at least one of the first set of network objects and at least one of the second set of network objects using a flow log source; 
 metric data from the selected network object and at least one of the first set of network objects and at least one of the second set of network objects using a metrics source; 
 configuration data from the selected network object and at least one of the first set of network objects and at least one of the second set of network objects using a configuration source; 
 
 generating configuration data and time data for each of the selected network object and the first set of network objects and at least one of the second set of network objects using the gathered flow log data, metric data and configuration data; 
 creating topology information that maps the first macro-component and the second macro-component using the configuration data and time data; and 
 storing the topology information in a memory. 
 
     
     
       2. The method of  claim 1 , wherein the topology information includes a unique key for the first macro-component and the second macro-component, types of macro-components of the first macro-component and the second macro-component, types of relationships between the first macro-component and the second macro-component, and time dimension data for the first macro-component and the second macro-component. 
     
     
       3. The method of  claim 1 , wherein each of the first macro-component and the second macro-component are one of a micro-service, a cluster subset or a service subset; and the topology information includes a communication relationship and type of communication between the first macro-component and the second macro-component based on the communication between the first macro-component and the second macro-component. 
     
     
       4. The method of  claim 1 , wherein the type of relationship includes the first macro-component controlling actions performed by the second macro-component, the first macro-component depending on the second macro-component in order to operate correctly or the first macro-component streaming data to the second macro-component. 
     
     
       5. The method of  claim 1 , wherein determining the topology information includes automatically grouping network objects having a common function into the first macro-component and the second macro-component; and wherein the automatically grouping includes creating a number of abstraction layers by interpreting low level relations between pairs of the network objects and collapsing functional groups of the network objects into higher level macro-components. 
     
     
       6. The method of  claim 1 , wherein the gathered flow log data, metric data and configuration data including network configuration information and time dimension (CTD) for the first macro-component and the second macro-component and timeseries (TS) information for the first macro-component and the second macro-component. 
     
     
       7. The method of  claim 1 , wherein:
 gathering and generating are performed at a first network object of a mapping system, the first network object coupled to the first and second sets of network objects; and 
 selecting, creating and storing are performed at a second network object of the mapping system, the second network object coupled to the first network object; 
 and further comprising: 
 the second network object communicating the selection of the remoted network object to the first network object; 
 the first network object communicating the gathered flow log, metric and configuration data to the second network object; and 
 the second network object feeding back the topology information to the first network object. 
 
     
     
       8. The method of  claim 1 , the topology information including:
 unique keys of the network objects, types of objects of the network objects, types of relationships between pairs of the network objects; 
 time dimension data for each of the network objects; 
 timeseries data of the selected network object and the related network objects; and 
 the time data including the time dimension data and the timeseries data of the selected network object and the related network objects. 
 
     
     
       9. The method of  claim 1 , further comprising:
 gathering the flow log, metric and configuration data over a period of time; and 
 changing the topology information over the period of time based on the flow log, metric and configuration data gathered over the period of time. 
 
     
     
       10. A system for mapping a network object topology of network objects of a computer communication network comprising:
 a first network object coupled to a first set and a second set of network objects, the first network object for:
 receiving a selection of a remote network object that is part of a first macro-component of the network that includes a first set of the network objects, the network including a second macro-component that includes a second set of the network objects, the first macro-component communicating with the second macro-component; 
 gathering:
 flow log data from communication flow logs of the selected network object and at least one of the first set of network objects and at least one of the second set of network objects using a flow log source; 
 metric data from the selected network object and at least one of the first set of network objects and at least one of the second set of network objects using a metrics source; 
 configuration data from the selected network object and at least one of the first set of network objects and at least one of the second set of network objects using a configuration source; 
 
 generating configuration data and time data for each of the selected network object and the first set of network objects and at least one of the second set of network objects using the gathered flow log data, metric data and configuration data; an 
 
 a second network object coupled to the first network object, the second network object for:
 creating topology information that maps the first macro-component and the second macro-component using the configuration data and time data; and 
 storing the first topology information in a memory. 
 
 
     
     
       11. The system of  claim 10 , wherein the topology information includes a unique key for the first macro-component and the second macro-component, types of macro-components of the first macro-component and the second macro-component, types of relationships between the first macro-component and the second macro-component, and time dimension data for the first macro-component and the second macro-component. 
     
     
       12. The system of  claim 10 , wherein each of the first macro-component and the second macro-component are one of a micro-service, a cluster subset or a service subset; and the topology information includes a communication relationship and type of communication between the first macro-component and the second macro-component based on the communication between the first macro-component and the second macro-component. 
     
     
       13. The system of  claim 10 , wherein the type of relationship includes the first macro-component controlling actions performed by the second macro-component, the first macro-component depending on the second macro-component in order to operate correctly or the first macro-component streaming data to the second macro-component. 
     
     
       14. The system of  claim 10 , wherein determining the topology information includes automatically grouping network objects having a common function into the first macro-component and the second macro-component; and wherein the automatically grouping includes creating a number of abstraction layers by interpreting low level relations between pairs of the network objects and collapsing functional groups of the network objects into higher level macro-components. 
     
     
       15. The system of  claim 10 , wherein the gathered flow log data, metric data and configuration data including network configuration information and time dimension (CTD) for the first macro-component and the second macro-component and timeseries (TS) information for the first macro-component and the second macro-component. 
     
     
       16. The system of  claim 10 , further comprising:
 the second network object communicating the selection of the remoted network object to the first network object; 
 the first network object communicating the gathered flow log, metric and configuration data to the second network object; and 
 the second network object feeding back the topology information to the first network object. 
 
     
     
       17. The system of  claim 1 , the topology information including:
 unique keys of the network objects, types of objects of the network objects, types of relationships between pairs of the network objects; 
 time dimension data for each of the network objects; 
 timeseries data of the selected network object and the related network objects; and 
 the time data including the time dimension data and the timeseries data of the selected network object and the related network objects. 
 
     
     
       18. The system of  claim 10 , further comprising:
 the first network object:
 gathering the flow log, metric and configuration data over a period of time; and 
 
 the second network object:
 changing the topology information over the period of time based on the flow log, metric and configuration data gathered over the period of time. 
 
 
     
     
       19. The system of  claim 10 , the system further comprising:
 a user input device 
 a display device 
 a processor 
 a memory 
 wherein the processor and the memory comprise circuits and software for performing the mapping the network object topology. 
 
     
     
       20. A non-volatile machine readable medium storing a program having instructions which when executed by a processor will cause the processor to map a network object topology of network objects of a computer communication network, the instructions of the program for:
 selecting a remote network object that is part of a first macro-component of the network that includes a first set of the network objects, the network including a second macro-component that includes a second set of the network objects, the first macro-component communicating with the second macro-component; 
 gathering:
 flow log data from communication flow logs of the selected network object and at least one of the first set of network objects and at least one of the second set of network objects using a flow log source; 
 metric data from the selected network object and at least one of the first set of network objects and at least one of the second set of network objects using a metrics source; 
 configuration data from the selected network object and at least one of the first set of network objects and at least one of the second set of network objects using a configuration source; 
 
 generating configuration data and time data for each of the selected network object and the first set of network objects and at least one of the second set of network objects using the gathered flow log data, metric data and configuration data; 
 creating topology information that maps the first macro-component and the second macro-component using the configuration data and time data; and 
 storing the topology information in a memory. 
 
     
     
       21. The medium of  claim 20 , further comprising:
 a user input device 
 a display device 
 a processor 
 a memory 
 wherein the processor and the memory comprise circuits and software for performing the instructions on the storage medium.