Remediation of a system to new desired state using configuration dependency graph

Abstract

A method of remediating system services relies on a first dependency graph with super nodes, to each of which a set of system services are assigned, and second dependency graphs, each of which represents dependencies among the system services that are in one of the sets. The method includes the steps of: comparing an actual configuration state of the system services to a desired configuration state of the system services to identify configurations that are in drift; identifying system services corresponding to the configurations that are in drift; determining an order of updating configurations of the identified system services using the first and second dependency graphs; and applying the desired configurations of the identified system services in the order determined, wherein after the configurations of an identified system service are updated, the system service executes with the updated configurations.

What is claimed is: 
     
       1. A method of remediating system services to a desired configuration state, wherein the system services are running in a host configured with a virtualization software for supporting execution of virtual machines therein, and the system services include a plurality of sets of system services, said method comprising:
 comparing actual configurations of the system services to desired configurations of the system services to identify configurations that are in drift; 
 identifying system services corresponding to the configurations that are in drift including system services in a first set and system services in a second set; 
 retrieving a first dependency graph representing dependencies among the sets of system services, wherein, according to the first dependency graph, the first set is dependent on the second set; 
 retrieving a plurality of second dependency graphs, wherein each second dependency graph represents dependencies among system services that are in one of the sets of system services; 
 determining an order of updating the configurations that are in drift, using the first and second dependency graphs; and 
 applying desired configurations of the identified system services corresponding to the configurations that are in drift, in the order determined, wherein desired configurations of the system services in the first set are applied after desired configurations of the system services in the second set, and after the configurations that are in drift are updated to no longer be in drift, the identified system services corresponding to the configurations that are no longer in drift execute with the desired configurations of the identified system services corresponding to the configurations that are no longer in drift. 
 
     
     
       2. The method of  claim 1 , wherein the applying of the desired configurations of the identified system services includes:
 updating a configuration database in accordance with a master configuration schema defined for the system services based on a desired configuration file that contains the desired configurations of the system services, 
 wherein the configuration database is a key-value database and each of the system services has a corresponding key in the configuration database. 
 
     
     
       3. The method of  claim 2 , wherein during said comparing, the actual configurations of the system services are determined from the configuration database, and the desired configurations of the system services are determined from the desired configuration file. 
     
     
       4. The method of  claim 2 , wherein
 the first set includes storage system services, and the second set includes network system services. 
 
     
     
       5. The method of  claim 4 , wherein
 the first set includes a first system service that is dependent on a second system service as specified by the second dependency graph for the storage system services, and 
 during said applying, desired configurations for the first system service are applied after desired configurations for the second system service. 
 
     
     
       6. The method of  claim 5 , wherein the desired configuration file defines one or more configuration properties of the first system service, one or more internal states of the first system service, one or more configuration properties of the second system service, and one or more internal states of the second system service. 
     
     
       7. The method of  claim 1 , further comprising:
 generating a master configuration schema from a plurality of individual configuration schemas, each of the individual configuration schemas corresponding to one of the system services, 
 wherein the desired configurations of the identified system services corresponding to the configurations that are in drift, are applied in accordance with the master configuration schema. 
 
     
     
       8. A computing system comprising:
 a cluster of servers each configured with a virtualization software for supporting execution of virtual machines therein, wherein each of the servers includes a plurality of system services running therein, and each of the servers is configured to carry out a method of remediating system services to a desired configuration state, said method comprising: 
 comparing actual configurations of the system services to desired configurations of the system services to identify configurations that are in drift; 
 identifying system services corresponding to the configurations that are in drift including system services in a first set and system services in a second set; 
 retrieving a first dependency graph representing dependencies among sets of system services, wherein, according to the first dependency graph, the first set is dependent on the second set; 
 retrieving a plurality of second dependency graphs, wherein each second dependency graph represents dependencies among system services that are in one of the sets of system services; 
 determining an order of updating the configurations that are in drift, using the first and second dependency graphs; and 
 applying desired configurations of the identified system services corresponding to the configurations that are in drift, in the order determined, wherein desired configurations of the system services in the first set are applied after desired configurations of the system services in the second set, and after the configurations that are in drift are updated to no longer be in drift, the identified system services corresponding to the configurations that are no longer in drift execute with the desired configurations of the identified system services corresponding to the configurations that are no longer in drift. 
 
     
     
       9. The computing system of  claim 8 , wherein the applying of the desired configurations of the identified system services includes:
 updating a configuration database in accordance with a master configuration schema defined for the system services based on a desired configuration file that contains the desired configurations of the system services, 
 wherein the configuration database is a key-value database and each of the system services has a corresponding key in the configuration database. 
 
     
     
       10. The computing system of  claim 9 , wherein during said comparing, the actual configurations of the system services are determined from the configuration database, and the desired configurations of the system services are determined from the desired configuration file. 
     
     
       11. The computing system of  claim 9 , wherein
 the first set includes storage system services, and the second set includes network system services. 
 
     
     
       12. The computing system of  claim 11 , wherein
 the first set includes a first system service that is dependent on a second system service as specified by the second dependency graph for the storage system services, and 
 during said applying, desired configurations of the first system service are applied after desired configurations of the second system service. 
 
     
     
       13. The computing system of  claim 12 , wherein the desired configuration file defines one or more configuration properties of the first system service, one or more internal states of the first system service, one or more configuration properties of the second system service, and one or more internal states of the second system service. 
     
     
       14. The computing system of  claim 8 , further comprising:
 generating a master configuration schema from a plurality of individual configuration schemas, each of the individual configuration schemas corresponding to one of the system services, 
 wherein the desired configurations of the identified system services corresponding to the configurations that are in drift, are applied in accordance with the master configuration schema. 
 
     
     
       15. A non-transitory computer readable medium comprising instructions to be executed in one of a plurality of hosts that are each configured with a virtualization software for supporting execution of virtual machines therein, to carry out a method of remediating system services to a desired configuration state, wherein the system services are running in the hosts and include a plurality of sets of system services, said method comprising:
 comparing actual configurations of the system services to desired configurations of the system services to identify configurations that are in drift; 
 identifying system services corresponding to the configurations that are in drift including system services in a first set and system services in a second set; 
 retrieving a first dependency graph representing dependencies among the sets of system services, wherein, according to the first dependency graph, the first set is dependent on the second set; 
 retrieving a plurality of second dependency graphs, wherein each second dependency graph represents dependencies among system services that are in one of the sets of system services; 
 determining an order of updating the configurations that are in drift, using the first and second dependency graphs; and 
 applying desired configurations of the identified system services corresponding to the configurations that are in drift, in the order determined, wherein desired configurations of the system services in the first set are applied after desired configurations of the system services in the second set, and after the configurations that are in drift are updated to no longer be in drift, the identified system services corresponding to the configurations that are no longer in drift execute with the desired configurations of the identified system services corresponding to the configurations that are no longer in drift. 
 
     
     
       16. The non-transitory computer readable medium of  claim 15 , wherein the applying of the desired configurations of the identified system services includes:
 updating a configuration database in accordance with a master configuration schema defined for the system services, based on a desired configuration file that contains the desired configurations of the system services, 
 wherein the configuration database is a key-value database and each of the system services has a corresponding key in the configuration database. 
 
     
     
       17. The non-transitory computer readable medium of  claim 16 , wherein during said comparing, the actual configurations of the system services are determined from the configuration database, and the desired configurations of the system services are determined from the desired configuration file. 
     
     
       18. The non-transitory computer readable medium of  claim 16 , wherein
 the first set includes storage system services, and the second set includes network system services. 
 
     
     
       19. The non-transitory computer readable medium of  claim 18 , wherein
 the first set includes a first system service that is dependent on a second system service as specified by the second dependency graph for the storage system services, and 
 during said applying, desired configurations of the first system service are applied after desired configurations of the second system service. 
 
     
     
       20. The non-transitory computer readable medium of  claim 19 , wherein the desired configuration file defines one or more configuration properties of the first system service, one or more internal states of the first system service, one or more configuration properties of the second system service, and one or more internal states of the second system service.