Systems and methods for enabling a highly available managed failover service

Abstract

a computing system that receives and stores configuration information for the application in a data store. The configuration information comprises (1) identifiers for a plurality of cells of the application that include at least a primary cell and a secondary cell, (2) a defined state for each of the plurality of cells, (3) one or more dependencies for the application, and (4) a failover workflow defining actions to take in a failover event. The computing system receives an indication, from a customer, of a change in state of the primary cell or a request to initiate the failover event. The computing system updates, in the data store, the states for corresponding cells of the plurality of cells based on the failover workflow and updates, in the data store, the one or more dependencies for the application based on the failover workflow.

What is claimed is: 
     
       1. A computing system comprising:
 a computer-readable memory storing executable instructions; and 
 one or more computer processors in communication with the computer-readable memory, wherein the one or more computer processors are configured to execute the executable instructions to at least receive configuration information comprising:
 identifiers for individual cells of a plurality of cells of an application distributed across a plurality of zones of a network-based system, the plurality of cells including at least a primary cell and a secondary cell, 
 a defined state for individual cells of the plurality of cells, 
 one or more dependencies for the application, and 
 a failover workflow defining actions to take in a failover event based on the defined state for individual cells and the one or more dependencies; 
 
 wherein the one or more computer processors are further configured to execute the executable instructions to at least:
 receive a request to initiate the failover event, 
 update the states for corresponding cells of the plurality of cells based on the failover workflow, and 
 update the one or more dependencies for the application based on the failover workflow. 
 
 
     
     
       2. The system of  claim 1 , wherein the one or more computer processors are further configured to execute the executable instructions to at least:
 identify that the primary cell changes state to a fenced state; and 
 route at least a portion of read traffic to the primary cell for at least a duration of time after the primary cell changes state to the fenced state. 
 
     
     
       3. The system of  claim 2 , wherein to identify that the primary cell changes state to the fenced state, the one or more computer processors are further configured to execute the executable instructions to at least identify that at least a quorum of data store partitions across the plurality of zones indicates that a current state value of the primary cell is not an active state. 
     
     
       4. The system of  claim 2 , wherein to route the at least a portion of read traffic to the primary cell for at least a duration of time after the primary cell changes state to the fenced state, the one or more computer processors are further configured to execute the executable instructions to at least provide routing instructions to at least one of a traffic management service or a Domain Name System (DNS) service provider. 
     
     
       5. The system of  claim 1 , wherein the configuration information defines the primary cell to have an active state and the secondary cell to have a standby state. 
     
     
       6. The system of  claim 5 , wherein to update the states for corresponding cells of the plurality of cells based on the failover workflow, the one or more computer processors are further configured to execute the executable instructions to at least update the primary cell to have a fenced state and update the secondary cell to have the active state. 
     
     
       7. The system of  claim 1 , wherein the one or more computer processors are further configured to execute the executable instructions to at least propagate the configuration information to a plurality of data stores distributed across the plurality of zones. 
     
     
       8. The system of  claim 1 , wherein to update the one or more dependencies for the application based on the failover workflow by the computing system, the one or more computer processors are further configured to execute the executable instructions to at least update traffic routing for the application between the primary cell and the secondary cell. 
     
     
       9. The system of  claim 1 , wherein the one or more computer processors are further configured to execute the executable instructions to at least propagate updates to the states for the corresponding cells of the plurality of cells to data stores of other computing systems hosted in at least one different zone from the computing system, and wherein the propagation of the updates to the states for the corresponding cells enables the computing system and the other computing systems to provide a highly available service for managing the states of the corresponding cells. 
     
     
       10. The system of  claim 9 , wherein the one or more computer processors are further configured to execute the executable instructions to at least identify a current state value for one of the primary cell and the secondary cell based on at least a quorum of the data stores, wherein the quorum of the data stores provides highly consistent data. 
     
     
       11. A method comprising:
 receiving configuration information for an application, the configuration information comprising:
 identifiers for individual cells of a plurality of cells of the application distributed across a plurality of zones, the plurality of cells including at least a primary cell and a secondary cell, 
 a defined state for individual cells of the plurality of cells, 
 one or more dependencies for the application, and 
 a failover workflow defining actions to take in a failover event based on the defined state for individual cells and based on the one or more dependencies; 
 
 receiving an indication of a change in state of the primary cell; 
 updating the states for corresponding cells of the plurality of cells based on the failover workflow; and 
 updating the one or more dependencies for the application based on the failover workflow. 
 
     
     
       12. The method of  claim 11 , wherein the configuration information defines the primary cell to have an active state and the secondary cell to have a standby state and wherein the failover workflow updates the primary cell to have a fenced state and updates the secondary cell to have the active state. 
     
     
       13. The method of  claim 11 , further comprising propagating the configuration information to a plurality of data stores distributed across the plurality of zones. 
     
     
       14. The method of  claim 11 , wherein updating the one or more dependencies comprises updating traffic routing for the application between the primary cell and the secondary cell. 
     
     
       15. The method of  claim 11 , further comprising:
 identifying that the primary cell changes state to a fenced state; and 
 routing at least a portion of read traffic to the primary cell for at least a duration of time after the primary cell changes state to the fenced state. 
 
     
     
       16. The method of  claim 15 , wherein identifying that the primary cell changes state to the fenced state comprises identifying that at least a quorum of data store partitions across the plurality of zones indicates that a current state value of the primary cell is not an active state. 
     
     
       17. The method of  claim 15 , wherein routing the at least a portion of read traffic to the primary cell for at least a duration of time after the primary cell changes state to the fenced state comprises routing instructions to at least one of a traffic management service or a Domain Name System (DNS) service provider. 
     
     
       18. The method of  claim 11 , further comprising propagating updates to the states for the corresponding cells of the plurality of cells to data stores of other computing systems hosted in at least one different zone from a computing system implementing the method. 
     
     
       19. The method of  claim 18 , wherein propagating of the updates to the states for the corresponding cells enables the computing system and the other computing systems to provide a highly available service for managing the states of the corresponding cells. 
     
     
       20. The method of  claim 18 , further comprising identifying a current state value for one of the primary cell or the secondary cell based on at least a quorum of the data stores, wherein the quorum of the data stores provides highly consistent data.