Automated failover of a metadata node in a distributed file system

Abstract

Switching an active metadata node is disclosed. An indication that a standby metadata node of a distributed file system should replace an active metadata node of the distributed file system as a new active metadata node of the distributed file system is received. The standby metadata node is included in a server. A request that indicates that the standby metadata node would like to become an exclusive metadata node writer of a transaction log is sent. A confirmation that the standby metadata node is the exclusive metadata node writer of the transaction log is received. Based at least in part on the confirmation, an update that the standby metadata node has become the new active metadata node of the distributed file system is provided.

What is claimed is: 
     
       1. A system for switching an active metadata node, comprising:
 a processor configured to:
 receive an indication that a standby metadata node of a distributed file system should replace an active metadata node of the distributed file system as a new active metadata node of the distributed file system, wherein the new active metadata node of the distributed file system includes a NameNode of a HADOOP file system; 
 send a request that indicates that the standby metadata node requests to become an exclusive metadata node writer of a single shared transaction log, wherein in response to the request, the active metadata node is provided a notification that the active metadata node is no longer authorized to write to the transaction log and the single shared transaction log is shared at least between the active metadata node and the standby metadata node; 
 receive, before the standby metadata node has become the new active metadata node of the distributed file system, a confirmation that the standby metadata node is the exclusive metadata node writer of the single shared transaction log; and 
 in response to receiving the confirmation that the standby metadata node is the exclusive metadata node writer of the transaction log, provide an update that the standby metadata node has become the new active metadata node of the distributed file system, wherein the standby metadata node becomes the new active metadata node; and 
 
 a memory coupled to the processor and configured to provide the processor with instructions. 
 
     
     
       2. The system of  claim 1 , wherein the distributed file system stores user data of the distributed file system across a plurality of connected servers. 
     
     
       3. The system of  claim 1 , wherein the active metadata node manages metadata of the distributed file system. 
     
     
       4. The system of  claim 3 , wherein the metadata includes a namespace of the distributed file system. 
     
     
       5. The system of  claim 3 , wherein the metadata includes an identifier of a data node that stores a data block of a file of the distributed file system. 
     
     
       6. The system of  claim 1 , wherein the standby metadata node reads the transaction log to maintain an in-memory representation of metadata of the distributed file system prior to receiving the indication that the standby metadata node should replace the active metadata node of the distributed file system as the new active metadata node of the distributed file system. 
     
     
       7. The system of  claim 1 , wherein the transaction log records changes to metadata of the distributed file system. 
     
     
       8. The system of  claim 1 , wherein the indication is associated with an error of a component of the active metadata node. 
     
     
       9. The system of  claim 1 , wherein the indication is associated with maintenance of the active metadata node. 
     
     
       10. The system of  claim 1 , wherein the indication was sent by a coordination node that manages configuration of components of the distributed file system. 
     
     
       11. The system of  claim 10 , wherein the update is sent to the coordination node to update a record of the coordination node. 
     
     
       12. The system of  claim 1 , wherein the processor is further configured to receive a message requesting a status verification of the standby metadata node, and in the event the metadata does not provide a sufficient verification response, the standby metadata node is identified for repair or replacement. 
     
     
       13. The system of  claim 1 , wherein the active metadata node becomes a new standby metadata node. 
     
     
       14. The system of  claim 1 , wherein the distributed file system includes a plurality of standby metadata nodes. 
     
     
       15. The system of  claim 1 , wherein the confirmation has been sent based on a verification that the standby metadata node is to become the new active metadata node. 
     
     
       16. The system of  claim 1 , wherein the confirmation ensures that no other metadata node is able to commit changes to the distributed file system. 
     
     
       17. A method of switching an active metadata node, comprising:
 receiving an indication that a standby metadata node of a distributed file system should replace an active metadata node of the distributed file system as a new active metadata node of the distributed file system, wherein the standby metadata node is included in a server, wherein the new active metadata node of the distributed file system includes a NameNode of a HADOOP file system; 
 sending a request that indicates that the standby metadata node requests to become an exclusive metadata node writer of a single shared transaction log, wherein in response to the request, the active metadata node is provided a notification that the active metadata node is no longer authorized to write to the transaction log and the single shared transaction log is shared at least between the active metadata node and the standby metadata node; 
 receiving, before the standby metadata node has become the new active metadata node of the distributed file system, a confirmation that the standby metadata node is the exclusive metadata node writer of the single shared transaction log; and 
 in response to receiving the confirmation that the standby metadata node is the exclusive metadata node writer of the transaction log, providing an update that the standby metadata node has become the new active metadata node of the distributed file system, wherein the standby metadata node becomes the new active metadata node. 
 
     
     
       18. A computer program product for switching an active metadata node, the computer program product being embodied in a non-transitory tangible computer readable storage medium and comprising computer instructions for:
 receiving an indication that a standby metadata node of a distributed file system should replace an active metadata node of the distributed file system as a new active metadata node of the distributed file system, wherein the standby metadata node is included in a server, wherein the new active metadata node of the distributed file system includes a NameNode of a HADOOP file system; 
 sending a request that indicates that the standby metadata node requests to become an exclusive metadata node writer of a single shared transaction log, wherein in response to the request, the active metadata node is provided a notification that the active metadata node is no longer authorized to write to the transaction log and the single shared transaction log is shared at least between the active metadata node and the standby metadata node; 
 receiving, before the standby metadata node has become the new active metadata node of the distributed file system, a confirmation that the standby metadata node is the exclusive metadata node writer of the single shared transaction log; and 
 in response to receiving the confirmation that the standby metadata node is the exclusive metadata node writer of the transaction log, providing an update that the standby metadata node has become the new active metadata node of the distributed file system, wherein the standby metadata node becomes the new active metadata node. 
 
     
     
       19. The method of  claim 17 , wherein the standby metadata node reads the transaction log to maintain an in-memory representation of metadata of the distributed file system prior to receiving the indication that the standby metadata node should replace the active metadata node of the distributed file system as the new active metadata node of the distributed file system. 
     
     
       20. The method of  claim 17 , wherein the active metadata node becomes a new standby metadata node.