Workload estimation of data resynchronization

Abstract

Embodiments include estimating a workload of a resynchronization task for a data component in a logical storage system. Embodiments include detecting a connection from the data component, wherein the data component was previously disconnected. Embodiments further include determining a layout of the data storage system. Embodiments further include estimating the workload of the resynchronization task based on the layout. If the layout indicates that the data component has an active mirror component, embodiments include estimating the workload to be a used capacity of the active mirror component. If the layout indicates that the data component does not have an active mirror component and that the data component has a stale mirror component, embodiments include estimating the workload to be a sum of a used capacity of the stale mirror component and an estimated size of missed input/output operations. One or more actions may be performed based on the estimate.

What is claimed is: 
     
       1. A method for estimating a workload of a resynchronization task for a data component in a logical storage system, comprising:
 detecting a connection from the data component, wherein the data component was previously disconnected; 
 determining a layout of the data storage system; 
 estimating the workload of the resynchronization task based on the layout by:
 if the layout indicates that the data component has an active mirror component:
 determining a used capacity of the active mirror component; and 
 estimating the workload to be the used capacity of the active mirror component; 
 
 if the layout indicates that the data component does not have an active mirror component and that the data component has a stale mirror component:
 determining a used capacity of the stale mirror component; 
 determining an estimated size of missed input and output operations at the stale mirror component; and 
 estimating the workload to be the sum of the used capacity of the stale mirror component and the estimated size of missed input and output operations; and 
 
 
 performing one or more actions based on the workload estimate, wherein the one or more actions comprise one or more of the following:
 scheduling the re-synchronization task based on the workload estimate; 
 determining whether to perform the re-synchronization task or to choose a different process for restoring the data component based on the workload estimate; and 
 reducing an amount of resources allocated to another task in the logical storage system based on the workload estimate. 
 
 
     
     
       2. The method of  claim 1 , wherein estimating the workload of the resynchronization task based on the layout further comprises:
 if the layout indicates that the data component does not have either an active or stale mirror component:
 determining a used capacity of each of one or more qualified sibling nodes of the data component in the logical storage system, wherein a sibling node is qualified if it is active and has full data availability; and 
 estimating the workload to be the average of the used capacities of the one or more qualified sibling nodes. 
 
 
     
     
       3. The method of  claim 2 , wherein determining the used capacity of each of the one or more qualified sibling nodes of the data component in the logical storage system comprises:
 for each qualified sibling node of the one or more qualified sibling nodes that is not a leaf node:
 determining a raid node type of the qualified sibling node; and 
 determining the used capacity of the qualified sibling node based on the raid node type. 
 
 
     
     
       4. The method of  claim 3 , wherein determining the used capacity of the qualified sibling node based on the raid node type comprises:
 if the raid node type is RAID1: determining the used capacity of the qualified sibling node to be a capacity of a child node of the qualified sibling node; 
 if the raid node type is RAID0 or RAID CONCAT: determining the used capacity of the qualified sibling node to be a sum of a used capacity of all child nodes of the qualified sibling node; 
 if the raid node type is RAID1 over RAID0: determining the used capacity of the qualified sibling node to be a sum of a used capacity of a plurality of available grandchild nodes of the qualified sibling node, wherein the plurality of available grandchild nodes cover the address of the qualified sibling node; 
 if the raid node type is erasure coding (EC): determining the used capacity of the qualified sibling node to be an average of the used capacity of all available child nodes of the qualified sibling node multiplied by the number of data nodes of the qualified sibling node. 
 
     
     
       5. The method of  claim 1 , wherein estimating the workload of the resynchronization task based on the layout further comprises:
 if the layout indicates that the data component does not have either an active or stale mirror component, that the data component has no qualified sibling nodes, and that the data component has a qualified parent node, wherein a sibling node or a parent node is qualified if it is active and has full data availability:
 determining a used capacity of the qualified parent node of the data component in the logical storage system; 
 determining a number of child nodes of the qualified parent node; and 
 estimating the workload to be the used capacity of the qualified parent node divided by the number of child nodes. 
 
 
     
     
       6. The method of  claim 5 , wherein estimating the workload of the resynchronization task based on the layout further comprises:
 if the layout indicates that the data component does not have either an active or stale mirror component, that the data component has no qualified sibling nodes, and that the data component does not have a qualified parent node:
 determining a logical space of the data component; and 
 estimating the workload to be the logical space of the data component. 
 
 
     
     
       7. The method of  claim 1 , wherein determining the estimated size of missed input and output operations at the stale mirror component comprises:
 determining a log sequence number of a last write operation of an object corresponding to the logical storage system; 
 determining a log sequence number of a last write operation of the stale mirror component; 
 determining the estimated size of the missed input by calculating the difference between the log sequence number of the last write operation of the object and the log sequence number of the last write operation of the stale mirror component. 
 
     
     
       8. A non-transitory computer-readable storage medium comprising instructions that, when executed in a computing device, perform a method for estimating a workload of a resynchronization task for a data component in a logical storage system, the method comprising:
 detecting a connection from the data component, wherein the data component was previously disconnected; 
 determining a layout of the data storage system; 
 estimating the workload of the resynchronization task based on the layout by:
 if the layout indicates that the data component has an active mirror component:
 determining a used capacity of the active mirror component; and 
 estimating the workload to be the used capacity of the active mirror component; 
 
 if the layout indicates that the data component does not have an active mirror component and that the data component has a stale mirror component:
 determining a used capacity of the stale mirror component; 
 determining an estimated size of missed input and output operations at the stale mirror component; and 
 estimating the workload to be the sum of the used capacity of the stale mirror component and the estimated size of missed input and output operations; and 
 
 
 performing one or more actions based on the workload estimate, wherein the one or more actions comprise one or more of the following:
 scheduling the re-synchronization task based on the workload estimate; 
 determining whether to perform the re-synchronization task or to choose a different process for restoring the data component based on the workload estimate; and 
 reducing an amount of resources allocated to another task in the logical storage system based on the workload estimate. 
 
 
     
     
       9. The non-transitory computer-readable storage medium of  claim 8 , wherein estimating the workload of the resynchronization task based on the layout further comprises:
 if the layout indicates that the data component does not have either an active or stale mirror component:
 determining a used capacity of each of one or more qualified sibling nodes of the data component in the logical storage system, wherein a sibling node is qualified if it is active and has full data availability; and 
 estimating the workload to be the average of the used capacities of the one or more qualified sibling nodes. 
 
 
     
     
       10. The non-transitory computer-readable storage medium of  claim 9 , wherein determining the used capacity of each of the one or more qualified sibling nodes of the data component in the logical storage system comprises:
 for each qualified sibling node of the one or more qualified sibling nodes that is not a leaf node:
 determining a raid node type of the qualified sibling node; and 
 determining the used capacity of the qualified sibling node based on the raid node type. 
 
 
     
     
       11. The non-transitory computer-readable storage medium of  claim 10 , wherein determining the used capacity of the qualified sibling node based on the raid node type comprises:
 if the raid node type is RAID1: determining the used capacity of the qualified sibling node to be a capacity of a child node of the qualified sibling node; 
 if the raid node type is RAID0 or RAID CONCAT: determining the used capacity of the qualified sibling node to be a sum of a used capacity of all child nodes of the qualified sibling node; 
 if the raid node type is RAID1 over RAID0: determining the used capacity of the qualified sibling node to be a sum of a used capacity of a plurality of available grandchild nodes of the qualified sibling node, wherein the plurality of available grandchild nodes cover the address of the qualified sibling node; 
 if the raid node type is erasure coding (EC): determining the used capacity of the qualified sibling node to be an average of the used capacity of all available child nodes of the qualified sibling node multiplied by the number of data nodes of the qualified sibling node. 
 
     
     
       12. The non-transitory computer-readable storage medium of  claim 8 , wherein estimating the workload of the resynchronization task based on the layout further comprises:
 if the layout indicates that the data component does not have either an active or stale mirror component, that the data component has no qualified sibling nodes, and that the data component has a qualified parent node, wherein a sibling node or a parent node is qualified if it is active and has full data availability:
 determining a used capacity of the qualified parent node of the data component in the logical storage system; 
 determining a number of child nodes of the qualified parent node; and 
 estimating the workload to be the used capacity of the qualified parent node divided by the number of child nodes. 
 
 
     
     
       13. The non-transitory computer-readable storage medium of  claim 12 , wherein estimating the workload of the resynchronization task based on the layout further comprises:
 if the layout indicates that the data component does not have either an active or stale mirror component, that the data component has no qualified sibling nodes, and that the data component does not have a qualified parent node:
 determining a logical space of the data component; and 
 estimating the workload to be the logical space of the data component. 
 
 
     
     
       14. The non-transitory computer-readable storage medium of  claim 8 , wherein determining the estimated size of missed input and output operations at the stale mirror component comprises:
 determining a log sequence number of a last write operation of an object corresponding to the logical storage system; 
 determining a log sequence number of a last write operation of the stale mirror component; 
 determining the estimated size of the missed input by calculating the difference between the log sequence number of the last write operation of the object and the log sequence number of the last write operation of the stale mirror component. 
 
     
     
       15. A computer system, comprising:
 a storage device comprising a file system; 
 a processor (CPU) configured to perform a method for estimating a workload of a resynchronization task for a data component in a logical storage system, the method comprising:
 detecting a connection from the data component, wherein the data component was previously disconnected; 
 determining a layout of the data storage system; 
 estimating the workload of the resynchronization task based on the layout by:
 if the layout indicates that the data component has an active mirror component:
 determining a used capacity of the active mirror component; and 
 estimating the workload to be the used capacity of the active mirror component; 
 
 if the layout indicates that the data component does not have an active mirror component and that the data component has a stale mirror component:
 determining a used capacity of the stale mirror component; 
 determining an estimated size of missed input and output operations at the stale mirror component; and 
 
 estimating the workload to be the sum of the used capacity of the stale mirror component and the estimated size of missed input and output operations; and 
 
 performing one or more actions based on the workload estimate, wherein the one or more actions comprise one or more of the following:
 scheduling the re-synchronization task based on the workload estimate; 
 determining whether to perform the re-synchronization task or to choose a different process for restoring the data component based on the workload estimate; and 
 reducing an amount of resources allocated to another task in the logical storage system based on the workload estimate. 
 
 
 
     
     
       16. The computer system of  claim 15 , wherein estimating the workload of the resynchronization task based on the layout further comprises:
 if the layout indicates that the data component does not have either an active or stale mirror component:
 determining a used capacity of each of one or more qualified sibling nodes of the data component in the logical storage system, wherein a sibling node is qualified if it is active and has full data availability; and 
 estimating the workload to be the average of the used capacities of the one or more qualified sibling nodes. 
 
 
     
     
       17. The computer system of  claim 16 , wherein determining the used capacity of each of the one or more qualified sibling nodes of the data component in the logical storage system comprises:
 for each qualified sibling node of the one or more qualified sibling nodes that is not a leaf node:
 determining a raid node type of the qualified sibling node; and 
 determining the used capacity of the qualified sibling node based on the raid node type. 
 
 
     
     
       18. The computer system of  claim 17 , wherein determining the used capacity of the qualified sibling node based on the raid node type comprises:
 if the raid node type is RAID1: determining the used capacity of the qualified sibling node to be a capacity of a child node of the qualified sibling node; 
 if the raid node type is RAID0 or RAID CONCAT: determining the used capacity of the qualified sibling node to be a sum of a used capacity of all child nodes of the qualified sibling node; 
 if the raid node type is RAID1 over RAID0: determining the used capacity of the qualified sibling node to be a sum of a used capacity of a plurality of available grandchild nodes of the qualified sibling node, wherein the plurality of available grandchild nodes cover the address of the qualified sibling node; 
 if the raid node type is erasure coding (EC): determining the used capacity of the qualified sibling node to be an average of the used capacity of all available child nodes of the qualified sibling node multiplied by the number of data nodes of the qualified sibling node. 
 
     
     
       19. The computer system of  claim 15 , wherein estimating the workload of the resynchronization task based on the layout further comprises:
 if the layout indicates that the data component does not have either an active or stale mirror component, that the data component has no qualified sibling nodes, and that the data component has a qualified parent node, wherein a sibling node or a parent node is qualified if it is active and has full data availability:
 determining a used capacity of the qualified parent node of the data component in the logical storage system; 
 determining a number of child nodes of the qualified parent node; and 
 estimating the workload to be the used capacity of the qualified parent node divided by the number of child nodes. 
 
 
     
     
       20. The computer system of  claim 19 , wherein estimating the workload of the resynchronization task based on the layout further comprises:
 if the layout indicates that the data component does not have either an active or stale mirror component, that the data component has no qualified sibling nodes, and that the data component does not have a qualified parent node:
 determining a logical space of the data component; and 
 estimating the workload to be the logical space of the data component. 
 
 
     
     
       21. The computer system of  claim 15 , wherein determining the estimated size of missed input and output operations at the stale mirror component comprises:
 determining a log sequence number of a last write operation of an object corresponding to the logical storage system; 
 determining a log sequence number of a last write operation of the stale mirror component; 
 determining the estimated size of the missed input by calculating the difference between the log sequence number of the last write operation of the object and the log sequence number of the last write operation of the stale mirror component.