Adaptive throttling of metadata requests

Abstract

An identification of a primary snapshot created for a primary storage system is received. A first request for a first metadata of a first file directory structure object associated with the primary snapshot is issued. A second request for data content of the first file directory structure object associated with the primary snapshot is determined to be sent to a recipient device based on a received response to the first request. A third request for a second metadata of a second file directory structure object associated with the primary snapshot is determined to be sent to the recipient device. Timing and ordering of issuance of a plurality of requests that at least include the second request and the third request to the recipient device are managed based on a determined performance metric of the recipient device and corresponding relative impact to the performance metric of the recipient device.

What is claimed is: 
     
       1. A method, comprising:
 determining a performance metric associated with a recipient device; 
 assigning a cost for performing a type of request based on the determined performance metric associated with the recipient device; 
 adding an initial set of metadata requests to a queue; 
 determining whether a backup node has a threshold number of tokens to issue a first request included in the initial set of metadata requests at a front of the queue, wherein the first request has the assigned cost; and 
 issuing to the recipient device the first request at the front of the queue in response to a determination that the backup node has the threshold number of tokens to issue the request at the front of the queue. 
 
     
     
       2. The method of  claim 1 , wherein the queue is a queue for object metadata requests and object data content requests. 
     
     
       3. The method of  claim 1 , wherein the queue is a queue for object metadata requests. 
     
     
       4. The method of  claim 1 , wherein the determined performance metric associated with the recipient device is a CPU utilization of the recipient device. 
     
     
       5. The method of  claim 1 , wherein the determined performance metric associated with the recipient device is directly determined by the backup node of a backup system. 
     
     
       6. The method of  claim 1 , wherein the determined performance metric associated with the recipient device is indirectly determined by the backup node of a backup system. 
     
     
       7. The method of  claim 1 , wherein the backup node generates a particular number of tokens per second. 
     
     
       8. The method of  claim 7 , wherein a rate at which the backup node generates the particular number of tokens is based on the determined performance metric associated with the recipient device. 
     
     
       9. The method of  claim 7 , wherein a rate at which the backup node generates the particular number of tokens is reduced in response to a determination that the performance metric is above a performance metric threshold. 
     
     
       10. The method of  claim 7 , wherein a rate at which the backup node generates the particular number of tokens is increased in response to a determination that the performance metric is below a performance metric threshold. 
     
     
       11. The method of  claim 1 , the cost assigned to a metadata request associated with a file directory structure object is greater than the cost assigned to a data content request associated with the file directory structure object. 
     
     
       12. The method of  claim 1 , wherein the assigned cost is dynamically determined based on the performance of the recipient device. 
     
     
       13. The method of  claim 1 , wherein the assigned cost is periodically updated. 
     
     
       14. The method of  claim 1 , wherein the initial set of metadata requests is a preconfigured number of requests. 
     
     
       15. The method of  claim 1 , wherein each metadata request included in the initial set of metadata requests corresponds to a node of a file directory structure. 
     
     
       16. The method of  claim 1 , further comprising resetting a number of tokens generated by the backup node after a predetermined interval. 
     
     
       17. The method of  claim 1 , further comprising adding a subsequent set of metadata requests to the queue. 
     
     
       18. The method of  claim 17 , further comprising:
 waiting a particular amount of time in response to a determination that the backup node does not have the threshold number of tokens to issue a second request at the front of the queue; and 
 re-evaluating whether the backup node has the threshold number of tokens to issue the second request at the front of the queue after waiting the particular amount of time. 
 
     
     
       19. A computer program product embodied in a non-transitory computer readable medium and comprising computer instructions for:
 determining a performance metric associated with a recipient device; 
 assigning a cost for performing a type of request based on the determined performance metric associated with the recipient device; 
 adding an initial set of metadata requests to a queue; 
 determining whether a backup node has a threshold number of tokens to issue a first request included in the initial set of metadata requests at a front of the queue, wherein the first request has the assigned cost; and 
 issuing to the recipient device the first request at the front of the queue in response to a determination that the backup node has the threshold number of tokens to issue the request at the front of the queue. 
 
     
     
       20. A system, comprising:
 a processor configured to:
 determine a performance metric associated with a recipient device; 
 assign a cost for performing a type of request based on the determined performance metric associated with the recipient device; 
 add an initial set of metadata requests to a queue; 
 determine whether a backup node has a threshold number of tokens to issue a first request at a front of the queue; and 
 issue to the recipient device the first request at the front of the queue in response to a determination that the backup node has the threshold number of tokens to issue the request included in the initial set of metadata requests at the front of the queue, wherein the first request has the assigned cost; and 
 
 a memory coupled to the processor and configured to provide the processor with instructions.