Memory hierarchy reconfiguration for energy and performance in general-purpose processor architectures

Abstract

A cache and TLB layout and design leverage repeater insertion to provide dynamic low-cost configurability trading off size and speed on a per application phase basis. A configuration management algorithm dynamically detects phase changes and reacts to an application's hit and miss intolerance in order to improve memory hierarchy performance while taking energy consumption into consideration.

1. A method of reconfiguring a data cache for caching data in a computing device, the data cache operating at a plurality of levels in a memory hierarchy and comprising a portion having a variable size operating at a first level of the plurality of levels, the method comprising:
 (a) storing performance information for the data cache;  
 (b) determining, from the performance information, whether the data cache has a miss rate exceeding a threshold;  
 (c) determining whether the variable size is equal to a maximum size; and  
 (d) if the miss rate exceeds the threshold and the variable size is not equal to the maximum size, controlling the data cache to increase the variable size.  
 
     
     
       2. The method of  claim 1 , further comprising:
 (e) if the miss rate does not exceed the threshold or the variable size is equal to the maximum size, (i) determining, from the performance information, an optimal data cache configuration which optimizes a number of cycles per instruction in the computing device and (ii) setting the data cache to the optimal data cache configuration.  
 
     
     
       3. The method of  claim 2 , wherein, in each of a plurality of time periods during which the data cache operates, steps (a)-(c) and one of steps (d) and (e) are performed. 
     
     
       4. The method of  claim 3 , wherein each of the time periods is a fixed number of cycles of the computing device. 
     
     
       5. The method of  claim 3 , wherein each of the time periods is a time period in which the computing device performs a subroutine. 
     
     
       6. The method of  claim 3 , wherein:
 the data cache is designated as either stable or unstable; and  
 steps (a)-(c) are performed only during intervals in which the data cache is designated as unstable.  
 
     
     
       7. The method of  claim 6 , further comprising, during intervals in which the data cache is designated as stable:
 (f) determining, from the performance information, whether the data cache is actually unstable; and  
 (g) if the data cache is actually unstable, (i) designating the data cache as unstable and (ii) setting the variable size to a minimum value.  
 
     
     
       8. The method of  claim 7 , wherein:
 the performance indication comprises a hit counter for a second portion of the data cache which is outside the  
 portion having the variable size; and  
 when the data cache is designated as stable and the hit counter is below a hit counter threshold, the second portion of the data cache is bypassed.  
 
     
     
       9. The method of  claim 1 , wherein:
 the data cache comprises tag arrays and data arrays;  
 the first level is L1; and  
 in the portion having the variable size, the tag arrays and the data arrays are read in series.  
 
     
     
       10. A method of reconfiguring a translation look-aside buffer for use in a computing device, the translation look-aside buffer having a variable size, the method comprising:
 (a) storing performance information for the translation look-aside buffer;  
 (b) determining, from the performance information, whether the translation look-aside buffer has a miss rate exceeding a first threshold;  
 (c) determining, from the performance information, whether the translation look-aside buffer has a usage less than a second threshold;  
 (d) if the miss rate exceeds the first threshold, controlling the translation look-aside buffer to increase the variable size; and  
 (e) if the use is less than the second threshold, controlling the translation look-aside buffer to decrease the variable size.  
 
     
     
       11. The method of  claim 10 , wherein, in each of a plurality of time periods during which the data cache operates, steps (a)-(c) and one of steps (d) and (e) are performed. 
     
     
       12. The method of  claim 11 , wherein each of the time periods is a fixed number of cycles of the computing device. 
     
     
       13. A method for configuring a cache, comprising:
   storing performance information for a data cache having at least one portion with a variable size, wherein the data cache is configured to operate at a plurality of levels in a memory hierarchy;        determining, from the performance information, whether a miss rate for the data cache exceeds a threshold; and        if the miss rate exceeds the threshold, increasing the variable size.     
     
     
       14. The method of  claim 13 , further comprising:
   determining whether the variable size is equal to a maximum size; and        increasing the variable size if the variable size is determined to be less than a maximum size.     
     
     
       15. The method of  claim 14 , further comprising not increasing the variable size if the variable size is determined to be at least the maximum size. 
     
     
       16. The method of  claim 13 , further comprising:
   if the miss rate does not exceed the threshold or the variable size is equal to the maximum size;        determining, from the performance information, an optimal data cache configuration which optimizes a number of cycles per instruction in the computing device; and        setting the data cache to the optimal data cache configuration.     
     
     
       17. A non- transitory tangible computer - readable medium having instructions stored thereon, the instructions comprising:      instructions to store performance information for a data cache having at least a portion thereof with a variable size, wherein the data cache is configured to operate at a plurality of levels in a memory hierarchy;        instructions to determine, from the performance information, whether a miss rate for the data cache exceeds a threshold; and        instructions to increase the variable size in response to the miss rate exceeding the threshold.     
     
     
       18. The non- transitory tangible computer - readable medium of    claim 17   , further comprising:      instructions to determine whether the variable size is equal to a maximum size; and        instructions to increase the variable size if the variable size is determined to be less than a maximum size.     
     
     
       19. The non- transitory tangible computer - readable medium of    claim 18   , further comprising instructions to not increase the variable size if the variable size is determined to be at least the maximum size.   
     
     
       20. The non- transitory tangible computer - readable medium of    claim 17   , further comprising:      if the miss rate does not exceed the threshold or the variable size is equal to the maximum size;        instructions to determine, from the performance information, an optimal data cache configuration which optimizes a number of cycles per instruction in the computing device; and        instructions to set the data cache to the optimal data cache configuration.     
     
     
       21. A method, comprising:
   storing performance information for a translation look - aside buffer having a variable size;        determining from the performance information whether a miss rate for the translation look - aside buffer exceeds a first threshold; and        if the miss rate exceeds the first threshold, increasing the variable size.     
     
     
       22. The method of  claim 21 , further comprising:
   determining from the performance information whether the translation look - aside buffer has a usage less than a second threshold; and        if the use is less than the second threshold, controlling the translation look - aside buffer to decrease the variable size.     
     
     
       23. A non- transitory machine readable medium having stored thereon instructions that, if executed by a processor, result in a method comprising:      storing performance information for a translation look - aside buffer having a variable size;        determining from the performance information whether a miss rate for the translation look - aside buffer exceeds a first threshold; and        if the miss rate exceeds the first threshold, increasing the variable size.     
     
     
       24. The non- transitory machine readable medium of    claim 23   , further comprising:      determining from the performance information whether the translation look - aside buffer has a usage less than a second threshold; and        if the use is less than the second threshold, controlling the translation look - aside buffer to decrease the variable size.