System and method for improving memory usage in virtual machines at a cost of increasing CPU usage

Abstract

An apparatus includes at least one processor executing a method for managing memory among a plurality of concurrently-running virtual machines, and a non-transitory memory device that stores a set of computer readable instructions for implementing and executing said memory management method. A memory optimization mechanism can reduce a memory usage of a virtual machine at a cost of increasing a central processing unit (CPU) usage. Information on a memory usage and a CPU usage of each virtual machine is periodically collected. When a first virtual machine exhibits high memory use, at least one second virtual machine with an extra CPU capacity is identified. A memory optimization mechanism is applied to the second virtual machine to reduce memory used by the second virtual machine, thereby providing a portion of freed memory that is then allocated to the first virtual machine.

What is claimed is: 
     
       1. An apparatus, comprising:
 at least one processor executing a method for managing memory among a plurality of concurrently-running virtual machines; and 
 a non-transitory memory device that stores a set of computer readable instructions for implementing and executing said memory management method, 
 wherein said method comprises:
 providing at least one memory optimization mechanism, as executed by at least one processor on at least one computer, which can reduce a memory usage of a virtual machine at a cost of increasing a central processing unit (CPU) usage; 
 periodically collecting information on a memory usage and a CPU usage of each virtual machine; 
 in response to a first virtual machine exhibiting a high level of memory use, identifying at least one second virtual machine with an extra CPU capacity; 
 applying, to the at least one second virtual machine the at least one memory optimization mechanism, to reduce memory used by the at least one second virtual machine, thereby providing a portion of freed memory; and 
 allocating the portion of freed memory to the first virtual machine. 
 
 
     
     
       2. The apparatus of  claim 1 , wherein the at least one memory optimization mechanism comprises reducing a heap size corresponding to a memory area managed by a garbage collector. 
     
     
       3. The apparatus of  claim 2 , the method further comprising:
 collecting empirical data on a garbage collection overhead as a function of heap size; and 
 using the empirical data to determine a heap size for said at least one second virtual machine. 
 
     
     
       4. The apparatus of  claim 2 , wherein the memory area managed by the garbage collector comprises memory supporting the execution of at least one program written in at least one of Java, Lisp, C#, and Smalltalk. 
     
     
       5. The apparatus of  claim 1 , wherein the at least one memory optimization mechanism comprises at least one of memory compression, memory deduplication, and delta encoding. 
     
     
       6. The apparatus of  claim 1 , wherein said first virtual machine exhibiting a high level of memory usage comprises one of a level of memory usage of said first virtual machine exceeding a threshold and said first virtual machine exhibiting memory paging. 
     
     
       7. The apparatus of  claim 1 , said method further comprising:
 maintaining information on memory usage from the past; and 
 using said information on memory usage from the past to determine that said first virtual machine will likely exceed the amount of memory currently assigned to it in the near future. 
 
     
     
       8. The apparatus of  claim 1 , the method further comprising:
 in response to detecting that a virtual machine vm 1 , which is applying a memory optimization mechanism o 1  which can reduce the memory usage of the virtual machine at a cost of increasing a central processing unit (CPU) usage, has a high level of CPU usage, reducing or stopping application of said memory optimization mechanism o 1 . 
 
     
     
       9. The apparatus of  claim 1 , wherein any one of a plurality of memory optimization mechanisms, each of which can reduce a memory usage of a virtual machine at a cost of increasing a central processing unit (CPU) usage, can be selectively applied to a virtual machine vm 1 , the method further comprising selecting a memory optimization mechanism o 1  from the plurality of memory optimization mechanism by giving preference to memory optimization mechanisms which at least one of:
 consume fewer CPU cycles; and 
 free up more memory. 
 
     
     
       10. The apparatus of  claim 1 , said method further comprising:
 upon determining that said first virtual machine is exhibiting said high level of memory use, initially determining whether additional memory is available without having to free memory from another virtual machine; and 
 if such additional memory is available, then allocating at least a portion of the additional memory to said first virtual machine. 
 
     
     
       11. The apparatus of  claim 1 , said method further comprising:
 setting a predetermined threshold of CPU usage for each said virtual machine; 
 determining whether a virtual machine will continue to operate below its CPU usage threshold if memory currently allocated to that virtual machine were to be freed; and 
 freeing memory from that virtual machine only if its CPU usage will remain below its CPU usage threshold, by said applying the at least one memory optimization mechanism to that virtual machine and deallocating memory from that virtual machine of memory freed up by the at least one memory optimization mechanism. 
 
     
     
       12. The apparatus of  claim 1 , said method further comprising:
 setting a target CPU usage level for each said virtual machine; and 
 providing a priority among said virtual machines for said applying the at least one memory optimization mechanism, said priority being based upon a difference between a current CPU usage level of a virtual machine and its target CPU usage level. 
 
     
     
       13. The apparatus of  claim 12 , said method further comprising:
 detecting that a virtual machine is exceeding its target CPU usage level during a period in which a memory optimization mechanism processing is being executed; and 
 reducing a level of the memory optimization mechanism being executed on said virtual machine detected as exceeding its target CPU usage level in order to bring the CPU usage level below the target CPU usage level. 
 
     
     
       14. The apparatus of  claim 1 , said method further comprising:
 setting a target memory usage level for each said virtual machine; and 
 providing a priority among said virtual machines for said applying the at least one memory optimization mechanism, said priority being based upon a difference between a current memory usage level of a virtual machine and its target memory usage level. 
 
     
     
       15. The apparatus of  claim 1 , wherein said plurality of concurrently-running virtual machines comprises a plurality of virtual machines operating on a plurality of computers interconnected in a network. 
     
     
       16. An apparatus comprising:
 at least one processor; and 
 a non-transitory memory device, said memory device storing instructions for a method of controlling a memory allocation among a plurality of concurrently-running virtual machines (VMs), 
 said method comprising:
 monitoring a central processing unit (CPU) usage and a memory usage of each said VM, as executed by at least one processor on at least one computer; 
 detecting that a first VM of the plurality of concurrently-running VMs requires more memory; 
 exercising a memory optimization mechanism for allocating additional memory to the first VM wherein the memory optimization mechanism increases a central processing unit (CPU) usage of at least one of the concurrently-running VMs; and 
 allocating the additional memory to the first VM, 
 
 wherein the additional memory to be allocated is determined based upon considering said increase in central processing unit (CPU) usage of at least one of the concurrently-running VMs in order to reallocate memory to said first VM. 
 
     
     
       17. The apparatus of  claim 16 , wherein the memory optimization mechanism comprises one or more of:
 a memory compression mechanism, wherein a contents of memory are compressed using a predetermined compression algorithm; 
 a memory deduplication mechanism, wherein duplicate memory pages are identified and a single copy of the duplication memory pages are retained and shared among more than one virtual machine; and 
 a delta encoding mechanism, wherein, for two pages having similar but not identical contents, only one of the two pages is retained along with a delta which encodes differences between the two pages. 
 
     
     
       18. The apparatus of  claim 17 , wherein a criterion for selecting which memory optimization mechanism to apply, from a plurality of memory optimization mechanisms, comprises one or more of:
 if a VM's predicted CPU utilization after applying a memory optimization mechanism is below a threshold value, then apply the memory optimization mechanism; 
 if a VM's predicted CPU utilization after applying a memory optimization mechanism is below a threshold value and an amount of memory predicted to be freed by the memory optimization mechanism exceeds a threshold, then apply the memory optimization mechanism; 
 if a VM's predicted CPU utilization after applying a memory optimization mechanism is below a threshold value and an amount of memory predicted to be freed by the memory optimization mechanism exceeds a threshold proportion of a memory amount needed by the first VM, then apply the memory optimization mechanism; 
 if each VM has a target CPU utilization, then give a priority to applying a memory optimization mechanism to VMs which have a highest value of a target CPU utilization minus an actual CPU utilization; and 
 if each VM has a target memory usage, then give a priority to applying a memory optimization mechanism to VMs which have a lowest value of a target memory utilization minus an actual memory utilization. 
 
     
     
       19. The apparatus of  claim 16 , wherein the detecting that the first VM needs more memory comprises one or more of:
 detecting that an amount of free memory available to said first VM is decreasing and approaching zero; 
 detecting that said first VM has no free memory and is paging; and 
 making a prediction of memory needs of said first VM based on a past performance. 
 
     
     
       20. An apparatus, comprising:
 at least one processor; and 
 a non-transitory memory device storing instructions for a method of controlling a memory allocation among a plurality of concurrently-running virtual machines (VMs), 
 said method comprising:
 setting a target central processing unit (CPU) usage level for each said VM; 
 monitoring a CPU usage and a memory usage of each said VM; 
 applying a memory optimization to a VM which reduces memory usage at a cost of increasing CPU usage; 
 detecting that the VM is exceeding its target CPU usage level; and 
 reducing a level of the memory optimization being executed on the VM.