Method and apparatus for testing software

Abstract

Techniques are provided for testing new software slated to be deployed on a target machine population with a number of target machines having a number of existing software programs. Method steps can include obtaining a list of depending software programs on selected ones of the target machines. The list can include those of the existing software programs which depend on the new software and/or those of tile existing software programs on which the new software depends. The steps can further include identifying relevant test cases based on the list of depending software programs, instantiating a plurality of test client platforms characteristic of the target machine population, and testing the relevant test cases on the plurality of test client platforms to identify failures.

1 . A method for testing new software to be deployed on a target machine population having a plurality of target machines, the target machines having a plurality of existing software programs, said method comprising the steps of: 
 obtaining a list of depending software programs on selected ones of the target machines, said list comprising: 
 those of the existing software programs that depend on the new software; and  
 those of the existing software programs on which the new software depends;  
   identifying relevant test cases based on said list of depending software programs;    instantiating a plurality of test client platforms characteristic of the target machine population; and    testing said relevant test cases on said plurality of test client platforms to identify failures.    
     
     
         2 . The method of  claim 1 , further comprising the additional step of matching said relevant test cases to given ones of said test client platforms.  
     
     
         3 . The method of  claim 2 , further comprising the additional step of collecting test results from said plurality of test client platforms.  
     
     
         4 . The method of  claim 3 , wherein said relevant test cases comprise at least installation test cases and said testing comprises at least testing for installation failures.  
     
     
         5 . The method of  claim 3 , wherein said relevant test cases comprise at least test cases associated with the existing software programs that depend on the new software and said testing comprises at least testing for runtime failures with said test cases associated with the existing software programs that depend on the new software.  
     
     
         6 . The method of  claim 3 , wherein said relevant test cases comprise at least test cases associated with the new software and said testing comprises at least testing for runtime failures with said test cases associated with the new software.  
     
     
         7 . The method of  claim 3 , wherein: 
 said relevant test cases comprise at least two of: 
 installation test cases;  
 test cases associated with the existing software programs that depend on the flew software; and  
 test cases associated with the new software; and  
   said testing comprises at least running at least two of said relevant cases in combination.    
     
     
         8 . The method of  claim 7 , further comprising the additional step of generating a list of those of said test client platforms characteristic of the target machine population that experienced at least one of said installation failures and said runtime failures.  
     
     
         9 . The method of  claim 8 , further comprising the additional step of fixing the new software to obtain fixed new software and to address said at least one of said installation failures and said runtime failures.  
     
     
         10 . The method of  claim 9 , further comprising the additional step of repeating at least said obtaining, identifying, instantiating, testing, and generating steps with said fixed new software and with said test client platforms characteristic of the target machine population that experienced at least one of said installation failures and said runtime failures substituted for said plurality of test client platforms characteristic of the target machine population in said instantiating step.  
     
     
         11 . The method of  claim 8 , further comprising the additional step of modifying installation instructions associated with said new software to obtain modified installation instructions and to bypass installation of the new software on those of said machines in the target machine population having a configuration similar to those of said test client platforms that experienced said at least one of said installation failures and said runtime failures.  
     
     
         12 . The method of  claim 11 , further comprising the additional step of repeating said obtaining, identifying, instantiating, testing, and generating steps with said modified installation instructions and with said test client platforms characteristic of the target machine population that experienced at least one of said installation failures and said runtime failures substituted for said plurality of test client platforms characteristic of the target machine population in said instantiating step.  
     
     
         13 . The method of  claim 1 , wherein said obtaining step comprises: 
 generating a set of distinct candidate test client platform configurations based on at least one predetermined criterion;    obtaining first software dependencies for the new software;    obtaining second software dependencies for the plurality of existing software programs; and    selectively combining said dependencies to obtain said list of depending software programs.    
     
     
         14 . The method of  claim 13 , further comprising the additional step of pruning said list of distinct candidate test client platform configurations to eliminate those of said distinct candidate test client platform configurations that satisfy at least one of the following: 
 do not employ at least one of the new software and the existing software programs on which the new software depends;    are found on a predetermined exclusion list; and    are determined by rule-based criteria to qualify for exclusion from test.    
     
     
         15 . The method of  claim 1 , further comprising the additional steps of: 
 monitoring the target machine population to detect changes to the target machine population during said testing; and    dynamically updating said plurality of test client platforms, responsive to said changes detected during said monitoring.    
     
     
         16 . A computer program product comprising a computer useable medium having computer useable program code for testing new software to be deployed on a target machine population having a plurality of target machines, the target machines having a plurality of existing software programs, said computer program product including: 
 computer useable program code for obtaining a list of depending software programs on selected ones of the target machines, said list comprising: 
 those of the existing software programs that depend on the new software; and  
 those of the existing software programs on which the new software depends;  
   computer useable program code for identifying relevant test cases based on said list of depending software programs;    computer useable program code for instantiating a plurality of test client platforms characteristic of the target machine population; and    computer useable program code for testing said relevant test cases on said plurality of test client platforms to identify failures.    
     
     
         17 . The computer program product of  claim 16 , wherein said product further comprises computer useable program code for matching said relevant test cases to given ones of said test client platforms.  
     
     
         18 . The computer program product of  claim 17 , wherein said product further comprises computer useable program code for collecting test results from said plurality of test client platforms.  
     
     
         19 . An apparatus for testing new software to be deployed on a target machine population having a plurality of target machines, the target machines having a plurality of existing software programs, comprising: 
 a memory; and    at least one processor coupled to said memory and operative to: 
 obtain a list of depending software programs on selected ones of the target machines, said list comprising: 
 those of the existing software programs which depend on the new software; and  
 those of the existing software programs on which the new software depends;  
 
 identify relevant test cases based on said list of depending software programs;  
 instantiate a plurality of test client platforms characteristic of the target machine population; and  
 test said relevant test cases on said plurality of test client platforms to identify failures.  
   
     
     
         20 . The apparatus of  claim 19 , wherein said processor is further operative to match said relevant test cases to given ones of said test client platforms.