System and method for application configuration for performance test

Abstract

The present invention enables a performance testing framework that enables multiple components working together to test a deployed application automatically in an unattended manner and to analyze the test results easily. At very high level, the performance testing framework can run performance tests on a tested system with one or more variations without user intervention and save the test results and configuration metadata to a database for later analysis. In addition, the framework can configure the tested system by mapping its components to a plurality of hosts running the tested system. This description is not intended to be a complete description of, or limit the scope of, the invention. Other features, aspects, and objects of the invention can be obtained from a review of the specification, the figures, and the claims.

1 . A system to support performance testing, comprising: 
 a tested system, comprising one or more of: 
 one or more application servers operable to deploy and provide an application to a user; and  
 a load generating tool operable to execute a plurality of test runs to measure performance of the application automatically;  
   an infrastructure, comprising one or more of: 
 an automation controller operable to analyze performance data of the plurality of test runs within and/or between a plurality of variations; and  
   a variation controller operable to: 
 generate the plurality of test runs based on a plurality of variations of configuration properties of the tested system;  
 configure the tested system and/or the infrastructure on a per-variation basis by specifying their component instances and configurations of the component instances to run on a plurality of hosts; and  
 configure, manage, and monitor the component instances for each of the plurality of variations and test runs.  
   
     
     
         2 . The system according to  claim 1 , wherein: 
 the application is a Web-based application.    
     
     
         3 . The system according to  claim 1 , wherein the tested system further comprising one or more of: 
 an admin server operable to manage the application; and    one or more proxy servers for the application provided.    
     
     
         4 . The system according to  claim 1 , wherein: 
 one primary host of the plurality of hosts runs the variation controller.    
     
     
         5 . The system according to  claim 1 , wherein: 
 each of the plurality of variations specifies one or more of: 
 one or more component instances to run on each of the plurality of hosts of the tested system and/or the infrastructure; and  
 how each component instance of the tested system and/or the infrastructure is configured on each of the plurality of hosts.  
   
     
     
         6 . The system according to  claim 1 , wherein: 
 some of the configuration properties of the tested system are host-specific and/or server specific.    
     
     
         7 . The system according to  claim 1 , wherein: 
 host-specific configuration properties of the tested system is stored in a network configuration file.    
     
     
         8 . The system according to  claim 7 , wherein: 
 the network configuration file is a multi-level XML file.    
     
     
         9 . The system according to  claim 7 , wherein: 
 the network configuration file specifies location of the component instances of the tested system, wherein multiple component instances of a given component type is configured to run on one or more of the plurality of hosts.    
     
     
         10 . The system according to  claim 1 , wherein: 
 performance of the application is measured by one or more of: response time, throughput, capacity, and scalability.    
     
     
         11 . The system according to  claim 10 , wherein: 
 the variation controller is operable to run the plurality of test runs with varying configuration properties of the tested system across the plurality of test runs to see how such varying affects performance of the application.    
     
     
         12 . A method to support performance testing, comprising: 
 generating and executing a plurality of test runs based on a plurality of variations of configuration properties of a tested system;    configuring the tested system and/or its infrastructure on a per-variation basis by specifying their component instances and configuration properties of the component instances to run on a plurality of hosts;    configuring, managing, and monitoring the component instances for each of the plurality of variations and test runs; and    analyzing and reporting performance data of the plurality of test runs within and/or between the plurality of variations.    
     
     
         13 . The method according to  claim 12 , further comprising: 
 configuring the tested system based on its configuration properties.    
     
     
         14 . The method according to  claim 12 , further comprising: 
 accessing configuration properties of the component instances of the tested system from a network configuration file.    
     
     
         15 . The method according to  claim 14 , further comprising: 
 specifying location of the component instances of the tested system, wherein multiple component instances of a given component type is configured to run on one or more of the plurality of hosts.    
     
     
         16 . The method according to  claim 12 , further comprising: 
 measuring performance of the application is by one or more of: response time, throughput, capacity, and scalability.    
     
     
         17 . The method according to  claim 16 , further comprising: 
 running the plurality of test runs with varying configuration properties of the tested system across the plurality of test runs to see how such varying affects performance of the application.    
     
     
         18 . The method according to  claim 12 , further comprising one or more of: 
 summarizing and reporting analysis result of the performance data to a user;    storing the performance data and/or the analysis result to a database;    presenting the analysis result to the user on a Web browser; and    exporting the analysis result to a spreadsheet.    
     
     
         19 . A machine readable medium having instructions stored thereon that when executed cause a system to: 
 generate and execute a plurality of test runs based on a plurality of variations of configuration properties of a tested system;    configure the tested system and/or its infrastructure on a per-variation basis by specifying their component instances and configuration properties of the component instances to run on a plurality of hosts;    configure, manage, and monitor the component instances for each of the plurality of variations and test runs; and    analyze and report performance data of the plurality of test runs within and/or between the plurality of variations.    
     
     
         20 . A system to support performance testing, comprising: 
 means for generating and executing a plurality of test runs based on a plurality of variations of configuration properties of a tested system;    means for configuring the tested system and/or its infrastructure on a per-variation basis by specifying their component instances and configuration properties of the component instances to run on a plurality of hosts;    means for configuring, managing, and monitoring the component instances for each of the plurality of variations and test runs; and    means for analyzing and reporting performance data of the plurality of test runs within and/or between the plurality of variations.