Automated benchmarking of software performance

Abstract

Methods for automated benchmarking of views of one or more windows of an application. The methods analyzes content of a window to benchmark views in the window automatically once a target application and/or target window have been specified for benchmarking by a user. Benchmarking is performed without requiring the addition or modification of any code in the target application. Views of a window may be analyzed directly through the window or through a view list or view hierarchy associated with the window. In some embodiments, views of a window are benchmarked through the application that creates the window. In some embodiments, a window is benchmarked directly through a resource file that defines the window without use of an application. In some embodiments, the method benchmarks one or more commands of a data file. In some embodiments, one or more introspectable objects in a collection is automatically benchmarked.

1 .- 53 . (canceled) 
     
     
         54 . A method for benchmarking a graphical user interface window, comprising:
 processing a set of files used for providing functionality of a window in a graphical user interface of a software application, the set of files including a first file including a definition of the window and a second file including a command used for altering a display of the window;   generating the window from the definition of the window without execution of the software application, the generated window having a control visible in a first view of the window to change the generated window from the first view to a second view;   performing the command upon the generated window, the command associated with the control to change the generated window from the first view to the second view; and   benchmarking a performance of the command associated with the control by measuring a time required to complete the performing of the command.   
     
     
         55 . The method of  claim 54 , wherein performing the command includes executing a user-defined operation on the window to be benchmarked. 
     
     
         56 . The method of  claim 54 , wherein performing the command including executing a script included in an AppleScript data file to perform the command. 
     
     
         57 . The method of  claim 54 , wherein performing the command includes executing a run command for generating a particular view of the window and wherein the benchmarking includes measuring a time required for the run command to complete generating the particular view of the window. 
     
     
         58 . The method of  claim 54 , wherein performing the command includes executing a run command followed by a reset command, and wherein the run command alters a state of the generated window from an original state to a modified state and the reset command alters the state of the window from the modified state to the original state. 
     
     
         59 . The method of  claim 54 , wherein the first view has one or more associated commands to change a drawing-related appearance of the window to the second view, and wherein the benchmarking includes measuring the time required for the command to complete. 
     
     
         60 . The method of  claim 54 , wherein the generated window includes predefined content, and wherein benchmarking the performance of the command associated with the control is performed regardless of the predefined content. 
     
     
         61 . The method of  claim 54 , wherein generating the window from the definition included in the first file and performing the command included in the second file both include accessing a common data file to obtain the definition and the command. 
     
     
         62 . The method of  claim 54 , wherein the first file further defines a plurality of windows, and wherein the second file further defines a plurality of commands, with respective windows of the plurality of windows including different sets of views, and wherein the method further comprises, for each window:
 selecting a particular window from the plurality of windows; and   performing a particular command associated with a particular view of the selected window to change a state of the selected window to the particular view.   
     
     
         63 . The method of  claim 62 , wherein the benchmarking is performed for the respective windows to measure a time required to complete the performing of the particular command to change the state of the selected window to the particular view. 
     
     
         64 . A computer readable storage medium including instructions for benchmarking a window of a graphical user interface that, when executed by a computing system, cause the computing system to:
 generate the window of the graphical user interface based on a definition from a resource file, the resource file defining visible characteristics of the window;   perform a particular set of commands based on a definition from a data file, the set of commands associated with a particular control to change a view of the generated window; and   benchmark a performance of the particular set of commands by measuring a time required to complete the performance of the particular set of commands to change the view of the generated window.   
     
     
         65 . The computer readable storage medium of  claim 64 , wherein a command in the particular set of commands includes a user-defined operation to be benchmarked. 
     
     
         66 . The computer readable storage medium of  claim 64 , wherein the data file includes a run command, and wherein the instructions for benchmarking a window of a graphical user interface further cause the computing system to measure the time required for the run command to complete. 
     
     
         67 . The computer readable storage medium of  claim 64 , wherein the data file includes a run command for a particular view of the window, the particular view identifying a particular region of the generated window, and wherein the instructions for benchmarking a window of a graphical user interface further cause the computing system to benchmark a performance of the run command for the particular view. 
     
     
         68 . The computer readable storage medium of  claim 64 , wherein the particular set of commands include a run command and a reset command, wherein the instructions for benchmarking further include instructions to execute the run command for altering a state of the generated window from an original state to a current state and instructions to execute the reset command for altering the state of the window from the current state to the original state. 
     
     
         69 . The computer readable storage medium of  claim 64 , wherein the resource file and the data file are provided from different data files. 
     
     
         70 . The computer readable storage medium of  claim 69 , wherein the resource file is a resource file that defines a set of views of the window and the data file defines script commands, which when performed, change a state of the window. 
     
     
         71 . A computing device, comprising:
 a memory and a processor configured for generating a graphical user interface window for benchmarking the graphical user interface independent of generating the graphical user interface with a software application, the memory further configured for storing a first file, a second file, and instructions that, when executed by the processor, cause the computing device to:   generate the window based on a definition from the first file, the first file defining a plurality of views for the window;   perform a particular command defined in the second file, the particular command configured to change a particular view of the generated window from the particular view to another view associated with the command; and   benchmark a performance of the particular command by measuring a time required to complete the performance of the particular command in the second file of the window to change to the another view associated with the command.   
     
     
         72 . The computing device of  claim 71 , the memory further configured for storing instructions for execution that when executed by the processor cause the computing device to:
 perform a second command defined in the second file, to restore a view of the generated window to a state prior to performing the particular command; and   iteratively perform the particular command, benchmarking operations, and the second command a predetermined number of times.   
     
     
         73 . The computing device of  claim 71 , wherein the first file is a resource file that defines a set of views of the window and the second file is a data file that defines script commands, which when performed, change respective states of the set of views of the window.