Automated application interaction using a virtual operator

Abstract

A computer-implemented method for automating interaction with a computer system includes linking a control computer system to an input interface and to an output interface of a client computer system, which is operative for producing user interface images on a display device. The control computer system executes distinct software modules that include a virtual operator for simulating actions of a human operator. Execution of the software modules causes the control computer system to capture an image from the output interface, and to recognize information in the image. In response to the information, the virtual operator controls an input device to automatically execute predetermined operations on the client computer system via the input interface.

1 . A computer-implemented method for automating interaction with a computer system comprising:
 linking a control computer system to an input interface of a client computer system and to an output interface of the client computer system operative for producing user interface images on a display device; and   executing with the control computer system distinct software modules that include a virtual operator for simulating actions of a human operator, wherein execution of the software modules causes the control computer system to perform the steps of:
 capturing an image from the output interface; 
 recognizing information in the image; and 
 responsively to the information, controlling an input device by the virtual operator to thereby automatically execute predetermined operations on the client computer system via the input interface. 
   
     
     
         2 . The method according to  claim 1 , wherein capturing the image from the output interface comprises receiving the image from an image scanner. 
     
     
         3 . The method according to  claim 1 , wherein recognizing information in the image comprises verifying that the recognized information matches a pattern that is known to the virtual operator. 
     
     
         4 . The method according to  claim 1 , wherein the predetermined operations comprise entering data. 
     
     
         5 . The method according to  claim 1 , wherein recognizing information comprises optical character recognition. 
     
     
         6 . The method according to  claim 1 , wherein the client computer system comprises a plurality of client computers, and the virtual operator comprises a plurality of instances of the virtual operator, the method comprising coordinating the plurality of instances of the virtual operator to concurrently control the plurality of client computers. 
     
     
         7 . The method according to  claim 1 , wherein recognizing information in the image comprises providing a verification image comprising the recognized information to the human operator. 
     
     
         8 . The method according to  claim 1 , wherein the predetermined operations comprise executing data input scripts comprising simulated device events. 
     
     
         9 . A computer software product for automating interaction with a computer system, comprising a computer storage medium in which computer program instructions are stored, wherein the instructions comprise distinct modules that include a virtual operator for simulating actions of a human operator, which instructions, when executed by a control computer linked to an input interface of a client computer system and to an output interface of the client computer system operative for producing user interface images on a display device, cause the control computer to perform the steps of:
 capturing an image from the output interface;   recognizing information in the image; and   responsively to the information, controlling an input device by the virtual operator to thereby automatically execute predetermined operations on the client computer system via the input interface.   
     
     
         10 . The computer software product according to  claim 9 , wherein capturing the image from the output interface comprises receiving the image from an image scanner. 
     
     
         11 . The computer software product according to  claim 9 , wherein recognizing information in the image comprises verifying that the recognized information matches a pattern that is known to the virtual operator. 
     
     
         12 . The computer software product according to  claim 9 , wherein the predetermined operations comprise entering data. 
     
     
         13 . The computer software product according to  claim 9 , wherein recognizing information comprises optical character recognition. 
     
     
         14 . The computer software product according to  claim 9 , wherein the client computer system comprises a plurality of client computers, and the virtual operator comprises a plurality of instances of the virtual operator, the method comprising coordinating the plurality of instances of the virtual operator to concurrently control the plurality of client computers. 
     
     
         15 . The computer software product according to  claim 9 , wherein recognizing information in the image comprises providing a verification image comprising the recognized information to the human operator. 
     
     
         16 . The computer software product according to  claim 9 , wherein the predetermined operations comprise executing data input scripts comprising simulated device events. 
     
     
         17 . A data processing system automating interaction with a computer system, comprising:
 a client computer system operative for producing user interface images on a display device;   a control computer system linked to an input interface and to an output interface of the client computer system;   a processor; and   a memory accessible to the processor storing programs and data objects therein, wherein execution of the programs causes the processor to execute with the control computer system distinct software modules that include a virtual operator for simulating actions of a human operator, and to perform the steps of:
 capturing an image from the output interface; 
 recognizing information in the image; and 
 responsively to the information, controlling an input device by the virtual operator to thereby automatically execute predetermined operations on the client computer system via the input interface. 
   
     
     
         18 . The data processing system according to  claim 17 , wherein capturing the image from the output interface comprises receiving the image from an image scanner. 
     
     
         19 . The data processing system according to  claim 17 , wherein recognizing information in the image comprises verifying that the recognized information matches a pattern that is known to the virtual operator. 
     
     
         20 . The data processing system according to  claim 17 , wherein the predetermined operations comprise entering data.