System & Method For Acquiring Online Data By Mimicking Human Behavior

Abstract

A networked computer system includes a number of different data acquisition components for obtaining case data from an online web portal. The acquisition scanner can be configured in a number of different ways, including so as to imitate a human browser behavior, initiate separate instantiations for better case control, and maintain a download log to improve efficiency and performance.

What is claimed is: 
     
         1 . A computing system for automatically acquiring case records comprising:
 a case application scanner routine, which is adapted to run as a process as an independent thread on the computing system; said scanner routine further being configured to control an instantiation of an Internet browser for retrieving case records at a database accessible through a query page of a web portal;   wherein said web portal case records are not accessible until a human interactive proof test has been satisfied;   an access control routine configured to cooperate with said case application scanner routine which is adapted to engage with and satisfy said human interactive proof test and permit access to said web portal case records by said case application scanner routine;   said scanner routine being further configured to interact with tabs, lists, links and buttons so as to retrieve said case record, and case record data from any and all pages associated with said first case including a data presentation page for said case records;   wherein the system uses an independent and separate case application scanner routine for each case record to be scanned.   
     
     
         2 . The system of  claim 1  wherein said scanner routine is adapted to select tabs, links and buttons at a rate that is comparable to a human selection rate. 
     
     
         3 . The system of  claim 1  wherein said query page includes only a query field that permits only a single parameter to be specified for a query used for locating said case records, and only one case record can be retrieved for said query. 
     
     
         4 . The system of  claim 1  further including a scheduler routine configured to manage and trigger activities of a batch of independent and separate application scanner routines. 
     
     
         5 . A computing system for automatically acquiring case records from a non-indexed database through a webpage comprising:
 a case application scanner routine, which is adapted to run as a process as an independent thread on the computing system; said scanner routine further being configured to control an instantiation of an Internet browser for retrieving case records at a database accessible through a query page of a web portal;   wherein said web portal case records are not accessible until a human interactive proof test has been satisfied;   an access control routine configured to cooperate with said case application scanner routine which is adapted to engage with and satisfy said human interactive proof test and permit access to said web portal case records by said case application scanner routine;   said scanner routine being further configured to interact with tabs, lists, links and buttons so as to retrieve said case record, and case record data from any and all pages associated with said first case including a data presentation page for said case records;   wherein said scanner routine maintains a history log for each case record which is accessible during a data capture session so that case record data is retrieved and stored by the system only for case records which have changed since a prior data capture.   
     
     
         6 . The system of  claim 5  wherein said scanner routine is adapted to select tabs, links and buttons at a rate that is comparable to a human selection rate. 
     
     
         7 . The system of  claim 5  wherein said scanner routine is adapted to request individual case records from said query page at a rate that is comparable to a human request rate. 
     
     
         8 . A computing system for acquiring case records from a non-indexed database through a webpage comprising:
 a case application scanner routine, which is adapted to run as a process as an independent thread on the computing system; said scanner routine further being configured to control an instantiation of an Internet browser for retrieving case records at a database accessible through a query page of a web portal;   wherein said web portal case records are not accessible until a human interactive proof test has been satisfied;   an access control routine configured to cooperate with said case application scanner routine which is adapted to engage with and satisfy said human interactive proof test and permit access to said web portal case records by said case application scanner routine;   said scanner routine being further configured to interact with tabs, lists, links and buttons so as to retrieve said case record, and case record data from any and all pages associated with said first case including a data presentation page for said case records;   further wherein said scanner routine is configured to mimic a human browsing behavior during said interacting and so as to satisfy a data access limit or data retrieval rate limit specified by said web portal.   
     
     
         9 . The system of  claim 8  wherein said scanner routine is adapted to select tabs, links and buttons at a rate that is comparable to a human selection rate. 
     
     
         10 . The system of  claim 8  wherein said scanner routine is adapted to request individual case records from said query page at a rate that is comparable to a human request rate.