Data acquisition system with on-demand and prioritized data fetching

Abstract

A networked computer system is configured to obtain case data from an online web portal dynamically and on-demand. Data acquisition modules can be configured to prefetch cases predicted to be of interest to users by a separate prediction module, and to resolve access control pages on the latter's behalf to reduce latency. The system also can be configured to exploit existing data connections to piggyback and retrieve additional data items likely to be of interest.

What is claimed is: 
     
         1 . A computing system for automatically expediting acquisition of case records from a web portal which includes an access control page requiring a human interactive proof to be satisfied prior to allowing access to the case records through an online database, the system comprising:
 one or more case acquisition worker routines executing on the computing system which are configured to retrieve case records through a first data link by automatically providing a query including on one or more search parameters inserted in a query page of said web portal for the online database;   each of said case acquisition worker routines further cooperating with an access control routine which is responsible for satisfying said human interactive proof test at the access control page to enable said first data link to said query page;   a case queue monitor routine executing on the computing system and configured to invoke and control said one or more case acquisition worker modules;   wherein said case queue monitor routine is configured to attempt to maintain at least one active link to said web portal query page using said one or more case acquisition worker routines;   a case requesting routine executing on the computing system and configured to receive and process a user first request for one or more of said case records from a web application or application program interface call;   wherein said case requesting routine cooperates with said case queue monitor routine and said one or more case acquisition worker routines to provide said case records for said user first request, and without requiring said user, said web application or application program interface call to satisfy said human interactive proof test at the access control page.   
     
     
         2 . The system of  claim 1  wherein said case acquisition worker routines are configured to locate additional case records specified by said case queue monitor routine from the online database beyond said user first request, and during a single data session enabled for said user first request through said access control page. 
     
     
         3 . The system of  claim 1  wherein said web portal query page includes only a query field that permits at most a single parameter to be specified for a query used for locating said case records, and is further limited such that only one case record can be retrieved for said query. 
     
     
         4 . The system of  claim 1  further including a prioritized acquisition queue which stores data access requests tagged with a corresponding access priority. 
     
     
         5 . The system of  claim 4  where said access priority is determined by a prediction routine based on observing a log of current user requests and calculating an identity and access value of a set of record records. 
     
     
         6 . The system of  claim 1  wherein said user first request specifies a range of cases spanning a particular numerical range and/or spanning a particular date range. 
     
     
         7 . The system of  claim 1  wherein said user first request is specified manually as a list of alerts desired by a user for one or more customized cases, which alerts are generated by the system whenever new data appears in said one or more customized cases. 
     
     
         8 . A computing system for automatically retrieving case records on demand from a web portal which includes an access control page requiring a human interactive proof to be satisfied prior to allowing access to the case records through an online database, the system comprising:
 a case record database coupled to the computing system that is configured to store case data from said case records, which case record database is separate from said online database;   wherein said case record data can be designated by the computing system as current or not current based on an age of said case record data;   one or more case acquisition worker modules executing on the computing system which are configured to retrieve case records through a first data link by automatically providing a query including on one or more search parameters inserted in a query page of said web portal for said online database;   each of said case acquisition worker modules further cooperating with an access control routine which is responsible for satisfying said human interactive proof test at the access control page to enable said first data link to said query page;   a case queue monitor routine executing on the computing system and configured to invoke and control said one or more case acquisition worker modules and to store said case data in said case record database;   wherein said case queue monitor routine is configured to attempt to maintain at least one active link to said web portal query page using said one or more case acquisition worker modules;   a case requesting routine executing on the computing system and configured to receive and process a user first request for one or more of said case records from a web application or application program interface call;   wherein said case requesting routine cooperates with said case queue monitor routine and said one or more case acquisition worker modules to provide said case records for said user first request, and is configured such that:   1) when case data in said case record database for said user first request is designated current, said user first request can be satisfied by accessing records in such database;   2) when case data in said case record database for said user first request is designated not current, said user first request can be satisfied by invoking said one or more case acquisition worker modules to retrieve corresponding data records from said online database through said at least one active link.   
     
     
         9 . The system of  claim 8  wherein said case acquisition worker modules are configured to locate additional case records specified by said case queue monitor routine from the online database beyond said user first request, and during a data session enabled for said user first request through said access control page. 
     
     
         10 . The system of  claim 8  wherein said case requesting routine is configured to cooperate with said case queue monitor routine and said one or more case acquisition worker modules to provide said case records for said user first request, and without requiring said user, said web application or application program interface call to satisfy said human interactive proof test at the access control page. 
     
     
         11 . The system of  claim 8  wherein said case requesting routine is further configured such that when case data in said case record database for said user first request is designated as not current, said user first request can be also satisfied by nonetheless presenting said case data from said case record as initial data; and said case requesting routine also being configured to invoke said one or more case acquisition worker modules to retrieve more current corresponding data records from said online database through said at least one active link to supplement said initial case data. 
     
     
         12 . A computing system for automatically predicting and acquiring case records from a web portal which includes an access control page requiring a human interactive proof to be satisfied prior to allowing access to the case records through an online database, the system comprising:
 one or more case acquisition worker modules executing on the computing system which are configured to retrieve case records through a first data link by automatically providing a query including on one or more search parameters inserted in a query page of said web portal for the online database;   a case queue monitor routine executing on the computing system and configured to invoke and control said one or more case acquisition worker modules;   wherein said case queue monitor routine is configured to attempt to maintain at least one active link to said web portal query page using said one or more case acquisition worker modules;   a case requesting routine executing on the computing system and configured to receive and process a user first request for one or more of said case records from a web application or application program interface call;   a case prediction routine executing on the computing system which is configured to predict and generate prefetchable data requests, including new case records at the web portal online database as well as generation of new data at the web portal online database for existing case records;   wherein said case prediction routine stores said prefetchable data requests in a prioritized case acquisition queue;   further wherein said prefetchable data requests are serviced by said one or more case acquisition worker modules in accordance with a priority specified for each of such requests.   
     
     
         13 . The system of  claim 12  wherein said case requesting routine is configured to cooperate with said case queue monitor routine and said one or more case acquisition worker routines to provide said case records for said user first request, and without requiring said user, said web application or application program interface call to satisfy said human interactive proof test at the access control page. 
     
     
         14 . The system of  claim 12  wherein said case requesting routine is further configured such that when case data in said case record database for said user first request is designated as not current, said user first request can be also satisfied by nonetheless presenting said case data from said case record as initial data; and said case requesting routine also being configured to invoke said one or more case acquisition worker modules to retrieve more current corresponding data records from said online database through said at least one active link to supplement said initial case data. 
     
     
         15 . The system of  claim 12  wherein said case prediction routine uses historical data and transaction events in a set of said online database records to determine an expected event time for such set of records, and such that said case prediction routine is automatically configured to inspect said set of records on or about said expected event time(s). 
     
     
         16 . The system of  claim 12  wherein said case prediction routine is configured to extract content from online publications to identify search parameters related to said case records, and is further configured to process said search parameters to extract query parameters for identifying said prefetchable data requests to said online database case records. 
     
     
         17 . A method of automatically expediting acquisition of case records with a computing system from a web portal which includes an access control page requiring a human interactive proof to be satisfied prior to allowing access to the case records through an online database, the method comprising:
 configuring one or more case acquisition worker routines executing on the computing system to retrieve case records through a first data link by automatically providing a query including on one or more search parameters inserted in a query page of said web portal for the online database;   automatically satisfying said human interactive proof test at the access control page to enable said first data link to said query page;   controlling said one or more case acquisition worker modules such that the computing system attempts to maintain at least one active link to said web portal query page using said one or more case acquisition worker routines;   processing a user first request for one or more of said case records from a web application or application program interface call;   providing said case records for said user first request without requiring said user, said web application or application program interface call to satisfy said human interactive proof test at the access control page.   
     
     
         18 . A method for automatically retrieving case records on demand with a computing system from a web portal which includes an access control page requiring a human interactive proof to be satisfied prior to allowing access to the case records through an online database, the method comprising:
 providing a case record database coupled to the computing system that is configured to store case data from said case records, which case record database is separate from said online database;   wherein said case record data is designated by the computing system as current or not current based on an age of said case record data;   configuring one or more case acquisition worker modules executing on the computing system to retrieve case records through a first data link by automatically providing a query including on one or more search parameters inserted in a query page of said web portal for said online database;   automatically satisfying said human interactive proof test at the access control page under control of the computing system to enable said first data link to said query page;   wherein said one or more case acquisition worker modules store said case data in said case record database;   maintaining at least one active link to said web portal query page with the computing system using said one or more case acquisition worker modules;   processing a user first request for one or more of said case records with the computing system from a web application or application program interface call and providing said case records for said user first request such that:   1) when case data in said case record database for said user first request is designated current, said user first request is satisfied by accessing records in such database;   2) when case data in said case record database for said user first request is designated not current, said user first request is satisfied by invoking said one or more case acquisition worker modules to retrieve corresponding data records from said online database through said at least one active link.   
     
     
         19 . The method of  claim 18  wherein said case acquisition worker modules are configured to locate additional case records specified by said case queue monitor routine from the online database beyond said user first request, and during a single data session enabled for said user first request through said access control page. 
     
     
         20 . A method of automatically predicting and acquiring case records with a computing system from a web portal which includes an access control page requiring a human interactive proof to be satisfied prior to allowing access to the case records through an online database, the method comprising:
 configuring one or more case acquisition worker modules executing on the computing system to retrieve case records through a first data link by automatically providing a query including on one or more search parameters inserted in a query page of said web portal for the online database;   maintaining at least one active link to said web portal query page with the computing system using said one or more case acquisition worker modules;   processing a user first request for one or more of said case records with the computing system from a web application or application program interface call;   automatically predicting and generating prefetchable data requests, including new case records at the web portal online database as well as generation of new data at the web portal online database for existing case records;   wherein said prefetchable data requests are stored in a prioritized case acquisition queue;   servicing said prefetchable data requests with said one or more case acquisition worker modules in accordance with a priority specified for each of such requests.