System and method for retrieving data using agents in a distributed network

Abstract

A method and apparatus for data retrieval by a computing system and a plurality of agent computers in a distributed network is disclosed. The computing system sends a request to each agent computer to perform a search at a node. The agents perform the searches. The agents thereupon send the resulting data to the computing system for storage in a central database.

1 . A computing system for retrieving data from a node using a plurality of agent computers in a distributed network, comprising:
 a memory system for storing code; and
 a processing system associated with the memory system and configured to run the code, wherein the code when run is configured to:
 deliver a request to each agent computer to retrieve data at the node; 
 receive from each agent computer the data obtained in response to the request; and 
 store the data in a database. 
 
   
     
     
         2 . The computing system of  claim 1  wherein the code is further configured to distribute client-executable code to each agent computer for execution at each agent computer, the client-executable code being configured to send to the computing system the data retrieved by the agent computer in response to the request. 
     
     
         3 . The computing system of  claim 1  wherein the code is further configured to receive from the agent computer an identity of an asset, wherein the data to be retrieved comprises a characteristic of the asset. 
     
     
         4 . The computing system of  claim 1  wherein the code is further configured to:
 attempt to retrieve the data at the node prior to delivering the request; and 
 determine that the node has prevented the computing system from retrieving the data. 
 
     
     
         5 . The computing system of  claim 1  wherein the node comprises a website. 
     
     
         6 . The computing system of  claim 5  wherein the agent computers are configured to retrieve data from one or more web pages on the website. 
     
     
         7 . The computing system of  claim 3  wherein the asset comprises a ticket for air travel. 
     
     
         8 . The computing system of  claim 1  wherein the code is further configured to:
 receive from a first agent computer an identity of an asset; 
 deliver the request to a second agent computer to retrieve the data at the node, the data comprising a characteristic of the asset; and 
 receive the requested data from the second agent computer. 
 
     
     
         9 . The computing system of  claim 8  wherein the code is further configured to send the data to the first agent computer. 
     
     
         10 . The computing system of  claim 1  wherein the code is configured when run to:
 receive an identity of a product from one of the agent computers; and 
 determine that another one of the agent computers is unable to execute the request at a specified time. 
 
     
     
         11 . An article comprising a machine-readable medium including code including machine-executable instructions, the instructions operative to cause a machine in a distributed network of machines to:
 receive from a central computing system a query for data at a node;   query the node for the data; and   return the data to the central computing system for storage in a database.   
     
     
         12 . The article of  claim 11  wherein the instructions are further operative to cause the machine to send to the central computing system an identity of an asset, the data comprising a characteristic of the asset. 
     
     
         13 . The article of  claim 11  wherein the instructions are further operative to cause the machine to send an identity of the node to the central computing system. 
     
     
         14 . The article of  claim 11  wherein the data comprises a price of an asset. 
     
     
         15 . The article of  claim 11  wherein the code is further operative to cause the machine to execute the query on behalf of another machine in response to a request received from the central computer. 
     
     
         16 . The article of  claim 11  wherein the node comprises a website. 
     
     
         17 . The computer system of  claim 3  wherein the data comprises a price of an asset. 
     
     
         18 . The computer system of  claim 3  wherein the code is further operative to cause the machine to execute the query on behalf of another machine in response to a request received from the central computer.