Real-time distributed in memory search architecture

Abstract

Disclosed here are distributed computing system connection configurations having multiple connection bandwidth and latency tiers. Also disclosed are connection configurations including a suitable number of network segments, where network segments may be connected to external servers and clusters including search managers, analytics agents, search conductors, dependency managers, supervisors, and partitioners, amongst others. In one or more embodiments, modules may be connected to the network segments using a desired bandwidth and latency tier. Disclosed here are hardware components suitable for running one or more types of modules on one or more suitable nodes. One or more suitable hardware components included in said clusters include CPUs, Memory, and Hard Disk, amongst others.

What is claimed is: 
     
         1 . A system comprising:
 a node hosting an in-memory database, wherein the in-memory database comprises:
 a search manager in communication with a first network segment and a second network segment via a first type of communication link; 
 an analytics agent in communication with the second network segment via the first type of communication link; 
 a search conductor in communication with the second network segment via a second type of communication link and a third network segment via a third type of communication link, wherein the first type of communication link is of higher bandwidth tier than the second type of communication link, wherein the second type of communication link is of higher bandwidth tier than the third type of communication link, wherein the first type of communication link and the second type of communication link are of a first latency tier; 
 a partitioner in communication with the third network segment via the third type of communication link; 
 a dependency manager in communication with the second network segment and the third network segment via a fourth type of communication link, wherein the third type of communication link is of higher bandwidth tier than the fourth type of communication link, wherein the third type of communication link and the fourth type of communication link are of a second latency tier, wherein the first latency tier is of latency lower than the second latency tier; and 
 a supervisor in communication with the second network segment and the third network segment via the fourth type of communication link. 
   
     
     
         2 . The system of  claim 1 , wherein the first network segment, the second network segment, the third network segment, the search manager, the analytics agent, the search conductor, the partitioner, the dependency manager, and the supervisor define a network which communicates with a server outside of the network via at least one of the first type of communication link, the second type of communication link, or the third type of communication link. 
     
     
         3 . The system of  claim 1 , wherein the search manager is configured to generate a search conductor query based on a user query received from a user computing device, to send the search conductor query to the search conductor, and to send a search result to the analytics agent based on a receipt of the search result from the search conductor, wherein the search result is based on the search conductor query. 
     
     
         4 . The system of  claim 3 , wherein the search conductor is configured to query a collection containing a record of the in-memory database based on the search conductor query received from the search manager and to send the search result to the search manager based on an identification of the search result, wherein the record is stored at a storage node comprising a main memory, wherein the search result contains data which satisfies the search conductor query. 
     
     
         5 . The system of  claim 4 , wherein the partitioner is configured to partition the collection and to distribute the collection onto at least the storage node based on a schema file and a receipt of the collection. 
     
     
         6 . The system of  claim 3 , wherein the analytics agent is configured to perform an analytics algorithm based on a receipt of the search result from the search manager. 
     
     
         7 . The system of  claim 1 , wherein the search manager, the search conductor, and the analytics agent define a cluster. 
     
     
         8 . The system of  claim 1 , wherein the supervisor is configured to perform node status monitoring periodically and to send a configuration package file to a failed node based on a receipt of a status indicating a failed resource of the failed node. 
     
     
         9 . The system of  claim 1 , wherein the dependency manager stores a configuration package file and a dependency tree, wherein the dependency manager is configured to determine the configuration package file for a failed node based on the dependency tree and to send the configuration package file to the supervisor based on a receipt of a request for the configuration package file from the supervisor. 
     
     
         10 . The system of  claim 9 , wherein the supervisor sends the request for the configuration package file based on the supervisor receiving a message informative of the failed node. 
     
     
         11 . A method comprising:
 configuring, by a computer, a search manager to be in communication with a first network segment and a second network segment via a first type of communication link;   configuring, by the computer, an analytics agent to be in communication with the second network segment via the first type of communication link;   configuring, by the computer, a search conductor to be in communication with the second network segment via a second type of communication link and a third network segment via a third type of communication link, wherein the first type of communication link is of higher bandwidth tier than the second type of communication link, wherein the second type of communication link is of higher bandwidth tier than the third type of communication link, wherein the first type of communication link and the second type of communication link are of a first latency tier;   configuring, by the computer, a partitioner to be in communication with the third network segment via the third type of communication link;   configuring, by the computer, a dependency manager to be in communication with the second network segment and the third network segment via a fourth type of communication link, wherein the third type of communication link is of higher bandwidth tier than the fourth type of communication link, wherein the third type of communication link and the fourth type of communication link are of a second latency tier, wherein the first latency tier is of latency lower than the second latency tier;   configuring, by the computer, a supervisor to be in communication with the second network segment and the third network segment via the fourth type of communication link, wherein a node hosts an in-memory database, wherein the in-memory database comprises the search manager, the analytics agent, the search conductor, the partitioner, the dependency manager, and the supervisor.   
     
     
         12 . The method of  claim 11 , wherein the first network segment, the second network segment, the third network segment, the search manager, the analytics agent, the search conductor, the partitioner, the dependency manager, and the supervisor define a network which communicates with a server outside of the network via at least one of the first type of communication link, the second type of communication link, or the third type of communication link. 
     
     
         13 . The method of  claim 11 , further comprising:
 generating, by the search manager, a search conductor query based on a user query received from a user computing device;   sending, by the search manager, the search conductor query to the search conductor;   sending a search result to the analytics agent based on a receipt of the search result from the search conductor, wherein the search result is based on the search conductor query.   
     
     
         14 . The method of  claim 13 , further comprising:
 querying, by the search conductor, a collection containing a record of the in-memory database based on the search conductor query received from the search manager;   sending, by the search conductor, the search result to the search manager based on an identification of the search result, wherein the record is stored at a storage node comprising a main memory, wherein the search result contains data which satisfies the search conductor query.   
     
     
         15 . The method of  claim 14 , further comprising:
 partitioning, by the partitioner, the collection;   distributing, by the partitioner, the collection onto at least the storage node based on a schema file and a receipt of the collection.   
     
     
         16 . The method of  claim 13 , further comprising:
 performing, by the analytics agent, an analytics algorithm based on a receipt of the search result from the search manager.   
     
     
         17 . The method of  claim 11 , wherein the search manager, the search conductor, and the analytics agent define a cluster. 
     
     
         18 . The method of  claim 11 , further comprising:
 performing, by the supervisor, node status monitoring periodically;   sending, by the supervisor, a configuration package file to a failed node based on a receipt of a status indicating a failed resource of the failed node.   
     
     
         19 . The method of  claim 11 , further comprising:
 storing, by the dependency manager, a configuration package file and a dependency tree;   determining, by the dependency manager, the configuration package file for a failed node based on the dependency tree;   sending, by the dependency manager, the configuration package file to the supervisor based on a receipt of a request for the configuration package file from the supervisor.   
     
     
         20 . The system of  claim 19 , further comprising:
 sending, by the supervisor, the request for the configuration package file based on the supervisor receiving a message informative of the failed node.