Virtual data source manager of data virtualization-based architecture

Abstract

Techniques are disclosed relating to managing virtual data sources (VDSs), including creating and using VDSs. A virtual data source manager (VDSM) that is executing on a computer system may receive a VDS creation request to create a particular VDS. The VDS creation request may include information that is usable to identify a data origin of the particular VDS and an ingestion routine. The VDSM may use the ingestion routine to create the particular VDS that includes a bitmap index that is representative of data stored at the origin. In response to a data request corresponding to the origin, the particular VDS may be used to initiate a query to the particular VDS for data without accessing the origin. The VDSM may further maintain various software plugins that are executable to perform corresponding functions in relation to the VDSM and the VDSs managed by the VDSM.

1 - 20 . (canceled) 
     
     
         21 . A method, comprising:
 maintaining, by a computer system, a bitmap index comprising, in a bitmap format, a data representation of particular data stored at an origin data source;   storing, by the computer system, plugin information that associates one or more software plugins with the bitmap index, wherein a given software plugin of the one or more software plugins is executable to perform a set of operations in relation to the bitmap index;   implementing, by the computer system, an application programming interface (API) that permits invocation of the one or more software plugins;   receiving, by the computer system and via the API, a request to invoke a particular one of the one or more software plugins; and   causing, by the computer system, execution of the particular software plugin with respect to the bitmap index.   
     
     
         22 . The method of  claim 21 , wherein the particular software plugin generates a result from one or more machine learning models executed against data included in the bitmap index. 
     
     
         23 . The method of  claim 21 , further comprising:
 maintaining, by the computer system, a pool of resources to facilitate functionality of the computer system, wherein the causing of the execution of the particular software plugin includes:
 allocating a compute node from the pool of resources; and 
 instantiating the particular software plugin on the compute node. 
   
     
     
         24 . The method of  claim 21 , further comprising:
 maintaining, by the computer system, information pertaining to a plugin library of software plugins executable to perform respective operations in relation to a given bitmap index; and   causing, by the computer system, presentation of a user interface to a user via a user device, wherein the user interface displays software plugins from the plugin library, and wherein the plugin information corresponds to a selection of the one or more software plugins by the user.   
     
     
         25 . The method of  claim 21 , wherein the particular software plugin is an ingest software plugin that accesses data of the origin data source and converts the accessed data from a data format used at the origin data source to the bitmap format of the bitmap index. 
     
     
         26 . The method of  claim 25 , wherein the origin data source is located externally to the computer system, and wherein the causing of the execution of the particular software plugin includes:
 causing the particular software plugin to be executed on a computer system of a local-area network that includes the origin data source.   
     
     
         27 . The method of  claim 21 , wherein the particular software plugin is a security software plugin that prevents accesses of data included in the bitmap index that are not compliant with a specified set of security policies. 
     
     
         28 . The method of  claim 21 , wherein the bitmap index is associated with at least one different software plugin than a different bitmap index. 
     
     
         29 . A non-transitory computer-readable medium having program instructions stored thereon that are executable to cause a computer system to perform operations comprising:
 maintaining a bitmap index comprising a data representation of particular data stored at an origin data source;   storing plugin information that associates one or more software plugins with the bitmap index, wherein a given software plugin of the one or more software plugins is executable to perform a set of operations in relation to the bitmap index;   implementing an application programming interface (API) that permits invocation of the one or more software plugins;   receiving, via the API, a request to invoke a particular one of the one or more software plugins; and   causing execution of the particular software plugin with respect to the bitmap index.   
     
     
         30 . The non-transitory computer-readable medium of  claim 29 , wherein the particular software plugin uses one or more machine learning models to produce one or more results from the bitmap index. 
     
     
         31 . The non-transitory computer-readable medium of  claim 29 , wherein the operations further comprise:
 maintaining a pool of resources to facilitate functionality of the computer system, wherein the causing of the execution of the particular software plugin includes:
 allocating a compute node from the pool of resources; and 
 instantiating the particular software plugin on the compute node. 
   
     
     
         32 . The non-transitory computer-readable medium of  claim 29 , wherein the particular software plugin is a monitoring software plugin that collects a set of metrics pertaining to the bitmap index. 
     
     
         33 . The non-transitory computer-readable medium of  claim 29 , wherein the operations further comprise:
 receiving a creation request to create the bitmap index based on the origin data source and an ingestion routine; and   spawning an ingestion software plugin to implement the ingestion routine to ingest, from the origin data source, the particular data into the bitmap index.   
     
     
         34 . The non-transitory computer-readable medium of  claim 29 , wherein the operations further comprise:
 causing presentation of a user interface, wherein the user interface permits a user to invoke a set of functions of the API by selecting one or more options displayed in the user interface.   
     
     
         35 . A system, comprising:
 at least one processor; and   memory having program instructions stored thereon that are executable by the at least one processor to cause the system to perform operations comprising:
 maintaining a bitmap index having a bitmap representation of particular data stored at an origin data source; 
 storing plugin information that associates one or more software plugins with the bitmap index, wherein a given software plugin of the one or more software plugins is executable to perform a set of operations in relation to the bitmap index; 
 implementing an application programming interface (API) that permits invocation of the one or more software plugins; 
 receiving, via the API, a first request to invoke a particular one of the one or more software plugins; and 
 executing the particular software plugin with respect to the bitmap index. 
   
     
     
         36 . The system of  claim 35 , wherein the particular software plugin executes a machine learning model against data included in the bitmap index. 
     
     
         37 . The system of  claim 35 , wherein the executing of the particular software plugin includes:
 allocating a compute node from a pool of resources maintained by the system; and   instantiating the particular software plugin on the compute node.   
     
     
         38 . The system of  claim 35 , wherein the particular software plugin is a security software plugin that establishes a secure connection for sending data from the bitmap index to an endpoint system. 
     
     
         39 . The system of  claim 35 , wherein the particular software plugin is executable to convert a query from a structured query language (SQL) format into a query format that is executable against the bitmap index. 
     
     
         40 . The system of  claim 35 , wherein the bitmap index is associated with at least one different software plugin than a different bitmap index.