Managed query execution platform, and methods thereof

Abstract

A query execution fabric is provided. The a plurality of computing machines; and a plurality of storage devices communicatively connected to the plurality of computing machines using a storage communication protocol and configured to maintain a plurality of indexed views comprising index information and a plurality of data chunks, wherein the index information includes at least one local index per data chunk and at least one global index per each of the indexed views of the plurality of indexed views, and wherein each of the plurality of computing machines is configured to: verify a global index to locate segments of the plurality of indexed views; and verify a plurality of local indexes to locate the data chunks containing values related to a plurality of statements.

What is claimed is: 
     
         1 . A query execution fabric, comprising:
 a plurality of computing machines; and   a plurality of storage devices communicatively connected to the plurality of computing machines using a storage communication protocol and configured to maintain a plurality of indexed views comprising index information and a plurality of data chunks, wherein the index information includes at least one local index per data chunk and at least one global index per each of the indexed views of the plurality of indexed views, and   wherein each of the plurality of computing machines is configured to:
 verify a global index to locate segments of the plurality of indexed views; and 
 verify a plurality of local indexes to locate the data chunks containing values related to a plurality of statements. 
   
     
     
         2 . The query execution fabric of  claim 1 , wherein each of the plurality of computing machines is further configured to:
 execute queries on the plurality of the indexed views.   
     
     
         3 . The query execution fabric of  claim 3 , wherein each of the plurality of computing machines is configured to:
 process an input query by accessing the data chunks located by the plurality of local indexes; and   respond to the input query based on the accessed data chunks.   
     
     
         4 . The query execution fabric of  claim 1 , wherein each of the indexed views of the plurality of indexed views is a presentation of customer data stored in a data lake. 
     
     
         5 . The query execution fabric of  claim 2 , wherein each of the plurality of computing machines further comprises:
 a query engine configured to parse each of the queries into a plurality of statements and to process the parsed statements on the plurality of indexed views.   
     
     
         6 . The query execution fabric of  claim 1 , wherein each of the indexed views of the plurality of indexed views is segmented into a plurality of ordered vertical segments, wherein the plurality of data chunks is based on intersections of the vertical segments and each column in the indexed views. 
     
     
         7 . The query execution fabric of  claim 1 , wherein the index information comprises:
 at least one local index per data chunk;   at least one meta-index per column; and   and at least one global index per the indexed view.   
     
     
         8 . The query execution fabric of  claim 1 , wherein the plurality of computing machines is configured to:
 access the plurality of local indexes associated with the plurality of data chunks of respective columns in parallel, wherein the plurality of data indexes is independent of each other.   
     
     
         9 . The query execution fabric of  claim 1 , wherein each of the plurality of storage devices comprises: a plurality of non-volatile memory pages, wherein each memory page is configured to maintain at least one local index and least one data chunk. 
     
     
         10 . The query execution fabric of  claim 4 , wherein the plurality of computing machines is configured to synchronize data between the plurality of indexed views and the data stored in the data lake. 
     
     
         11 . The query execution fabric of  claim 11 , wherein the synchronization is performed in real-time or at near real-time. 
     
     
         12 . The query execution fabric of  claim 11 , wherein the synchronization of data between the plurality of indexed views and the data stored in the data lake further comprises: detection of changes in the data stored in the data lake. 
     
     
         13 . The query execution fabric of  claim 2 , wherein the queries executed on the plurality of the indexed views comprise SQL queries and commands. 
     
     
         14 . The query execution fabric of  claim 1 , wherein at least one computing machine of the plurality of the computing machines is configured to orchestrate operations of the plurality of the computing machines. 
     
     
         15 . The query execution fabric of  claim 1 , wherein one or more of the plurality of the computing machines is allocated based on a demand. 
     
     
         16 . The query execution fabric of  claim 1 , wherein the storage communication protocol is any one of:
 remote direct memory access (RDMA) over converged ethernet (RoCE);   internet wide area RDMA protocol (iWARP);   non-volatile memory express (NVMe);   NVMe over fabric (NVMeF); and   NVMeF over TCP.   
     
     
         17 . The query execution fabric of  claim 1 , wherein the plurality of the computing machines and the plurality of storage devices is deployed in a cloud computing environment.