Grouping contacts using tiered warehouse levels

Abstract

Described herein are techniques for deploying a plurality of data processing pipelines to execute contact grouping queries to upload contact data into contact records of contact group tables in a cloud-based, virtual data warehouse. Each contact grouping query is assigned to a data pipeline of a plurality of data pipelines based on characteristics of the contact grouping query and the count of contact records associated with the entity on whose behalf the contact records are being maintained. Additionally, the execution runtime is obtained for each contact grouping query so that an average execution runtime for the contact grouping query can be cached. An expected execution runtime of all contact grouping queries allocated to a specific data pipeline can then be calculated and used in determining whether additional data pipelines may be necessary to meet a service level objective that indicates a time interval during which a contact grouping query is to be executed.

1 . (canceled) 
     
     
         2 . The computer-implemented method of claim  1 , wherein a contact grouping query is assigned a query classification of simple, when the contact grouping query does not include a SQL JOIN statement, and a contact grouping query is assigned a query classification of complex, when the contact grouping query does include a SQL JOIN statement. 
     
     
         3 . The computer-implemented method of claim  1 , wherein the predetermined size of the virtual data warehouse that the each data pipeline is configured to invoke determines a number of compute resources that the virtual data warehouse service will devote to executing contact grouping queries allocated to the data pipeline. 
     
     
         4 . The computer-implemented method of claim  1 , further comprising:
 subsequent to executing each contact grouping query to update a contact group table, updating a cache with data to indicate a contact grouping query execution runtime for the contact grouping query.   
     
     
         5 . The computer-implemented method of  claim 4 , further comprising:
 for a first contact grouping query, calculating an average contact grouping query execution runtime for a predetermined number of prior contact grouping query executions of the first contact grouping query; and   updating a cache to indicate the average contact grouping query execution runtime for the predetermined number of prior contact grouping query executions of the first contact grouping query.   
     
     
         6 . The computer-implemented method of  claim 5 , further comprising:
 calculating an expected total execution runtime, for all contact grouping queries allocated to a specific data pipeline in the plurality of data pipelines, using for the calculation the average contact grouping query execution runtime for each contact grouping query;   comparing the expected total execution runtime for all contact grouping queries allocated to the specific data pipeline to a time interval indicated in a service level objective; and   adding a new data pipeline configured to invoke a virtual data warehouse that is the same fixed size as the virtual data warehouse that the specific data pipeline is configured to invoke.   
     
     
         7 . The computer-implemented method of claim  1 , further comprising:
 subsequent to executing a first contact grouping query to update a first contact group table, updating a cache with data to indicate the count of contact records for the entity associated with the first contact grouping query.   
     
     
         8 . (canceled) 
     
     
         9 . (canceled) 
     
     
         10 . The system of claim  9 , wherein a contact grouping query is assigned a query classification of simple, when the contact grouping query does not include a SQL JOIN statement, and a contact grouping query is assigned a query classification of complex, when the contact grouping query does include a SQL JOIN statement. 
     
     
         11 . The system of claim  9 , wherein the predetermined size of the virtual data warehouse that each data pipeline is configured to invoke determines a number of compute resources that the virtual data warehouse service will devote to executing contact grouping queries allocated to the data pipeline. 
     
     
         12 . The system of claim  9 , wherein the operations further comprise:
 subsequent to executing each contact grouping query to update a contact group table, updating a cache with data to indicate a contact grouping query execution runtime for the contact grouping query.   
     
     
         13 . The system of  claim 12 , wherein the operations further comprise:
 for a first contact grouping query, calculating an average contact grouping query execution runtime for a predetermined number of prior contact grouping query executions executions of the first contact grouping query; and   updating a cache to indicate the average contact grouping query execution runtime for the predetermined number of prior contact grouping query executions of the first contact grouping query.   
     
     
         14 . The system of  claim 13 , wherein the operations further comprise:
 calculating an expected total execution runtime, for all contact grouping queries allocated to a specific data pipeline in the plurality of data pipelines, using for the calculation the average contact grouping query execution runtime for each contact grouping query;   comparing the expected total execution runtime for all contact grouping queries allocated to the specific data pipeline to a time interval indicated in a service level objective; and   adding a new data pipeline configured to invoke a virtual data warehouse that is the same fixed size as the virtual data warehouse that the specific data pipeline is configured to invoke.   
     
     
         15 . The system of claim  9 , wherein the operations further comprise:
 subsequent to executing a first contact grouping query to update a first contact group table, updating a cache with data to indicate the count of contact records for the entity associated with the first contact grouping query.   
     
     
         16 . (canceled) 
     
     
         17 . The system of claim  16 , wherein a contact grouping query is assigned a contact grouping query classification of simple, when the contact grouping query does not include a SQL JOIN statement, and a contact grouping query is assigned a contact grouping query classification of complex, when the contact grouping query does include a SQL JOIN statement. 
     
     
         18 . The system of claim  16 , wherein the predetermined size of the virtual data warehouse that the each data pipeline is configured to invoke determines a number of compute resources that the virtual data warehouse service will devote to executing contact grouping queries allocated to the data pipeline. 
     
     
         19 . The system of claim  23 , further comprising:
 subsequent to executing each contact grouping query to update a contact group table, means for updating a cache with data to indicate a contact grouping query execution runtime for the contact grouping query.   
     
     
         20 . The system of  claim 18 , further comprising:
 for a first contact grouping query, means for calculating an average contact grouping query execution runtime for a predetermined number of prior contact grouping query executions executions of the first contact grouping query; and   means for updating a cache to indicate the average contact grouping query execution runtime for the predetermined number of prior contact grouping query executions of the first contact grouping query.   
     
     
         21 . A computer-implemented method for managing contact grouping queries on a Software-as-a-Service (SaaS) platform, the method comprising:
 maintaining a plurality of contact grouping queries for a plurality of entities, each contact grouping query associated with an entity and each contact grouping query, when executed, to update contact records for the entity in a contact group table stored in a database of a virtual data warehouse service;   maintaining a plurality of data processing pipelines, each data processing pipeline to invoke, via the cloud-based virtual data warehouse service and according to a predefined schedule, a virtual data warehouse of a predetermined size to execute contact grouping queries to update contact group tables in the database;   maintaining a mapping of contact grouping queries to sizes of virtual data warehouses, the mapping of each contact grouping query to a size of a virtual data warehouse based on a combination of i) a count of contact records for the entity associated with the contact grouping query, and ii) a query classification for the contact grouping query;   assigning the execution of each contact grouping query to a data processing pipeline in the plurality of data processing pipelines based on the mapping; and   according to the predefined schedule, invoking a first virtual data warehouse of a first size and executing a plurality of contact grouping queries allocated to the first data processing pipeline with the first virtual data warehouse to update contact records in contact grouping tables.   
     
     
         22 . A system comprising:
 a processor for executing computer-readable instructions; and   a memory storage device storing instructions thereon, which, when executed by the processor, cause the system to perform operations comprising:   maintaining a plurality of contact grouping queries for a plurality of entities, each contact grouping query associated with an entity and each contact grouping query, when executed, to update contact records for the entity in a contact group table stored in a database of a virtual data warehouse service;   maintaining a plurality of data processing pipelines, each data processing pipeline to invoke, via the virtual data warehouse service and according to a predefined schedule, a virtual data warehouse of a predetermined size to execute contact grouping queries to update contact group tables in the database;   maintaining a mapping of contact grouping queries to sizes of virtual data warehouses, the mapping of each contact grouping query to a size of a virtual data warehouse based on a combination of i) a count of contact records for the entity associated with the contact grouping query, and ii) a query classification for the contact grouping query;   assigning the execution of each contact grouping query to a data processing pipeline in the plurality of data processing pipelines based on the mapping; and   according to the predefined schedule, invoking a first virtual data warehouse of a first size and executing a plurality of contact grouping queries allocated to the first data processing pipeline with the first virtual data warehouse to update contact records in contact grouping tables.   
     
     
         23 . A system comprising:
 means for storing a plurality of contact grouping queries for a plurality of entities, each contact grouping query associated with an entity and each contact grouping query, when executed, to update contact records for the entity in a contact group table stored in a database of a virtual data warehouse service;   means for facilitating a plurality of data processing pipelines, each data processing pipeline to invoke, via the virtual data warehouse service and according to a predefined schedule, a virtual data warehouse of a predetermined size to execute contact grouping queries to update contact group tables in the database;   means for mapping of contact grouping queries to sizes of virtual data warehouses, the mapping of each contact grouping query to a size of a virtual data warehouse based on a combination of i) a count of contact records for the entity associated with the contact grouping query, and ii) a query classification for the contact grouping query;   means for assigning the execution of each contact grouping query to a data processing pipeline in the plurality of data processing pipelines based on the mapping; and   means for invoking, according to the predefined schedule, a first virtual data warehouse of a first size and executing a plurality of contact grouping queries allocated to the first data processing pipeline with the first virtual data warehouse to update contact records in contact grouping tables.