System and method for performing live partitioning in a data store

Abstract

A system that implements a scalable data storage service may maintain tables in a data store on behalf of storage service clients. The service may maintain table data in multiple replicas of partitions that are stored on respective computing nodes in the system. In response to detecting an anomaly in the system, detecting a change in data volume on a partition or service request traffic directed to a partition, or receiving a service request from a client to split a partition, the data storage service may create additional copies of a partition replica using a physical copy mechanism. The data storage service may issue a split command defined in an API for the data store to divide the original and additional replicas into multiple replica groups, and to configure each replica group to maintain a respective portion of the table data that was stored in the partition before the split.

1 - 35 . (canceled) 
     
     
         36 . A system, comprising:
 a plurality of computing devices, respectively comprising at least one processor and a memory, that implement a non-relational database service;   wherein the non-relational database service is configured to:
 automatically create two or more new partitions of a database hosted by a non-relational database service responsive to detecting an increase in size of data stored in a current partition of the database, wherein the current partition of the database is stored on a plurality of storage nodes that implement a replica group for the current partition, and wherein automatically creating the two or more new partition comprises:
 split the data stored in the current partition of the database to respectively add different portions of the data to the two or more new partitions of the database, wherein different respective plurality of nodes implement new replica groups for the two or more new partitions; and 
 receive and perform one or more requests from a client application directed to the data of the current partition of the non-relational database service while the splitting of the data is performed. 
 
   
     
     
         37 . The system of  claim 36 , wherein the non-relational database service is further configured to monitor the size of the data stored in the current partition to detect the increase in the size of the data stored in the current partition. 
     
     
         38 . The system of  claim 36 , wherein the current partition of the database is determined according to a key specified in a request to the non-relational database service. 
     
     
         39 . The system of  claim 38 , wherein the key uniquely identifies items in the database. 
     
     
         40 . The system of  claim 38 , wherein a hash function is applied to the key to determine the current partition of the database. 
     
     
         41 . The system of  claim 36 , wherein the database has a throughput for performing the one or more requests specified in a request to the non-relational database service. 
     
     
         42 . The system of  claim 36 , wherein a range index is created for the database responsive to a request to the non-relational database service. 
     
     
         43 . A method, comprising:
 automatically creating two or more new partitions of a database hosted by a non-relational database service responsive to detecting an increase in size of data stored in a current partition of the database, wherein the current partition of the database is stored on a plurality of storage nodes that implement a replica group for the current partition, and wherein automatically creating the two or more new partition comprises:
 splitting the data stored in the current partition of the database to respectively add different portions of the data to the two or more new partitions of the database, wherein different respective plurality of nodes implement new replica groups for the two or more new partitions; and 
 receiving and performing one or more requests from a client application directed to the data of the current partition of the non-relational database service while the splitting of the data is performed. 
   
     
     
         44 . The method of  claim 43 , further comprising monitoring the size of the data stored in the current partition to detect the increase in the size of the data stored in the current partition. 
     
     
         45 . The method of  claim 43 , wherein the current partition of the database is determined according to a key specified in a request to the non-relational database service. 
     
     
         46 . The method of  claim 45 , wherein the key uniquely identifies items in the database. 
     
     
         47 . The method of  claim 45 , wherein a hash function is applied to the key to determine the current partition of the database. 
     
     
         48 . The method of  claim 43 , wherein the database has a throughput for performing the one or more requests specified in a request to the non-relational database service. 
     
     
         49 . The method of  claim 43 , wherein a range index is created for the database responsive to a request to the non-relational database service. 
     
     
         50 . One or more non-transitory computer-readable storage media storing program instructions that, when executed on or across one or more computing devices, cause the one or more computing devices to implement:
 automatically creating two or more new partitions of a database hosted by a non-relational database service responsive to detecting an increase in size of data stored in a current partition of the database, wherein the current partition of the database is stored on a plurality of storage nodes that implement a replica group for the current partition, and wherein automatically creating the two or more new partition comprises:
 splitting the data stored in the current partition of the database to respectively add different portions of the data to the two or more new partitions of the database, wherein different respective plurality of nodes implement new replica groups for the two or more new partitions; and 
 receiving and performing one or more requests from a client application directed to the data of the current partition of the non-relational database service while the splitting of the data is performed. 
   
     
     
         51 . The one or more non-transitory computer-readable storage media of  claim 50 , storing further program instructions that when executed on or across the one or more computing devices, cause the one or more computing devices to further implement monitoring the size of the data stored in the current partition to detect the increase in the size of the data stored in the current partition. 
     
     
         52 . The one or more non-transitory computer-readable storage media of  claim 50 , wherein the current partition of the database is determined according to a key specified in a request to the non-relational database service. 
     
     
         53 . The one or more non-transitory computer-readable storage media of  claim 52 , wherein the key uniquely identifies items in the database. 
     
     
         54 . The one or more non-transitory computer-readable storage media of  claim 52 , wherein a hash function is applied to the key to determine the current partition of the database. 
     
     
         55 . The one or more non-transitory computer-readable storage media of  claim 50 , wherein the database has a throughput for performing the one or more requests specified in a request to the non-relational database service.