Techniques for selective copying of file data of a data source

Abstract

Techniques and solutions are described for copying data from a source client to a target client. It may be selected, such as by a user, whether a full copy or a partial copy of a set of source client data should be made. If a partial copy is selected, a set of file types is provided. The file types can be used to define files that should be included in, or excluded from, the copy. A data source of the source client is analyzed to determine data associated with a file type that is to be included in a copy, or is associated with a file type that is not indicated as to be excluded from the copy. The determined data is copied from the source client to the target client.

What is claimed is: 
     
       1. A computing system comprising:
 memory; 
 one or more hardware processing units coupled to the memory; and 
 one or more computer readable storage media storing instructions that, when executed, cause the computing system to perform operations comprising:
 receiving a request to copy data from a source client to a target client to provide a copy of at least a portion of data stored for the source client, the request comprising an indication that, for a first data source, at least certain types of files should be excluded from data copied from the source client, wherein the data comprises structured or semi-structured data of a database system and one or more files linked to the structured or semi-structured data, wherein the structured or semi-structured data of the database system comprises a reference to one or more of a location of the one or more files or a file identifier of the one or more files, wherein the structured or semi-structured data further comprises data other than the reference to the one or more of the location or the file identifier; 
 determining a first set of file types that should be included in the copy; 
 analyzing the first data source of the source client; 
 determining first structured or semi-structured data in the first data source that is not associated with a file, the first structured or semi-structured data not including the reference to the one or more of the location of the one or more files or the file identifier of the one or more files; 
 copying contents of the first structured or semi-structured data to the target client; 
 determining second structured or semi-structured data in the first data source that is associated with a file type that is in the first set of file types, the second structured or semi-structured data including the reference to the one or more of the location of the one or more files or the file identifier of the one or more files; 
 copying contents of the second structured or semi-structured data to the target client; 
 copying files linked to the second structured or semi-structured data and having a file type that is in the first set of file types to the target client; 
 determining third structured or semi-structured data in the first data source that is associated with a file type that is not in the first set of file types, the third structured or semi-structured data including the reference to the one or more of the location of the one or more files or the file identifier of the one or more files; 
 copying contents of the third structured or semi-structured data to the target client; and 
 not copying contents of files having a file type that is not in the first set of file types to the target client. 
 
 
     
     
       2. The computing system of  claim 1 , wherein copying the first structured or semi-structured data to the target client comprises:
 determining identifiers for the first structured or semi-structured data; 
 partitioning identifiers for the first structured or semi-structured data into a plurality of packages; and 
 on a package-by-package basis, copying first structured or semi-structured data from the source client to the target client. 
 
     
     
       3. The computing system of  claim 2 , wherein the identifiers comprise primary key values. 
     
     
       4. The computing system of  claim 2 , wherein partitioning identifiers for the first structured or semi-structured data into a plurality of packages comprises:
 determining a size for a given package of the plurality of packages; 
 determining that the size satisfies a threshold; and 
 removing one or more identifiers from the package to provide a revised package size for the given package, the revised package size not satisfying the threshold. 
 
     
     
       5. The computing system of  claim 4 , wherein the determining a size and removing one or more identifiers are carried out if it is determined that data in the package comprises file data. 
     
     
       6. The computing system of  claim 1 , wherein the source client is a first tenant of a multitenant database system comprising a plurality of tenants. 
     
     
       7. The computing system of  claim 1 , wherein the set of file types is a default set of file types. 
     
     
       8. The computing system of  claim 1 , wherein the set of file types is based at least in part on user input received in generating the request to copy data. 
     
     
       9. The computing system of  claim 1 , wherein the request to copy data further comprises an indication that data copied should include change logs, wherein change logs are otherwise not copied from the source client. 
     
     
       10. The computing system of  claim 1 , wherein the set of file types comprises one or more file extensions, and wherein the contents of the third structured or semi-structured data copied to the target client comprise the data other than the reference to the one or more of the location or the file identifier. 
     
     
       11. The computing system of  claim 1 , wherein the set of file types comprises one or more metadata values for one or more metadata element types. 
     
     
       12. The computing system of  claim 11 , wherein determining second structured or semi-structured data comprises comparing the one or more metadata values with metadata values of the first data source. 
     
     
       13. The computing system of  claim 1 , wherein the set of file types is based at least in part on input provided by an application that generated the request to copy data. 
     
     
       14. The computing system of  claim 1 , wherein the source client comprises at least second and third data sources, the second and third data sources being assigned to at least a first group, the second data source is associated with an indication that the at least certain types of files should not be copied, the third data source is not associated with an indication that the at least certain types of files should not be copied and the at least certain types of files are copied or not copied for both the second and third data sources. 
     
     
       15. The computing system of  claim 1 , wherein the first data source is associated with processing logic specific to the first data source to determine the second structured or semi-structured data. 
     
     
       16. The computing system of  claim 15 , wherein the processing logic comprises a SELECT statement in a query language. 
     
     
       17. The computing system of  claim 1 , wherein the request to copy data is applied to at least a second data source, processing logic for determining data of the second data source associated with the first set of files types is not defined for the at least a second data source, and the operations further comprise:
 copying all data of the at least a second data source to the target client. 
 
     
     
       18. The computing system of  claim 1 , wherein the first set of files types are a proper subset of a second set of file types, the first and second sets of file types being part of a common general class of files. 
     
     
       19. One or more computer-readable storage media comprising:
 computer-executable instructions that, when executed by a computing system comprising at least one hardware processor and at least one memory coupled to the at least one hardware processor, cause the computing system to receive a request to copy data from a source client to a target client to provide a copy of at least a portion of data stored for the source client, wherein the data comprises structured or semi-structured data of a database system and one or more files linked to the structured or semi-structured data, the structured or semi-structured data of the database system comprising a reference to one or more of a location of the one or more files or a file identifier of the one or more files, the structured or semi-structured data further comprising data other than the reference to the one or more of the location or the file identifier, the request comprising an indication that (1) for a first data source, at least certain types of files should be excluded from data copied from the source client, and (2) change logs should be included in data copied from the source client to the target client, the indication that change logs should be included in data copied from the source client overriding a default action of not copying change logs; 
 computer-executable instructions that, when executed, cause the computing system to determine a first set of file types that should be included in the copy; 
 computer-executable instructions that, when executed, cause the computing system to analyze the first data source of the source client; 
 computer-executable instructions that, when executed, cause the computing system to determine first structured or semi-structured data in the first data source that is not associated with a file, the first structured or semi-structured data not including the reference to the one or more of the location of the one or more files or the file identifier of the one or more files; 
 computer-executable instructions that, when executed, cause the computing system to copy contents of the first structured or semi-structured data to the target client; 
 computer-executable instructions that, when executed, cause the computing system to determine second structured or semi-structured data in the first data source that is associated with a file type that is in the first set of file types, the second structured or semi-structured data including the reference to the one or more of the location of the one or more files or the file identifier of the one or more files; 
 computer-executable instructions that, when executed, cause the computing system to copy contents of the second structured or semi-structured data to the target client; 
 computer-executable instructions that, when executed, cause the computing system to copy files linked to the second structured or semi-structured data and having a file type that is in the first set of file types to the target client; 
 computer-executable instructions that, when executed, cause the computing system to determine third structured or semi-structured data in the first data source that is associated with a file type that is not in the first set of file types, the third structured or semi-structured data including the reference to the one or more of the location of the one or more files or the file identifier of the one or more files; 
 computer-executable instructions that, when executed, cause the computing system to not copy contents of files having a file type that is not in the first set of file types to the target client; 
 computer-executable instructions that, when executed, cause the computing system to copy contents of the third structured or semi-structured data to the target client; and 
 computer-executable instructions that, when executed, cause the computing system to, based on the indication that change logs should be included in the copy, copy change logs from the source client to the target client. 
 
     
     
       20. A method, implemented in a computing system comprising a memory and one or more hardware processors coupled to the memory, comprising:
 receiving a request to copy data from a source client to a target client to provide a copy of at least a portion of data stored for the source client, the request comprising an indication that, for a first data source, at least certain types of files should be excluded from data copied from the source client, wherein the data comprises structured or semi-structured data of a database system and one or more files linked to the structured or semi-structured data, wherein the structured or semi-structured data of the database system comprises a reference to one or more of a location of the one or more files or a file identifier of the one or more files, wherein the structured or semi-structured data further comprises data other than the reference to the one or more of the location or the file identifier; 
 determining a first set of file types that should be included in the copy; 
 analyzing the first data source of the source client; 
 using custom logic defined for the first data source, determining first structured or semi-structured data in the first data source that is not associated with a file, the first structured or semi-structured data not including the reference to the one or more of the location of the one or more files or the file identifier of the one or more files; 
 copying contents of the first structured or semi-structured data to the target client; 
 using the custom logic defined for the first data source, determining second structured or semi-structured data in the first data source that is associated with a file type that is in the first set of file types, the second structured or semi-structured data including the reference to the one or more of the location of the one or more files or the file identifier of the one or more files; 
 copying contents of the second structured or semi-structured data to the target client; 
 copying files linked to the second structured or semi-structured data and having a file type that is in the first set of file types to the target client; 
 using the custom logic defined for the first data source, determining third structured or semi-structured data in the first data source that is associated with a file type that is not in the first set of file types, the third structured or semi-structured data including the reference to the one or more of the location of the one or more files or the file identifier of the one or more files; 
 copying contents of the third structured or semi-structured data to the target client; and 
 not copying contents of files having a file type that is not in the first set of file types to the target client.