Server side sampling of databases

Abstract

A system and method for use with a data mining application for a large database having a large number of records. A selection attribute is chosen from one of a plurality of attributes contained by records within the database. Records are scanned in the database and a randomizing function is applied to the selection attribute of each record to create a randomized record value. A selection criteria is then applied to identify records for inclusion within a subset of records (smaller than the original data set) by comparing the randomized record value of each record with the selection criteria. The subset of records having a randomized record value satisfying the selection criteria approximates the entire database but takes up less memory and can be evaluated or scanned much more quickly.

1 . A method of identifying a subset of records within a database for purposes of representing said database comprising: 
 a) choosing a selection attribute from one of a plurality of attributes contained in records within the database;    b) scanning records in the database and applying a randomizing function to the selection attribute of each record to create a randomized record value; and    c) applying a selection criteria to identify records for inclusion within a subset of records by comparing the randomized record value of each record with the selection criteria.    
     
     
         2 . The method of  claim 1  wherein the selection attribute contains a numeric value and additionally comprising scaling the numeric value with a factor before applying the randomizing function.  
     
     
         3 . The method of  claim 1  wherein the selection attribute contains a text string that is used as an input parameter to the randomizing function.  
     
     
         4 . The method of  claim 1  wherein the scanning of records and applying the selection criteria is implemented by an SQL statement that includes a randomizing function.  
     
     
         5 . The method of  claim 1  wherein the subset of records that represent the database are transmitted from a server computer to a client computer by means of a network.  
     
     
         6 . The method of  claim 5  additionally the client performs a data mining analysis on the subset of data that is transmitted to said client by means of the network.  
     
     
         7 . Client/Server computer apparatus comprising: 
 one or more client computers coupled to a network and including communications instructions for requesting a data set by means of the network; and    a server computer coupled to the network and having access to a database having a number of records, said server computer including instructions for sending a dataset made up of a subset of the records in the database to a client computer via the network;    said server computer including instructions for scanning records in the database, applying a randomizing function to a specified record attribute of each record in the database to produce a randomized record value, and comparing the randomized record value with a selection criteria to determine whether to include a record in the subset of records from the database for transmission via the network to the client.    
     
     
         8 . The Client/Server computer apparatus of  claim 7  wherein the client determines the selection criteria and transmits said selection criteria to the server for use in building the subset of records from the database.  
     
     
         9 . The Client/Server apparatus of  claim 7  additionally comprising an SQL language interpreter for converting requests from a client to an SQL statement including a randomizing function.  
     
     
         10 . A computer readable medium including instructions for identifying a subset of records within a database for purposes of representing said database, said computer readable medium comprising instructions for: 
 a) choosing a selection attribute from one of a plurality of attributes contained by records with the database;    b) scanning records in the database and applying a randomizing function to the selection attribute of each record to create a randomized record value; and    c) applying a selection criteria to identify records for inclusion within a subset of records by comparing the randomized record value of each record with the selection criteria.    
     
     
         11 . The computer readable medium of  claim 10  wherein the selection attribute contains a numeric value and additionally comprising instructions for scaling the numeric value with a factor before applying the randomizing function.  
     
     
         12 . The computer readable medium of  claim 10  wherein the selection attribute contains a text string that is used by said instructions as an input parameter to the randomizing function.  
     
     
         13 . The computer readable medium of  claim 10  wherein the scanning of records and applying the selection criteria are performed by instructions that implement an SQL statement that includes a randomizing function.  
     
     
         14 . The computer readable medium of  claim 10  wherein the instructions transmit a subset of records that represent the database from a server computer to a client computer by means of a network.  
     
     
         15 . The computer readable medium of  claim 10  additionally comprising instructions for performing a data mining analysis on the subset of data that is transmitted to the client by means of the network.