Constructing ground truth when classifying data

Abstract

The present disclosure relates to evaluating whether two data records reflect the same entity using a classifier in the absence of ground truth. Without ground truth, it is difficult to determine the precision or recall of a classifier. The present disclosure generates a list comprising a series of unique feature signatures and a set of sample record pairs for each unique feature signature. In some embodiments, users may provide labels for the set of sample record pairs for each unique feature signature.

What is claimed is: 
     
         1 . A method comprising:
 generating feature signatures for pairs of records in a set of records, a given feature signature representing common features between a pair of records;   classifying the feature signatures; and   generating a list of unique feature signatures and a set of sample record pairs for each unique feature signature.   
     
     
         2 . The method of  claim 1 , further comprising selecting pairs of records that are likely to be related using a blocking operation. 
     
     
         3 . The method of  claim 1 , wherein the list further comprises a respective frequency that each unique feature signature occurs among the feature signatures. 
     
     
         4 . The method of  claim 3 , further comprising weighting each feature signature according to its respective frequency. 
     
     
         5 . The method of  claim 1 , further comprising obtaining user data for determining one of a precision value or recall value of the classifier. 
     
     
         6 . The method of  claim 5 , wherein the user data comprises labels for the set of sample record pairs for each unique feature signature. 
     
     
         7 . The method of  claim 1 , wherein further comprising determining one or more exact matches between the features of the pair of records being compared to generate the pairs of records. 
     
     
         8 . A non-transitory computer-readable storage medium for tangibly storing computer program instructions capable of being executed by a computer processor, the computer program instructions defining steps of:
 generating feature signatures for pairs of records in a set of records, a given feature signature representing common features between a pair of records;   classifying the feature signatures; and   generating a list of unique feature signatures and a set of sample record pairs for each unique feature signature.   
     
     
         9 . The non-transitory computer-readable storage medium of  claim 8 , the steps further comprising selecting pairs of records that are likely to be related using a blocking operation. 
     
     
         10 . The non-transitory computer-readable storage medium of  claim 8 , wherein the list further comprises a respective frequency that each unique feature signature occurs among the feature signatures. 
     
     
         11 . The non-transitory computer-readable storage medium of  claim 10 , the steps further comprising weighting each feature signature according to its respective frequency. 
     
     
         12 . The non-transitory computer-readable storage medium of  claim 8 , the steps further comprising obtaining user data for determining one of a precision value or recall value of the classifier. 
     
     
         13 . The non-transitory computer-readable storage medium of  claim 12 , wherein the user data comprises labels for the set of sample record pairs for each unique feature signature. 
     
     
         14 . The non-transitory computer-readable storage medium of  claim 8 , the steps further comprising determining one or more exact matches between the features of the pair of records being compared to generate the pairs of records. 
     
     
         15 . A device comprising:
 a processor configured to:
 generate feature signatures for pairs of records in a set of records, a given feature signature representing common features between a pair of records; 
 classify the feature signatures; and 
 generate a list of unique feature signatures and a set of sample record pairs for each unique feature signature. 
   
     
     
         16 . The device of  claim 15 , the processor further configured select pairs of records that are likely to be related using a blocking operation. 
     
     
         17 . The device of  claim 15 , wherein the list further comprises a respective frequency that each unique feature signature occurs among the feature signatures. 
     
     
         18 . The device of  claim 17 , the processor further configured to weight each feature signature according to its respective frequency. 
     
     
         19 . The device of  claim 15 , the processor further configured to obtain user data for determining one of a precision value or recall value of the classifier. 
     
     
         20 . The device of  claim 15 , the processor further configured to determine one or more exact matches between the features of the pair of records being compared to generate the pairs of records.