Fuzzy hash algorithms to calculate file similarity

Abstract

Methods, apparatus, systems and articles of manufacture to classify a first file are disclosed herein. Example apparatus include a feature hash generator to generate respective sets of one or more feature hashes for respective features of the first file. The number of the one or more feature hashes to be generated is based on an ability of the feature to distinguish the first file from a second file. The apparatus also includes a bit setter to set respective bits of a first fuzzy hash value based on respective ones of the one or more feature hashes, a classifier to assign the first file to a class associated with a second file based on a similarity between the first fuzzy hash value and a second fuzzy hash value for a second file.

What is claimed is: 
     
       1. An apparatus to determine a file cluster for a file, the apparatus comprising:
 at least one memory; 
 instructions; and 
 processor circuitry to execute the instructions to at least:
 identify a number of different hash algorithms to be used to generate feature hashes for a feature of a file, the number of different hash algorithms based on a clustering significance of the feature; 
 set bits of a fuzzy hash value based on the feature hashes; and 
 determine the file cluster for the file based on the fuzzy hash value. 
 
 
     
     
       2. The apparatus of  claim 1 , wherein the feature is a first feature, the clustering significance is a first clustering significance, the feature hashes are first feature hashes, and the processor circuitry is to set bits of the fuzzy hash value based on the first feature hashes and on second feature hashes generated for a second feature of the file, the bits of the fuzzy hash value to be set in an order based on the first clustering significance of the first feature relative to a second clustering significance of the second feature. 
     
     
       3. The apparatus of  claim 1 , wherein the bits of the fuzzy hash value reside in a storage having a plurality of bit positions, the feature hashes correspond to numerical values, and the processor circuitry is to set bits of the fuzzy hash value based on the feature hashes by setting bit positions that correspond to the numerical values of the feature hashes. 
     
     
       4. The apparatus of  claim 1 , wherein the processor circuitry is to discard the fuzzy hash value when the fuzzy hash value includes a number of set bits that does not satisfy a threshold value. 
     
     
       5. The apparatus of  claim 1 , wherein the processor circuitry is to select the different hash algorithms based on a type of the feature. 
     
     
       6. The apparatus of  claim 5 , wherein the type of the feature is at least one of a dynamic feature, a static feature or a binary feature. 
     
     
       7. The apparatus of  claim 1 , wherein the processor circuitry is to:
 compare the fuzzy hash value to one or more other fuzzy hash values generated for other files; and 
 use a result of the comparison to determine the file cluster for the file based on the fuzzy hash value. 
 
     
     
       8. At least one non-transitory computer readable medium comprising computer readable instructions that, when executed, cause one or more processors to at least:
 identify a number of different hash algorithms to be used to generate feature hashes for a feature of a file, the number of different hash algorithms based on a clustering significance of the feature; 
 set bits of a fuzzy hash value based on the feature hashes; and 
 determine a file cluster for the file based on the fuzzy hash value. 
 
     
     
       9. The at least one computer readable medium of  claim 8 , wherein the feature is a first feature, the feature hashes are first feature hashes, the clustering significance is a first clustering significance, and the instructions are to cause the one or more processors to set bits of the fuzzy hash value based on second feature hashes generated for a second feature of the file, the bits of the fuzzy hash value to be set in an order based on the first clustering significance of the first feature relative to a second clustering significance of the second feature. 
     
     
       10. The at least one computer readable medium of  claim 8 , wherein the bits of the fuzzy hash value reside in a register having a plurality of bit positions, the feature hashes correspond to numerical values, and the instructions are to cause the one or more processors to set bits of the fuzzy hash value based on the feature hashes by setting bits positions that correspond to the numerical values of the feature hashes. 
     
     
       11. The at least one computer readable medium of  claim 8 , wherein the instructions are to cause the one or more processors to discard the fuzzy hash value when the fuzzy hash value includes a number of set bits that does not satisfy a threshold value. 
     
     
       12. The at least one computer readable medium of  claim 8 , wherein the instructions are to cause the one or more processors to select the different hash algorithms based on a type of the feature. 
     
     
       13. The at least one computer readable medium of  claim 12 , wherein the type of the feature is at least one of a dynamic feature, a static feature or a binary feature. 
     
     
       14. A method comprising:
 identifying a number of different hash algorithms to be used to generate feature hashes for a feature of a file, the number of different hash algorithms based on a clustering significance of the feature; 
 setting bits of a fuzzy hash value based on the feature hashes; and 
 determining a file cluster for the file based on the fuzzy hash value. 
 
     
     
       15. The method of  claim 14 , wherein the feature is a first feature, the clustering significance is a first clustering significance, the feature hashes are first feature hashes, and the method includes setting bits of the fuzzy hash value based on the first feature hashes and on second feature hashes generated for a second feature of the file, the bits of the fuzzy hash value to be set in an order based on the first clustering significance of the first feature relative to a second clustering significance of the second feature. 
     
     
       16. The method of  claim 15 , wherein the bits of the fuzzy hash value reside in a register having a plurality of bit positions, the feature hashes correspond to numerical values, and the method includes setting the bits of the fuzzy hash value based on the feature hashes by setting bit positions that correspond to the numerical values of the feature hashes. 
     
     
       17. The method of  claim 15 , including discarding the fuzzy hash value having a number of set bits that does not satisfy a threshold value. 
     
     
       18. The method of  claim 15 , including selecting the different hash algorithms based on a type of the feature. 
     
     
       19. The method of  claim 18 , wherein the type of the feature is one of a dynamic feature, a static feature, or a binary feature. 
     
     
       20. The method of  claim 15 , including comparing the fuzzy hash value to one or more other fuzzy hash values generated for other files; and using a result of the comparison to determine the file cluster for the file.