Identifying multimedia objects based on multimedia fingerprint

Abstract

Embodiments of identifying multimedia objects based on multimedia fingerprints are provided. Query fingerprints are derived from a multimedia object according to differing fingerprint algorithms. For each fingerprint algorithm, decisions are calculated through at least one classifier corresponding to the fingerprint algorithm based on the query fingerprint and reference fingerprints, the reference fingerprints being derived from reference multimedia objects according to the same fingerprint algorithm. Each of the decisions indicates a possibility that the query fingerprint and the reference fingerprint are not derived from the same multimedia content. For each of the reference multimedia objects, a distance is calculated as a weighted sum of the decisions relating to the reference fingerprints. The multimedia object is identified as matching the reference multimedia object with the smallest distance less than a threshold.

We claim: 
     
         1 . An apparatus for identifying a multimedia object, comprising:
 an acquiring unit, implemented at least in part by one or more computing processors, that acquires query fingerprints f q,1  to f q,T  which are derived from the multimedia object according to fingerprint algorithms F 1  to F T  respectively, where the fingerprint algorithms F 1  to F T  are different from each other, and T>1;   a plurality of classifying units, wherein each fingerprint algorithm F t  corresponds to at least one of the classifying units, and each of the classifying units, implemented at least in part by one or more computing processors, calculates decisions through a classifier based on the query fingerprint f q,t  and reference fingerprints derived from a plurality of reference multimedia objects according to the fingerprint algorithm F t , each of the decisions indicating a possibility that the query fingerprint and the reference fingerprint for calculating the decision are not derived from the same multimedia content; and   a combining unit, implemented at least in part by one or more computing processors, that, for each of the reference multimedia objects, calculates a distance D as a weighted sum of the decisions relating to the reference fingerprints derived from the reference multimedia object according to the fingerprint algorithms F 1  to F T  respectively; and   an identifying unit, implemented at least in part by one or more computing processors, that identifies the multimedia object as matching the reference multimedia object with the smallest distance which is less than a threshold TH c ;   wherein for each of at least one of the classifiers, the fingerprints for the classifier are derived as hash values, and the corresponding classifying unit further calculates the decisions through the classifier based on the query fingerprint and the reference fingerprints by:
 calculating a distance d between the query fingerprint and each of the reference fingerprints; and 
 calculating the decisions by deciding that at least one of the reference fingerprints with the distance d less than a threshold and the query fingerprint are derived from the same multimedia content. 
   
     
     
         2 . The apparatus according to  claim 1 , wherein each fingerprint algorithms F t  corresponds to only one of the classifying units. 
     
     
         3 . The apparatus according to  claim 1 , wherein the multimedia object includes a number W of objects which are synchronous with each other, and each of the reference multimedia objects includes the number W of objects which are synchronous with each other, where W>1, and
 wherein for each of the W objects in the multimedia object and the reference multimedia objects, at least one of the fingerprints is derived from the object according to the same fingerprint algorithm respectively.   
     
     
         4 . The apparatus according to  claim 1 , wherein each of the hash values is divided into weak bits and reliable bits, where the weak bits are likely to flip when the multimedia object, from which the fingerprint is derived, is modified, and the reliable bits are less likely to flip as a result of content modification. 
     
     
         5 . An apparatus for training a model for identifying multimedia objects, comprising:
 a fingerprint calculator, implemented at least in part by one or more computing processors, that, for each of one or more samples including a training query multimedia object, a training reference multimedia object and a mark indicating whether the training query multimedia object matches the training reference multimedia object or not, derives training query fingerprints from the training query multimedia object according to fingerprint algorithms F 1  to F G  respectively, where the fingerprint algorithms F 1  to F G  are different from each other, and G>1, and derives training reference fingerprints from the training reference multimedia object according to the fingerprint algorithms F 1  to F G  respectively; and   a training unit, implemented at least in part by one or more computing processors, that:   for each fingerprint algorithm F t , generates at least one candidate classifier based on the training query fingerprints and the training reference fingerprints derived according to the fingerprint algorithm F t , the candidate classifier being adapted to calculate a decision for any two fingerprints derived according to the fingerprint algorithm F t , which indicates a possibility that the two fingerprints are not derived from the same multimedia content; and   generates the model including a weighted sum of classifiers selected from the candidate classifiers and a threshold TH c  for evaluating the weighted sum such that the identifying error obtained by applying the model to the training query fingerprints and the training reference fingerprints derived from the samples is minimized;   wherein for each of at least one of the candidate classifiers, the fingerprints for generating the candidate classifier are derived as hash values, and the candidate classifier is adapted to:
 calculate a distance d between the training query fingerprint and each of a set of training reference fingerprints, and 
 calculate the decisions by deciding that at least one of the training reference fingerprints with the distance d less than a threshold and the training query fingerprint are derived from the same multimedia content, and 
 wherein at least two candidate thresholds for calculating the decisions are provided and the candidate threshold resulting in the smallest identifying error is selected as the threshold for the candidate classifier. 
   
     
     
         6 . The apparatus according to  claim 5 , wherein the selected classifiers in the generated model correspond to more than one fingerprint algorithm. 
     
     
         7 . The apparatus according to  claim 5 , wherein the classifiers are generated and selected through an Adaboost method. 
     
     
         8 . The apparatus according to  claim 7 , wherein weights of the selected classifiers in the weighted sum are determined through the Adaboost method. 
     
     
         9 . The apparatus according to  claim 5 , wherein the generation of the model comprises:
 providing at least two sets of candidate weights of the selected classifiers in the weighted sum; and   selecting the set of candidate weights resulting in the smallest identifying error as the weights of the selected classifiers in the weighted sum.   
     
     
         10 . The apparatus according to  claim 5 , wherein for each fingerprint algorithm F t , only one classifier is selected. 
     
     
         11 . The apparatus according to  claim 5 , wherein the fingerprints for generating the candidate classifier are derived as hash values, each of the hash values is divided into weak bits and reliable bits, where the weak bits are likely to flip when the multimedia object, from which the fingerprint is derived, is modified, and the reliable bits are less likely to flip as a result of content modification. 
     
     
         12 . The apparatus according to  claim 5 , wherein each of the training query multimedia objects includes a number W of objects which are synchronous with each other, and each of the training reference multimedia objects includes the number W of objects which are synchronous with each other, where W>1, and
 wherein for each of the W objects in the training query multimedia objects and the training reference multimedia objects, at least one of the fingerprints is derived from the object according to the same fingerprint algorithm respectively.   
     
     
         13 . A method of identifying a multimedia object, comprising:
 acquiring query fingerprints f q,1  to f q,T  which are derived from the multimedia object according to fingerprint algorithms F 1  to F T  respectively, where the fingerprint algorithms F 1  to F T  are different from each other, and T>1;   for each fingerprint algorithm F t , calculating decisions through each of at least one classifier corresponding to the fingerprint algorithm F t  based on the query fingerprint f q,t  and reference fingerprints derived from a plurality of reference multimedia objects according to the fingerprint algorithm F t , each of the decisions indicating a possibility that the query fingerprint and the reference fingerprint for calculating the decision are not derived from the same multimedia content;   for each of the reference multimedia objects, calculating a distance D as a weighted sum of the decisions relating to the reference fingerprints derived from the reference multimedia object according to the fingerprint algorithms F 1  to F T  respectively; and   identifying the multimedia object as matching the reference multimedia object with the smallest distance which is less than a threshold TH c ;   wherein for each of at least one of the classifiers, the fingerprints for the classifier are derived as hash values, and wherein the decisions are calculated based on the query fingerprint and the reference fingerprints by:
 calculating a distance d between the query fingerprint and each of the reference fingerprints; and 
 calculating the decisions by deciding that at least one of the reference fingerprints with the distance d less than a threshold and the query fingerprint are derived from the same multimedia content. 
   
     
     
         14 . The method according to  claim 13 , wherein each fingerprint algorithms F t  corresponds to only one classifying unit. 
     
     
         15 . The method according to  claim 13 , wherein the multimedia object includes a number W of objects which are synchronous with each other, and each of the reference multimedia objects includes the number W of objects which are synchronous with each other, where W>1, and
 wherein for each of the W objects in the multimedia object and the reference multimedia objects, at least one of the fingerprints is derived from the object according to the same fingerprint algorithm respectively.   
     
     
         16 . The method according to  claim 13 , wherein each of the hash values is divided into weak bits and reliable bits, where the weak bits are likely to flip when the multimedia object, from which the fingerprint is derived, is modified, and the reliable bits are less likely to flip as a result of content modification. 
     
     
         17 . A method of training a model for identifying multimedia objects, comprising:
 for each of one or more samples including a training query multimedia object, a training reference multimedia object and a mark indicating whether the training query multimedia object matches the training reference multimedia object or not,
 deriving training query fingerprints from the training query multimedia object according to fingerprint algorithms F 1  to F G  respectively, where the fingerprint algorithms F t  to F G  are different from each other, and G>1; 
 deriving training reference fingerprints from the training reference multimedia object according to the fingerprint algorithms F 1  to F G  respectively; and 
   for each fingerprint algorithm F t , generating at least one candidate classifier based on the training query fingerprints and the training reference fingerprints derived according to the fingerprint algorithm F t , the candidate classifier being adapted to calculate a decision for any two fingerprints derived according to the fingerprint algorithm F t , which indicates a possibility that the two fingerprints are not derived from the same multimedia content;   generating the model including a weighted sum of classifiers selected from the candidate classifiers and a threshold TH c  for evaluating the weighted sum such that the identifying error obtained by applying the model to the training query fingerprints and the training reference fingerprints derived from the samples is minimized;   wherein for each of at least one of the candidate classifiers, the fingerprints for generating the candidate classifier are derived as hash values, and the candidate classifier is adapted to:
 calculate a distance d between the training query fingerprint and each of a set of training reference fingerprints, and 
 calculate the decisions by deciding that at least one of the training reference fingerprints with the distance d less than a threshold and the training query fingerprint are derived from the same multimedia content, and 
 wherein at least two candidate thresholds for calculating the decisions are provided and the candidate threshold resulting in the smallest identifying error is selected as the threshold for the candidate classifier. 
   
     
     
         18 . The method according to  claim 17 , wherein each fingerprint algorithms F t  corresponds to only one classifying unit. 
     
     
         19 . The method according to  claim 17 , wherein the multimedia object includes a number W of objects which are synchronous with each other, and each of the reference multimedia objects includes the number W of objects which are synchronous with each other, where W>1, and
 wherein for each of the W objects in the multimedia object and the reference multimedia objects, at least one of the fingerprints is derived from the object according to the same fingerprint algorithm respectively.   
     
     
         20 . The method according to  claim 17 , wherein the selected classifiers in the generated model correspond to more than one fingerprint algorithm. 
     
     
         21 . The method according to  claim 17 , wherein the classifiers are generated and selected through an Adaboost method. 
     
     
         22 . The method according to  claim 21 , wherein weights of the selected classifiers in the weighted sum are determined through the Adaboost method. 
     
     
         23 . The method according to  claim 17 , wherein the generation of the model comprises:
 providing at least two sets of candidate weights of the selected classifiers in the weighted sum; and   selecting the set of candidate weights resulting in the smallest identifying error as the weights of the selected classifiers in the weighted sum.   
     
     
         24 . The method according to  claim 17 , wherein for each fingerprint algorithm F t , only one classifier is selected.