Automatic speaker identification in calls

Abstract

A speaker identification system (“system”) automatically assigns a speaker to voiced segments in a conversation, without requiring any previously recorded voice sample or any other action by the speaker. The system enables unsupervised learning of speakers' fingerprints and using such fingerprints for identifying a speaker in a recording of a conversation. The system identifies one or more speakers, e.g., representatives of an organization, who are in conversation with other speakers, e.g., customers of the organization. The system processes recordings of conversations between a representative and one or more customers to generate multiple voice segments having a human voice, identifies the voice segments that have the same or a similar feature, and determines the voice in the identified voice segments as the voice of the representative.

1 - 34 . (canceled) 
     
     
         35 . A computer-implemented method comprising:
 receiving recordings of multiple conversations involving an individual,
 wherein each of the recordings includes voice data corresponding to a different conversation, and 
 wherein each of the recordings is of a conversation that the individual had with a different participant; 
   splitting each of the recordings into voice segments,
 wherein each voice segment corresponds to a voice of the individual or the corresponding participant; 
   analyzing the voice segments of the recordings to identify a set of voice segments that have a feature that corresponds to the individual;   generating a fingerprint for the individual based on the set of voice segments,
 wherein the fingerprint is representative of a vocal characteristic of the individual that is derived from the set of voice segments; and 
   storing the fingerprint in a data storage system.   
     
     
         36 . The computer-implemented method of  claim 35 , wherein the fingerprint is one of multiple fingerprints associated with multiple representatives of an organization that are stored in the data storage system. 
     
     
         37 . The computer-implemented method of  claim 36 , further comprising:
 receiving a stream of voice data corresponding to a conversation between multiple parties;   splitting the stream of voice data into voice segments,
 wherein each voice segment corresponds to a voice of one of the multiple parties; 
   comparing a given set of voice segments in the stream of voice data to the multiple fingerprints to identify a matching fingerprint;   identifying a representative associated with the matching fingerprint as the representative to be associated with the given set of voice segments; and   generating a marked conversation by appending information to the given subset of voice segments associated with the representative.   
     
     
         38 . The computer-implemented method of  claim 37 , wherein information identifying the source is appended only to the subset of voice segments associated with the representative. 
     
     
         39 . The computer-implemented method of  claim 35 , further comprising:
 associating the fingerprint with an identifier associated with the individual.   
     
     
         40 . The computer-implemented method of  claim 35 , wherein said analyzing comprises clustering the voice segments to generate multiple clusters of one or more voice segments, and wherein each cluster corresponds to the individual or one of the participants. 
     
     
         41 . The computer-implemented method of  claim 40 , further comprising:
 determining that a given cluster includes the set of voice segments associated with the individual since the given cluster has a highest number of voice segments.   
     
     
         42 . The computer-implemented method of  claim 40 , wherein said clustering comprises batching the voice segments such that two voice segments having a feature in common are clustered to a single cluster. 
     
     
         43 . The computer-implemented method of  claim 35 , wherein the feature is representative of a voice of the individual, an accent of the individual, a speech rate of the individual, a linguistic attribute of the individual, or a background noise. 
     
     
         44 . A non-transitory computer-readable medium with instructions stored thereon that, when executed by a processor, cause the processor to perform operations comprising:
 receiving recordings of multiple conversations involving an individual,
 wherein each recording includes voice data corresponding to a different conversation; 
   splitting each of the recordings into voice segments for which i-vector data structures are created,
 wherein each i-vector data structure includes a value representative of one or more features of the corresponding voice segment; 
   sorting the voice segments into at least two clusters based on the i-vector data structures,
 wherein each cluster includes voice segments associated with substantially similar i-vector data structures; 
   identifying a given cluster that is associated with the individual; and   generating a fingerprint for the individual based on the voice segments included in the given cluster,
 wherein the fingerprint is representative of a vocal characteristic of the individual that is derived from the voice segments included in the given cluster. 
   
     
     
         45 . The non-transitory computer-readable medium of  claim 44 , the operations further comprising:
 storing the fingerprint in a data storage system that maintains multiple fingerprints associated with different individuals.   
     
     
         46 . The non-transitory computer-readable medium of  claim 45 , wherein the different individuals are representatives who interact with customers on behalf of an enterprise. 
     
     
         47 . The non-transitory computer-readable medium of  claim 46 , the operations further comprising:
 receiving a stream of voice data corresponding to a conversation between multiple parties;   splitting the stream of voice data into voice segments,
 wherein each voice segment corresponds to a voice of one of the multiple parties; 
   comparing a set of voice segments in the stream of voice data to the multiple fingerprints to identify a matching fingerprint; and   identify a representative associated with the matching fingerprint as the representative involved in the conversation.   
     
     
         48 . The non-transitory computer-readable medium of  claim 47 , the operations further comprising:
 generating a marked conversation by appending information to the set of voice segments associated with the representative.   
     
     
         49 . The non-transitory computer-readable medium of  claim 44 , wherein the feature is indicative of an age of the individual, a gender of the individual, a physical ailment of the individual, a physical condition of the individual, or an education level of the individual. 
     
     
         50 . The non-transitory computer-readable medium of  claim 44 , wherein each of the recordings is a mono-channel recording in which speech of all participants in the corresponding conversation is over a single communication channel. 
     
     
         51 . A computer-implemented method comprising:
 receiving a stream of voice data corresponding to a conversation between multiple parties;   splitting the stream of voice data into voice segments,
 wherein each voice segment corresponds to a voice of one of the multiple parties; 
   sorting the voice segments into multiple clusters associated with the multiple parties;   identifying a given cluster associated with a party of interest;   comparing the voice segments in the given cluster to a series of fingerprints stored in a data storage system to identify a matching fingerprint,
 wherein each fingerprint is representative of a vocal characteristic of a different representative who interacts with customers on behalf of an enterprise; 
   identifying a representative associated with the matching fingerprint as the party of interest; and   generating a marked conversation by appending information regarding the representative only to the voice segments in the given cluster.   
     
     
         52 . The computer-implemented method of  claim 51 , wherein the given cluster is identified from amongst the multiple multiples since the given cluster has a highest number of voice segments. 
     
     
         53 . The computer-implemented method of  claim 51 , wherein the conversation is a video call, and wherein the stream of voice data is part of a multimedia stream that also includes a stream of video data. 
     
     
         54 . The computer-implemented method of  claim 51 , wherein voice segments of the stream of voice data other than the voice segments in the given cluster do not have information regarding the corresponding party appended thereto.