Call review tool for intelligent voice interface

Abstract

A method for facilitating reviews of caller interactions with an intelligent voice interface may include receiving raw voice data representing dialog between one or more callers and the intelligent voice interface during one or more respective voice calls, determining, by processing text translation of the raw voice data using one or more natural language processing models, one or more intents of the one or more callers during the one or more voice calls, generating one or more event labels indicative of one or more events associated with the one or more voice calls, and causing a user interface to be presented on a display device. The user interface may enable a user to (i) listen to the raw voice data, (ii) view the one or more intents, and/or (iii) view the one or more event labels.

The invention claimed is: 
     
       1. A computer-implemented method for facilitating reviews of caller interactions with an intelligent voice interface, the computer-implemented method comprising:
 receiving, by one or more processors, raw voice data representing dialog between one or more callers and the intelligent voice interface during one or more respective voice calls; 
 determining, by the one or more processors processing text translation of the raw voice data using one or more natural language processing models, one or more intents of the one or more callers during the one or more voice calls; 
 generating, by the one or more processors, one or more event labels indicative of one or more events associated with the one or more voice calls, wherein the one or more event labels include an event label indicating that a caller was not prepared, an event label indicating the presence of background noise, and an event label indicating that the intelligent voice interface did not understand a caller accent; and 
 causing, by the one or more processors, a user interface to be presented on a display device, the user interface enabling a reviewer to (i) listen to the raw voice data, (ii) view a transcript of the one or more voice calls between the caller and intelligent voice interface, (iii) view the one or more intents, and (iv) view the one or more event labels. 
 
     
     
       2. The computer-implemented method of  claim 1 , wherein the one or more voice calls include a plurality of voice calls, and wherein the computer-implemented method further comprises:
 generating, by the one or more processors, a rating for each of the plurality of voice calls, wherein each rating is indicative of performance of the intelligent voice interface and/or a result of the respective voice call, 
 wherein the user interface enables the reviewer to view a list of the plurality of voice calls and the respective ratings. 
 
     
     
       3. The computer-implemented method of  claim 1 , further comprising:
 translating, by the one or more processors, the raw voice data to the text translation. 
 
     
     
       4. The computer-implemented method of  claim 1 , wherein the one or more event labels further include an event label indicative of a determined intent of a caller. 
     
     
       5. The computer-implemented method of  claim 1 , wherein the one or more event labels further include an event label indicative of an error event. 
     
     
       6. The computer-implemented method of  claim 1 , wherein the one or more event labels further include an event label indicative of a state of a voice call. 
     
     
       7. The computer-implemented method of  claim 1 , wherein the one or more event labels further include an event label indicative of a determination made by the one or more processors based upon information provided by a caller during a voice call. 
     
     
       8. The computer-implemented method of  claim 1 , wherein generating the one or more event labels includes:
 generating at least one of the one or more event labels using a machine learning model, the machine learning model being trained using manually-entered event labels. 
 
     
     
       9. The computer-implemented method of  claim 8 , wherein the user interface enables the reviewer to manually enter event labels, and wherein the computer-implemented method further comprises:
 associating, by the one or more processors, one or more manual event labels entered by the reviewer with a particular voice call, or with a specific portion of the particular voice call. 
 
     
     
       10. A computing system comprising:
 one or more processors; and 
 one or more memories storing instructions that, when executed by the one or more processors, cause the one or more processors to:
 receive raw voice data representing dialog between one or more callers and an intelligent voice interface during one or more respective voice calls; 
 determine, by processing text translation of the raw voice data using one or more natural language processing models, one or more intents of the one or more callers during the one or more voice calls; 
 generate one or more event labels indicative of one or more events associated with the one or more voice calls, wherein the one or more event labels include an event label indicating that a caller was not prepared, an event label indicating the presence of background noise, and an event label indicating that the intelligent voice interface did not understand a caller accent; and 
 cause a user interface to be presented on a display device, the user interface enabling a reviewer to (i) listen to the raw voice data, (ii) view a transcript of the one or more voice calls between the caller and intelligent voice interface, (iii) view the one or more intents, and (iv) view the one or more event labels. 
 
 
     
     
       11. The computing system of  claim 10 , wherein:
 the one or more voice calls include a plurality of voice calls; 
 the instructions further cause the one or more processors to generate a rating for each of the plurality of voice calls; 
 each rating is indicative of performance of the intelligent voice interface and/or a result of the respective voice call; and 
 the user interface enables the reviewer to view a list of the plurality of voice calls and the respective ratings. 
 
     
     
       12. The computing system of  claim 10 , wherein:
 the instructions further cause the one or more processors to translate the raw voice data to the text translation. 
 
     
     
       13. The computing system of  claim 10 , wherein the one or more event labels further include an event label indicative of a determined intent of a caller. 
     
     
       14. The computing system of  claim 10 , wherein one or more of:
 the one or more event labels further include an event label indicative of an error event; 
 the one or more event labels further include an event label indicative of a state of a voice call; or 
 the one or more event labels further include an event label indicative of a determination made by the one or more processors based upon information provided by a caller during a voice call. 
 
     
     
       15. The computing system of  claim 10 , wherein the instructions further cause the one or more processors to:
 generate at least one event label of the one or more event labels using a machine learning model, the machine learning model being trained using manually-entered event labels. 
 
     
     
       16. The computing system of  claim 15 , wherein:
 the user interface enables the user to manually enter event labels; and 
 the instructions further cause the one or more processors to associate one or more manual event labels entered by the user with a particular voice call, or with a specific portion of the particular voice call.