Assessing the structural quality of conversations

Abstract

Input of a conversation is received. The conversation includes at least a first user. An utterance of the conversation is analyzed to identify a dialog act attribute, an emotion attribute, and a tone attribute. The dialog act attribute, emotion attribute, and tone attribute are annotated to the utterance of the conversation. The conversation is validated based on the annotated attributes compared with a threshold. The annotated conversation and the validation of the conversation are stored.

What is claimed is: 
     
         1 . A method, the method comprising:
 receiving, by one or more computer processors, an input of a conversation including at least a first user;   analyzing, by one or more computer processors, an utterance of the conversation to identify a dialog act attribute, an emotion attribute, and a tone attribute;   annotating, by one or more computer processors, the utterance of the conversation with the dialog act attribute, the emotion attribute, and the tone attribute;   validating, by one or more computer processors, the conversation based on the annotated attributes in comparison with a threshold; and   storing, by one or more computer processors, the annotated conversation and the validation of the conversation.   
     
     
         2 . The method of  claim 1 , wherein the step of validating, by one or more computer processors, the conversation, comprises:
 comparing, by one or more computer processors, each annotated utterance in the conversation to a known database of best practice annotated conversations;   scoring, by one or more computer processors, each annotated utterance based on a number of violations, based on the comparison of each annotated utterance to the known database of best practice annotated conversations;   ranking, by one or more computer processors, each utterance based on the scoring; and   suggesting, by one or more computer processors, a response for a second user to make to the first user based on each ranked utterance.   
     
     
         3 . The method of  claim 1 , wherein the conversation is between the first user and an automated response system, and wherein the automated response system comprises an application that performs an automated task. 
     
     
         4 . The method of  claim 1 , further comprising:
 retrieving, by one or more computer processors, a plurality of stored annotated conversations;   analyzing, by one or more computer processors, the plurality of stored annotated conversations, wherein the analysis determines a number of violations found in each annotated conversation;   ranking, by one or more computer processors, each annotated conversation in the plurality of stored annotated conversations based on the analysis; and   sending, by one or more computer processors, at least one recommendation to a second user for improving at least one annotated conversation in the plurality of stored annotated conversations.   
     
     
         5 . The method of  claim 4 , wherein the recommendation includes a change to a pre-defined dialog. 
     
     
         6 . The method of  claim 2 , further comprising:
 determining, by one or more computer processors, the number of violations using an algorithm selected from the group consisting of: a sequence alignment algorithm, an information retrieval method, a sequence-to-one model, and a weighted edit distance.   
     
     
         7 . A computer program product, the computer program product comprising:
 one or more computer readable storage media; and   program instructions stored on the one or more computer readable storage media, the program instructions comprising:
 program instructions to receive an input of a conversation including at least a first user; 
 program instructions to analyze an utterance of the conversation to identify a dialog act attribute, an emotion attribute, and a tone attribute; 
 program instructions to annotate the utterance of the conversation with the dialog act attribute, the emotion attribute, and the tone attribute; 
 program instructions to validate the conversation based on the annotated attributes in comparison with a threshold; and 
 program instructions to store the annotated conversation and the validation of the conversation. 
   
     
     
         8 . The computer program product of  claim 7 , wherein the program instructions to validate the conversation comprises:
 program instructions to compare each annotated utterance in the conversation to a known database of best practice annotated conversations;   program instructions to score each annotated utterance based on a number of violations, based on the comparison of each annotated utterance to the known database of best practice annotated conversations;   program instructions to rank each utterance based on the scoring; and   program instructions to suggest a response for a second user to make to the first user based on each ranked utterance.   
     
     
         9 . The computer program product of  claim 7 , wherein the conversation is between the first user and an automated response system, and wherein the automated response system comprises an application that performs an automated task. 
     
     
         10 . The computer program product of  claim 7 , further comprising program instructions stored on the one or more computer readable storage media, to:
 retrieve a plurality of stored annotated conversations;   analyze the plurality of stored annotated conversations, wherein the analysis determines a number of violations found in each annotated conversation;   rank each annotated conversation in the plurality of stored annotated conversations based on the analysis; and   send at least one recommendation to a second user for improving at least one annotated conversation in the plurality of stored annotated conversations.   
     
     
         11 . The computer program product of  claim 10 , wherein the recommendation includes a change to a pre-defined dialog. 
     
     
         12 . The computer program product of  claim 8 , further comprising program instructions stored on the one or more computer readable storage media, to:
 determine the number of violations using an algorithm selected from the group consisting of: a sequence alignment algorithm, an information retrieval method, a sequence-to-one model, and a weighted edit distance.   
     
     
         13 . A computer system, the computer system comprising:
 one or more computer processors;   one or more computer readable storage media; and   program instructions stored on the one or more computer readable storage media for execution by at least one of the one or more computer processors, the program instructions comprising:
 program instructions to receive an input of a conversation including at least a first user; 
 program instructions to analyze an utterance of the conversation to identify a dialog act attribute, an emotion attribute, and a tone attribute; 
 program instructions to annotate the utterance of the conversation with the dialog act attribute, the emotion attribute, and the tone attribute; 
 program instructions to validate the conversation based on the annotated attributes in comparison with a threshold; and 
 program instructions to store the annotated conversation and the validation of the conversation. 
   
     
     
         14 . The computer system of  claim 13 , wherein the program instructions to validate the conversation comprises:
 program instructions to compare each annotated utterance in the conversation to a known database of best practice annotated conversations;   program instructions to score each annotated utterance based on a number of violations, based on the comparison of each annotated utterance to the known database of best practice annotated conversations;   program instructions to rank each utterance based on the scoring; and   program instructions to suggest a response for a second user to make to the first user based on each ranked utterance.   
     
     
         15 . The computer system of  claim 13 , wherein the conversation is between the first user and an automated response system, and wherein the automated response system comprises an application that performs an automated task. 
     
     
         16 . The computer system of  claim 13 , further comprising program instructions stored on the one or more computer readable storage media for execution by at least one of the one or more computer processors, to:
 retrieve a plurality of stored annotated conversations;   analyze the plurality of stored annotated conversations, wherein the analysis determines a number of violations found in each annotated conversation;   rank each annotated conversation in the plurality of stored annotated conversations based on the analysis; and   send at least one recommendation to a second user for improving at least one annotated conversation in the plurality of stored annotated conversations.   
     
     
         17 . The computer system of  claim 16 , wherein the recommendation includes a change to a pre-defined dialog.