Real-time use of multiple parallel automatic speech recognition (ASR) modules in a conversational artificial intelligence (AI) architecture

Abstract

As an example, a conversational artificial intelligence (AI) that is in a conversation receives a human response from a human, augments the human response to create an augmented response, determines a context of the conversation, and provides the context and the augmented response to a plurality of automatic speech recognition (ASR) modules that individually process the augmented response in parallel. The conversational AI receives a plurality of intermediate text outputs from the plurality of ASR modules, wherein individual intermediate text outputs of the plurality of intermediate text outputs are received from individual ones of the ASR modules. A reconciliation AI performs a contextual reconciliation of the plurality of intermediate text outputs based at least in part on the context of the conversation to create a final text output. The conversational AI provides, in real-time, an artificial intelligence response to the human based on the final text output.

The invention claimed is: 
     
         1 . A method, comprising:
 initiating, by a conversational artificial intelligence executed by one or more processors, a conversation with a human;   receiving, by the conversational artificial intelligence, a human response from the human;   augmenting, by the one or more processors, the human response to create an augmented response;   determining, by the one or more processors, a context of the conversation;   providing, by the one or more processors, the context of the conversation and the augmented response to a plurality of automatic speech recognition (ASR) modules that individually process the augmented response in parallel;   receiving, by the one or more processors, a plurality of intermediate text outputs from the plurality of ASR modules, wherein individual intermediate text outputs of the plurality of intermediate text outputs are received from individual modules of the plurality of ASR modules;   performing in real-time, by a reconciliation artificial intelligence, a contextual reconciliation of the plurality of intermediate text outputs based at least in part on the context of the conversation and on a medical history of the human to create a final text output; and   generating, by the conversational artificial intelligence, an artificial intelligence response to the human based at least in part on the final text output.   
     
     
         2 . The method of  claim 1 , wherein augmenting the human response to create the augmented response comprises at least one of:
 performing an intonation analysis of the human response;   performing noise cancellation by reducing an amount of background noise present in the human response; or   any combination thereof.   
     
     
         3 . The method of  claim 1 , wherein determining the context of the conversation comprises:
 accessing electronic medical records (EMR) associated with the human; and   determining a conversation history of the conversation between the conversational artificial intelligence and the human.   
     
     
         4 . The method of  claim 1 , wherein the plurality of ASR modules comprise at least:
 a first ASR module implemented using a first artificial intelligence algorithm;   a second ASR module implemented using a second artificial intelligence algorithm; and   a third ASR module implemented using a third artificial intelligence algorithm;   wherein the first artificial intelligence algorithm, the second artificial intelligence algorithm, and the third artificial intelligence algorithm are different from each other.   
     
     
         5 . The method of  claim 1 , wherein the plurality of ASR modules comprise at least:
 a first ASR module trained using a first corpus of speech of people having a first type of accent when speaking a particular language;   a second ASR module trained using a second corpus of speech of people having a second type of accent when speaking the particular language; and   a third ASR module trained using a third corpus of speech of people having a third type of accent when speaking the particular language;   wherein the first type of accent, the second type of accent, and the third type of accent are different from each other.   
     
     
         6 . The method of  claim 1 , wherein the conversational artificial intelligence is trained using multi-turn reinforcement learning through human feedback (RLHF). 
     
     
         7 . The method of  claim 1 , wherein the conversational artificial intelligence, during the conversation with the human:
 identifies a turn-yielding cue;   performs interruption detection to detect when the human is attempting to interrupt the conversationa; artificial intelligence;   identifies a non-verbal cue associated with the human; or   any combination thereof.   
     
     
         8 . A server comprising:
 one or more processors; and   one or more computer-readable storage media to store instructions executable by the one or more processors to perform operations comprising:
 initiating, by a conversational artificial intelligence, a conversation with a human; 
 receiving, by the conversational artificial intelligence, a human response from the human; 
 augmenting the human response to create an augmented response; 
 determining a context of the conversation; 
 providing the context of the conversation and the augmented response to a plurality of automatic speech recognition (ASR) modules that individually process the augmented response in parallel; 
 receiving a plurality of intermediate text outputs from the plurality of ASR modules, wherein individual intermediate text outputs of the plurality of intermediate text outputs are received from individual modules of the plurality of ASR modules; 
 performing in real-time, by a reconciliation artificial intelligence, a contextual reconciliation of the plurality of intermediate text outputs based at least in part on the context of the conversation and on a medical history of the human to create a final text output; and 
 generating, by the conversational artificial intelligence, an artificial intelligence response to the human based at least in part on the final text output. 
   
     
     
         9 . The server of  claim 8 , wherein augmenting the human response to create the augmented response comprises at least one of:
 performing an intonation analysis of the human response including determining a volume, a pitch, and a rhythm of individual words in the human response;   performing noise cancellation by identifying speech content spoken by the human and reducing the volume of other content in the human response including other human speech; or   any combination thereof.   
     
     
         10 . The server of  claim 8 , wherein determining the context of the conversation comprises:
 determining electronic medical records (EMR) associated with the human; and   determining a conversation history of the conversation between the conversational artificial intelligence and the human.   
     
     
         11 . The server of  claim 10 , wherein the conversation history of the conversation between the conversational artificial intelligence and the human is stored in a cache memory. 
     
     
         12 . The server of  claim 8 , wherein the plurality of ASR modules comprise at least:
 a first ASR module implemented using a first artificial intelligence algorithm;   a second ASR module implemented using a second artificial intelligence algorithm; and   a third ASR module implemented using a third artificial intelligence algorithm;   wherein the first artificial intelligence algorithm, the second artificial intelligence algorithm, and the third artificial intelligence algorithm are different from each other.   
     
     
         13 . The server of  claim 8 , wherein the plurality of ASR modules comprise at least:
 a first ASR module trained using a first corpus of speech of people having a first type of accent when speaking a particular language;   a second ASR module trained using a second corpus of speech of people having a second type of accent when speaking the particular language; and   a third ASR module trained using a third corpus of speech of people having a third type of accent when speaking the particular language;   wherein the first type of accent, the second type of accent, and the third type of accent are different from each other.   
     
     
         14 . The server of  claim 8 , wherein the conversational artificial intelligence is configured to perform specialized healthcare-related functions comprising one or more of:
 gathering data related to performing Healthcare Effectiveness Data and Information Set (HEDIS) calculations;   performing a Health Records Assessment (HRA);   determining a Risk Adjustment Factor (RAF);   reviewing a pre-op checklist;   reviewing a discharge checklist;   reviewing a chronic care checklist;   determining social determinants of health (SDOH); or   any combination thereof.   
     
     
         15 . A non-transitory memory device to store instructions executable by one or more processors to perform operations comprising:
 initiating, by a conversational artificial intelligence, a conversation with a human;   receiving, by the conversational artificial intelligence, a human response from the human;   augmenting the human response to create an augmented response;   determining a context of the conversation;   providing the context of the conversation and the augmented response to a plurality of automatic speech recognition (ASR) modules that individually process the augmented response in parallel;   receiving a plurality of intermediate text outputs from the plurality of ASR modules, wherein individual intermediate text outputs of the plurality of intermediate text outputs are received from individual modules of the plurality of ASR modules;   performing in real-time, by a reconciliation artificial intelligence, a contextual reconciliation of the plurality of intermediate text outputs based at least in part on the context of the conversation and on a medical history of the patient to create a final text output; and   generating, by the conversational artificial intelligence, an artificial intelligence response to the human based at least in part on the final text output.   
     
     
         16 . The non-transitory memory device of  claim 15 , wherein augmenting the human response to create the augmented response comprises at least one of:
 performing an intonation analysis of the human response including determining a volume, a pitch, and a rhythm of individual words in the human response;   performing noise cancellation by identifying speech content spoken by the human and reducing the volume of other content in the human response including other human speech; or   any combination thereof.   
     
     
         17 . The non-transitory memory device of  claim 15 , wherein determining the context of the conversation comprises:
 determining electronic medical records (EMR) associated with the human; and   determining a conversation history of the conversation between the conversational artificial intelligence and the human.   
     
     
         18 . The non-transitory memory device of  claim 15 , wherein the plurality of ASR modules comprise at least:
 a first ASR module implemented using a first artificial intelligence algorithm;   a second ASR module implemented using a second artificial intelligence algorithm; and   a third ASR module implemented using a third artificial intelligence algorithm;   wherein the first artificial intelligence algorithm, the second artificial intelligence algorithm, and the third artificial intelligence algorithm are different from each other.   
     
     
         19 . The non-transitory memory device of  claim 15 , wherein the plurality of ASR modules comprise at least:
 a first ASR module trained using a first corpus of speech of people having a first type of accent when speaking a particular language;   a second ASR module trained using a second corpus of speech of people having a second type of accent when speaking the particular language; and   a third ASR module trained using a third corpus of speech of people having a third type of accent when speaking the particular language;   wherein the first type of accent, the type of second accent, and the third type of accent are different from each other.   
     
     
         20 . The non-transitory memory device of  claim 15 , wherein the conversational artificial intelligence is engaged in a task that includes one or more of:
 performing a preventative screening;   an intake-related task;   a scheduling-related task;   a pre-op related task;   a discharge-related task;   a chronic care related task; or   any combination thereof.