Intent-based query and response routing between users and backend services

Abstract

For a seamless and robust artificial intelligence-based assistant experience, an intent-based query and response router has been designed to operate as an intelligent layer between a user and multiple backend services that may respond to one or more queries over the course of a conversation with the user. The query router interacts with an intent classification service to obtain an intent classification for a prompt that is based on a user query. The query router uses the intent classification, which is used as an identifier of a backend service, to route the user query to an appropriate one (or more) of the backend services. When a response is detected, the query router determines a corresponding conversation and provides the response for the conversation.

The invention claimed is: 
     
       1. A method comprising:
 submitting a first prompt to a set of one or more large language model (LLM) based intent classifiers to obtain at least a first intent classification, wherein the first prompt is based on a first user query received via a first interface; 
 routing the first user query to at least one of a plurality of backend services based, at least partly, on the first intent classification; 
 detecting a first query response from at least one of the plurality of backend services; 
 determining that the first query response corresponds to the first user query; and 
 responding to the first user query based, at least in part, on the first query response. 
 
     
     
       2. The method of  claim 1 , wherein the set of one or more LLM based intent classifiers is trained with training data comprising tuples of example prompts and backend service identifiers. 
     
     
       3. The method of  claim 1 , wherein routing the first user query to at least one of the plurality of backend services comprises:
 creating a message with the first user query and with the first intent classification; and 
 publishing the message to a first message queue accessed by the plurality of backend services. 
 
     
     
       4. The method of  claim 3  further comprising:
 obtaining a second intent classification from the set of LLM based intent classifiers based on the first prompt or a second prompt generated based on the first user query; and 
 creating the message with the second intent classification in addition to the first intent classification or creating a second message with the first prompt and the second intent classification and publishing the second message to the first message queue. 
 
     
     
       5. The method of  claim 1  further comprising:
 creating the first prompt based on the first user query and at least one of conversation history based context and customer based context data. 
 
     
     
       6. The method of  claim 5 , wherein creating the first prompt is responsive to determining that a response from the set of one or more LLM based intent classifiers to a second prompt that was submitted prior to the first prompt and that is also based on the first user query does not indicate a defined backend service identifier. 
     
     
       7. The method of  claim 1  further comprising pre-processing the first user query to be a valid prompt. 
     
     
       8. The method of  claim 1  further comprising:
 detecting multiple query responses from the plurality of backend services, wherein the multiple query responses include the first query response; 
 determining that a query state for the first user query is complete; and 
 aggregating the multiple query responses into an aggregate response, 
 wherein responding to the first user query comprises responding with the aggregate response. 
 
     
     
       9. The method of  claim 8 , wherein aggregating the multiple query responses into the aggregate response comprises one of collating the multiple query responses, summarizing the multiple query responses, and merging the multiple query responses. 
     
     
       10. The method of  claim 8  further comprising ranking the multiple query responses and filtering the multiple query responses based, at least, in part on the ranking. 
     
     
       11. The method of  claim 1  further comprising editing the first query response based, at least in part, on an attribute of a digital identity associated with the first user query. 
     
     
       12. A non-transitory, machine-readable medium having program code stored thereon, the program code comprising instructions to:
 submit a first prompt to a set of one or more large language model (LLM) based intent classifiers to obtain at least a first intent classification, wherein the first prompt is based on a first user query received via a first interface; 
 communicate the first user query to at least one of a plurality of backend services based, at least partly, on the first intent classification; 
 detect a first query response from at least one of the plurality of backend services; 
 determine that the first query response corresponds to the first user query; and 
 respond to the first user query based, at least in part, on the first query response. 
 
     
     
       13. The machine-readable medium of  claim 12 , wherein the first interface is an application programming interface. 
     
     
       14. The machine-readable medium of  claim 12 , wherein the instructions to communicate the first user query comprise instructions to:
 create a message with the first user query and with the first intent classification; and 
 publish the message to a first message queue accessed by the plurality of backend services. 
 
     
     
       15. The machine-readable medium of  claim 14 , wherein the program code further has stored thereon instructions to:
 obtain a second intent classification from the set of LLM based intent classifiers based on the first prompt or a second prompt generated based on the first user query; and 
 create the message with the second intent classification in addition to the first intent classification or create a second message with the first prompt and the second intent classification and publish the second message to the first message queue. 
 
     
     
       16. The machine-readable medium of  claim 12 , wherein the program code to communicate the first user query comprises instructions to:
 for each intent classification from the set of LLM based intent classifiers for the first user query, supply the first user query to the one of the plurality of backend services identified by the intent classification. 
 
     
     
       17. The machine-readable medium of  claim 12 , wherein the program code further comprises instructions to:
 detect multiple query responses from the plurality of backend services including the first query response; 
 determine that a query state for the first user query is complete; and 
 aggregate the multiple query responses into an aggregate response, 
 wherein the instructions to respond to the first user query comprise instructions to respond with the aggregate response. 
 
     
     
       18. An apparatus comprising:
 a processor; and 
 a machine-readable medium having instructions stored thereon that are executable by the processor to cause the apparatus to, 
 submit a first prompt to a set of one or more large language model (LLM) based intent classifiers to obtain at least a first intent classification, wherein the first prompt is based on a first user query received via a first interface; 
 communicate the first user query to at least one of a plurality of backend services based, at least partly, on the first intent classification; 
 detect a first query response from at least one of the plurality of backend services; 
 determine that the first query response corresponds to the first user query; and 
 respond to the first user query based, at least in part, on the first query response. 
 
     
     
       19. The apparatus of  claim 18 , wherein the instructions to communicate the first user query comprise instructions executable by the processor to cause the apparatus to:
 for each intent classification from the set of LLM based intent classifiers for the first user query, supply the first user query to the one of the plurality of backend services identified by the intent classification. 
 
     
     
       20. The apparatus of  claim 18 , wherein the machine-readable medium further has stored thereon instructions executable by the processor to cause the apparatus to:
 detect multiple query responses from the plurality of backend services including the first query response; 
 determine that a query state for the first user query is complete; and 
 aggregate the multiple query responses into an aggregate response, 
 wherein the instructions to respond to the first user query comprise instructions to respond with the aggregate response.