Complex natural language processing

Abstract

Techniques for processing complex natural language inputs are described. A complex natural language input may be semantically tagged and parsed to identify individual clauses in the complex natural language input. An execution graph may be generated to represent the clauses and their dependencies. Nodes of the execution graph may be processed using NLU processing and/or a knowledge graph or other information storage and retrieval techniques, and results of such processing may be used to update clause variables with specific entities in the execution graph.

1 - 20 . (canceled) 
     
     
         21 . A computer-implemented method, comprising:
 receiving first data representing a natural language input;   determining the natural language input corresponds to a first query and a second query, the first query represented by a first portion of the natural language input and the second query represented by a second portion of the natural language input;   processing the first portion of the natural language input to determine first response data;   performing natural language understanding (NLU) processing using the first response data and the second portion of the natural language input to determine an action responsive to the natural language input; and   causing the action to be performed to determine output data.   
     
     
         22 . The computer-implemented method of  claim 21 , further comprising:
 using a machine learning component to determine the natural language input corresponds to a plurality of queries.   
     
     
         23 . The computer-implemented method of  claim 22 , wherein the first data comprises audio data representing a spoken natural language input, and wherein the computer-implemented method further comprises:
 performing automatic speech recognition (ASR) processing on the audio data to generate first ASR output data representing a first ASR hypothesis;   processing the first ASR output data using the machine learning component to determine the natural language input corresponds to the plurality of queries; and   using the first ASR hypothesis to determine the first portion of the natural language input corresponding to the first query.   
     
     
         24 . The computer-implemented method of  claim 22 , wherein determining the natural language input corresponds to the plurality of queries comprises determining that intent classification processing of the first data is likely to result in output failing to satisfy a threshold condition. 
     
     
         25 . The computer-implemented method of  claim 21 , further comprising:
 determining second data corresponding to a first semantic tag corresponding to the first portion of the natural language input; and   determining third data corresponding to a second semantic tag corresponding to the second portion of the natural language input,   wherein determining the natural language input corresponds to a first query and a second query is based at least in part on the second data and the third data.   
     
     
         26 . The computer-implemented method of  claim 21 , further comprising:
 determining the first query corresponds to information retrieval processing; and   performing information retrieval processing to determine the first response data, wherein the first response data represents an entity corresponding to the first query.   
     
     
         27 . The computer-implemented method of  claim 26 , further comprising:
 determining entity data corresponding to the entity,   wherein the output data is based at least in part on the entity data.   
     
     
         28 . The computer-implemented method of  claim 26 , further comprising:
 determining entity data corresponding to the entity;   determining the entity data corresponds to a variable of the second query; and   populating the variable using the entity data for purposes of the NLU processing.   
     
     
         29 . The computer-implemented method of  claim 28 , further comprising:
 generating a graph representation comprising:
 first node data corresponding to the first portion of the natural language input, 
 second node data corresponding to the second portion of the natural language input, and 
 connection data representing a result of processing of the second node data is to be used to update the first node data; 
   wherein populating the variable comprises generating updated first node data by replacing data representing the variable, in the first node data, with the entity data.   
     
     
         30 . The computer-implemented method of  claim 21 , wherein performing the NLU processing comprises:
 performing intent classification to determine intent data; and   determining the action based at least in part on the intent data.   
     
     
         31 . A system, comprising:
 at least one processor; and   at least one memory comprising instructions that, when executed by the at least one processor, cause the system to:
 receive first data representing a natural language input; 
 determine the natural language input corresponds to a first query and a second query, the first query represented by a first portion of the natural language input and the second query represented by a second portion of the natural language input; 
 process the first portion of the natural language input to determine first response data; 
 perform natural language understanding (NLU) processing using the first response data and the second portion of the natural language input to determine an action responsive to the natural language input; and 
 cause the action to be performed to determine output data. 
   
     
     
         32 . The system of  claim 31 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
 use a machine learning component to determine the natural language input corresponds to a plurality of queries.   
     
     
         33 . The system of  claim 32 , wherein the first data comprises audio data representing a spoken natural language input, and wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
 perform automatic speech recognition (ASR) processing on the audio data to generate first ASR output data representing a first ASR hypothesis;   process the first ASR output data using the machine learning component to determine the natural language input corresponds to the plurality of queries; and   use the first ASR hypothesis to determine the first portion of the natural language input corresponding to the first query.   
     
     
         34 . The system of  claim 32 , wherein the instructions that cause the system to determine the natural language input corresponds to the plurality of queries comprise instructions that, when executed by the at least one processor, cause the system to determine that intent classification processing of the first data is likely to result in output failing to satisfy a threshold condition. 
     
     
         35 . The system of  claim 31 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
 determine second data corresponding to a first semantic tag corresponding to the first portion of the natural language input; and   determine third data corresponding to a second semantic tag corresponding to the second portion of the natural language input,   wherein the instructions that cause the system to determine the natural language input corresponds to a first query and a second query are based at least in part on the second data and the third data.   
     
     
         36 . The system of  claim 31 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
 determine the first query corresponds to information retrieval processing; and   perform information retrieval processing to determine the first response data, wherein the first response data represents an entity corresponding to the first query.   
     
     
         37 . The system of  claim 36 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
 determine entity data corresponding to the entity,   wherein the output data is based at least in part on the entity data.   
     
     
         38 . The system of  claim 36 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
 determine entity data corresponding to the entity;   determine the entity data corresponds to a variable of the second query; and   populate the variable using the entity data for purposes of the NLU processing.   
     
     
         39 . The system of  claim 38 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
 generate a graph representation comprising:
 first node data corresponding to the first portion of the natural language input, 
 second node data corresponding to the second portion of the natural language input, and 
 connection data representing a result of processing of the second node data is to be used to update the first node data; 
   wherein population of the variable comprises generation of updated first node data by replacing data representing the variable, in the first node data, with the entity data.   
     
     
         40 . The system of  claim 31 , wherein the instructions that cause the system to perform the NLU processing comprise instructions that, when executed by the at least one processor, cause the system to:
 perform intent classification to determine intent data; and   determine the action based at least in part on the intent data.