Methods and Systems for Searching Using Spoken Input and User Context Information

Abstract

Systems and methods for searching using spoken input and user context information are disclosed. In systems and methods according to embodiments of the invention, continuous speech natural language queries are accepted from a user using a client device. Speech processing tasks are divided between the client device and one or more server systems. Along with speech in compressed form, the client device transmits to the one or more server systems user context information, which is used by the one or more server systems to dynamically load one or more of a dictionary, a grammar, or a vocabulary.

1 .- 28 . (canceled) 
   
   
       29 . A method for conducting a search using spoken input, comprising:
 accepting continuous user speech in the form of a natural language query using a client device;   processing the user speech on the client device using a first processing routine to form processed speech;   transmitting the processed speech to a server system using an application-level Internet protocol;   transmitting user context information from the client device to the server system;   dynamically selecting one or more of a grammar, a dictionary, or a vocabulary on the server system based on the user context information;   recognizing the processed speech on the server system using a second processing routine to form recognized speech;   searching a plurality of topics in a data repository using the recognized speech and environment variables derived from the user context information to arrive at a best answer; and   returning at least the best answer to the client device.   
   
   
       30 . The method of  claim 29 , further comprising initiating the accepting after a button on the client device is depressed. 
   
   
       31 . The method of  claim 29 , wherein the processed speech comprises speech in compressed form. 
   
   
       32 . The method of  claim 31 , wherein the speech in compressed form comprises a plurality of extracted features characteristic of the user speech. 
   
   
       33 . The method of  32 , wherein the plurality of extracted features comprise spectral representations of the user speech. 
   
   
       34 . The method of  claim 33 , wherein the extracted features comprise mel-frequency cepstral coefficients (MFCCs). 
   
   
       35 . The method of  claim 29 , wherein the Internet protocol comprises hypertext transfer protocol (HTTP). 
   
   
       36 . The method of  claim 29 , wherein transmitting the processed speech to the server system comprises transmitting the processed speech to the server system in streaming packets. 
   
   
       37 . A system for conducting a search using spoken input, comprising:
 a speech recognizer that receives continuous user speech in compressed form and processes the speech to recognize a natural language query;   a natural language engine that selects one or more of a grammar, a dictionary, or a vocabulary based, at least in part, on user context information supplied to the natural language engine and processes the recognized natural language query to formulate one or more queries to data repositories, the natural language engine being configured to dynamically switch between the selected grammar, dictionary or vocabulary and other grammars, dictionaries, and vocabularies based on the user context information; and   a search engine that accepts the one or more queries to data repositories and executes those queries on the data repositories to arrive at a best answer to the natural language query.   
   
   
       38 . The system of  claim 37 , wherein the speech recognizer, the natural language engine, and the search engine reside in one or more server systems. 
   
   
       39 . The system of  claim 38 , further comprising one or more client devices, each client device including:
 a speech processor that receives the continuous user speech and processes the continuous user speech into the compressed form; and   a transmission circuit that
 formats the continuous user speech in compressed form into a transmission format for transmission to the one or more server systems, and 
 transmits the continuous user speech in the transmission format to the one or more server systems using an application-level Internet protocol. 
   
   
   
       40 . The system of  claim 39 , wherein the Internet protocol comprises hypertext transfer protocol (HTTP). 
   
   
       41 . The system of  claim 39 , further comprising a designated button on the client device that, when pressed, begins speech processing using the speech processor. 
   
   
       42 . The system of  claim 39 , the system further comprising a second transmission circuit that returns at least the best answer from the search engine to a requesting one of the one or more client devices. 
   
   
       43 . The system of  claim 37 , wherein the continuous user speech in compressed form comprises a plurality of extracted features characteristic of the continuous user speech. 
   
   
       44 . The system of  claim 43 , wherein the plurality of extracted features comprise spectral representations of the user speech. 
   
   
       45 . The system of  claim 44 , wherein the plurality of extracted features comprise mel-frequency cepstral coefficients (MFCCs). 
   
   
       46 . A method for conducting a search using spoken input, comprising:
 accepting continuous speech from a user in the form of a natural language query using an application program running on a client device, the speech being associated with a resource the user wishes to locate;   processing the user speech on the client device using a first processing routine to form processed speech;   transmitting the processed speech to a server system using an application-level Internet protocol;   transmitting user context information from the client device to the server system;   dynamically selecting one or more of a grammar, a dictionary, or a vocabulary on the server system based on the user context information;   recognizing the processed speech on the server system using a second processing routine to form recognized speech;   searching a plurality of topics in a data repository using the recognized speech and environment variables derived from the user context information to arrive at a best answer associated with the resource; and   returning at least the best answer to the client device.   
   
   
       47 . The method of  claim 46 , wherein transmitting the processed speech to the server system comprises transmitting the processed speech to the server system in streaming packets. 
   
   
       48 . The method of  claim 46 , the processed speech is recognized to a specified confidence level. 
   
   
       49 . The method of  claim 46 , wherein the speech in compressed form comprises a plurality of extracted features characteristic of the user speech. 
   
   
       50 . The method of  claim 46 , wherein the extracted features comprise mel-frequency cepstral coefficients (MFCCs).