User interaction with building controller device using a remote server and a duplex connection

Abstract

An HVAC controller may be controlled in response to a natural language audio message that is not recognizable by the HVAC controller as a command, where the natural language audio message is translated into a command recognizable by the HVAC controller. Voice recognition software identifies a trigger phrase included in the natural language audio message and in response the HVAC controller performs an action. In response to identifying a trigger phrase at the HVAC controller, the HVAC controller may establish a single duplex connection with a remote server having a voice interaction module. An end user may then have a continuous dialog with the voice interaction module via the HVAC controller over the established single duplex connection. The voice interaction module may allow for an end user to interrupt the dialog at any time, as desired.

What is claimed is: 
     
       1. A thermostat configured to control one or more HVAC components of an HVAC system, the thermostat comprising:
 a housing, the housing configured to house:
 a temperature sensor; 
 a microphone; 
 a controller operatively coupled to the temperature sensor and the microphone; and 
 wherein the controller is configured to identify an audio trigger received by the microphone, and in response to identifying the audio trigger, establish a full duplex connection between the thermostat and a remote server, wherein the audio trigger is a trigger phrase contained in an audio voice stream. 
 
 
     
     
       2. The thermostat of  claim 1 , wherein the audio trigger is identified by the controller of the thermostat. 
     
     
       3. The thermostat of  claim 1 , further comprising:
 a speaker at least partially housed by the housing; and 
 wherein the controller is configured to play an audio voice message on the speaker, wherein the audio voice message is received from the remote server via the established full duplex connection. 
 
     
     
       4. The thermostat of  claim 1 , wherein the established full duplex connection between the thermostat and the remote server includes a plurality of sub-channels. 
     
     
       5. The thermostat of  claim 4 , wherein the plurality of sub-channels include a first sub-channel configured to send audio data to the remote server, a second sub-channel configured to receive audio data from the remote server, and a third sub-channel configured to receive one or more controller understandable commands from the remote server. 
     
     
       6. The thermostat of  claim 5 , wherein the controller of the thermostat is configured to receive audio data from the remote server over the second sub-channel while at the same time send audio data from the controller to the remote server over the first sub-channel. 
     
     
       7. The thermostat of  claim 1 , wherein the established full duplex connection between the thermostat and the remote server uses a single Transmission Control Protocol (TCP). 
     
     
       8. The thermostat of  claim 1 , wherein the full duplex connection between the controller and the remote server is established using a multi-channel protocol. 
     
     
       9. A method of operating a thermostat of an HVAC system, the method comprising:
 the thermostat identifying an audible trigger phrase provided by a user; 
 in response to identifying the trigger phrase, the thermostat initiating a full duplex connection between the thermostat and a remote server; 
 in response to identifying the trigger phrase, the thermostat providing an audible request for a voice command from the user; 
 in response to the audible request, the thermostat configured to receive a voice stream from the user, the voice stream including a voice command; 
 the thermostat providing the received voice stream from the user to the remote server via the full duplex connection; and 
 the remote server processing the voice stream to identify the voice command. 
 
     
     
       10. The method of  claim 9 , wherein
 the voice stream is received by the thermostat from the user via a microphone of the thermostat after providing the audible request via a speaker of the thermostat. 
 
     
     
       11. The method of  claim 9 , wherein identifying the audible trigger phrase includes identifying the trigger phrase in a voice stream that is received via a microphone of the thermostat. 
     
     
       12. The method of  claim 9 , further comprising:
 receiving an audio message at the thermostat from the remote server after the remote server has identified the voice command. 
 
     
     
       13. The method of  claim 12 , further comprising:
 sending a voice stream from the thermostat to the remote server over the full duplex connection while receiving the audio message from the remote server. 
 
     
     
       14. A server configured to communicate with one or more thermostats of one or more HVAC systems, the server comprising:
 a communications module for establishing a full duplex connection with a thermostat upon a request by the thermostat, wherein the thermostat includes a microphone configured to receive an audio trigger that initiates the request by the thermostat; 
 a voice interaction module operatively coupled to the communications module, wherein the voice interaction module is configured to provide an interactive audio dialog with a user of the thermostat by identifying one or more words in a voice stream received from the thermostat over a first channel of the full duplex connection, and in response, sending a return voice message to the thermostat over a second channel of the full duplex connection; and 
 wherein in response to identifying one or more words in a voice stream, the voice interaction module is further configured to determine and then send a control command to the thermostat over a third channel of the full duplex connection to change at least one control parameter of the thermostat. 
 
     
     
       15. The server of  claim 14 , wherein the return voice message is sent to the thermostat over the full duplex connection while still receiving the voice stream from the thermostats. 
     
     
       16. The server of  claim 15 , wherein the voice interaction module is configured to interrupt the return voice message if the voice interaction module identifies one or more subsequent words in the voice stream that indicate that the return voice message is no longer applicable. 
     
     
       17. The server of  claim 14 , wherein the voice interaction module is configured to terminate the full duplex connection between the server and the thermostat in response to the voice interaction module determining that the interactive audio dialogue is over.