Keyword-based audio source localization

Abstract

Systems, apparatuses, and methods are described for determining a direction associated with a detected spoken keyword, forming an acoustic beam in the determined direction, and listening for subsequent speech using the acoustic beam in the determined direction.

The invention claimed is: 
     
       1. A method comprising:
 receiving, by a computing device and from one or more microphones of a plurality of microphones, one or more indications of first speech; 
 determining, based on the one or more indications of the first speech, that the first speech comprises a keyword; 
 based on an absence of a currently active acoustic beam associated with any of the plurality of microphones, forming an acoustic beam, using a portion of the plurality of microphones, to listen for second speech associated with the keyword; 
 listening, using the formed acoustic beam and by ignoring one or more indications of third speech received via a microphone, of the plurality of microphones, not used to form the acoustic beam, for the second speech; and 
 recognizing the second speech to generate an indication of recognized speech. 
 
     
     
       2. The method of  claim 1 , wherein the determining that the first speech comprises a keyword comprises determining that the keyword was spoken by an authorized person. 
     
     
       3. The method of  claim 1 , wherein the one or more indications of the first speech comprise indications that the first speech was detected in a plurality of listening zones, and wherein the formed acoustic beam is different from each of the plurality of listening zones. 
     
     
       4. The method of  claim 1 , wherein the received one or more indications of the first speech comprise indications that the first speech was detected in two listening zones, and wherein the formed acoustic beam is between the two listening zones. 
     
     
       5. The method of  claim 1 , further comprising:
 receiving one or more images of a user associated with the second speech, wherein the recognizing the second speech is further based on the one or more images. 
 
     
     
       6. The method of  claim 1 , further comprising:
 determining, based on signals from the plurality of microphones, audio characteristics associated with the first speech; and 
 determining, based on the audio characteristics, a direction of the formed acoustic beam. 
 
     
     
       7. The method of  claim 1 , further comprising:
 determining, based on the indication of the recognized speech, a device; 
 determining, based on the indication of the recognized speech, a command; and 
 sending the command to the device. 
 
     
     
       8. The method of  claim 1 , wherein the one or more indications of the first speech comprise indications that the first speech was detected in a plurality of listening zones, wherein the plurality of microphones comprises a plurality of microphone arrays, and wherein each of the microphone arrays is associated with a different listening zone of the plurality of listening zones. 
     
     
       9. A method comprising:
 receiving, by a computing device and from a plurality of microphones, indications of first speech detected in a plurality of listening zones; 
 determining, based on the received indications, that the first speech comprises a keyword; 
 detecting, using an acoustic beam that is formed using a portion of the plurality of microphones and that is different from each of the plurality of listening zones, and by ignoring one or more indications of third speech received via a microphone, of the plurality of microphones, not used to form the acoustic beam, second speech associated with the keyword; and 
 recognizing the second speech to generate an indication of recognized speech. 
 
     
     
       10. The method of  claim 9 , further comprising:
 determining, based on signals from the plurality of microphones, audio characteristics associated with the first speech; 
 determining, based on the audio characteristics, a direction for the formed acoustic beam. 
 
     
     
       11. The method of  claim 10 , wherein the audio characteristics comprise one or more of signal-to-noise ratio (SNR), frequency, or amplitude. 
     
     
       12. The method of  claim 9 , wherein the determining that the first speech comprises a keyword comprises determining that the keyword was spoken by an authorized person. 
     
     
       13. The method of  claim 9 , wherein the plurality of listening zones comprises two listening zones, and wherein a direction for the formed acoustic beam is between the two listening zones. 
     
     
       14. The method of  claim 9 , further comprising:
 receiving one or more images of a user associated with the second speech, wherein the recognizing the second speech is further based on the one or more images. 
 
     
     
       15. The method of  claim 9 , further comprising:
 determining, based on the indication of the recognized speech, a device; 
 determining, based on the indication of the recognized speech, a command; and 
 sending the command to the device. 
 
     
     
       16. The method of  claim 9 , further comprising:
 determining, as a direction for the formed acoustic beam, a weighted average of directions associated with the plurality listening zones. 
 
     
     
       17. A method comprising:
 receiving, by a computing device and from one or more microphones of a plurality of microphones, one or more indications of first speech; 
 determining, based on the one or more indications of the first speech, that the first speech comprises a keyword; 
 based on a determination that the keyword was spoken by an authorized person, detecting, using an acoustic beam formed using some of the plurality of microphones and by ignoring one or more indications of third speech received via a microphone, of the plurality of microphones, not used to form the acoustic beam, second speech associated with the keyword; and 
 recognizing the second speech to generate an indication of recognized speech. 
 
     
     
       18. The method of  claim 1 , further comprising:
 inactivating, based on determining that the second speech has ended, the formed acoustic beam. 
 
     
     
       19. The method of  claim 9 , further comprising:
 inactivating, based on determining that the second speech has ended, the formed acoustic beam. 
 
     
     
       20. The method of  claim 17 , further comprising:
 inactivating, based on determining that the second speech has ended, the formed acoustic beam.