Convolutional neural networks

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for keyword spotting. One of the methods includes training, by a keyword detection system, a convolutional neural network for keyword detection by providing a two-dimensional set of input values to the convolutional neural network, the input values including a first dimension in time and a second dimension in frequency, and performing convolutional multiplication on the two-dimensional set of input values for a filter using a frequency stride greater than one to generate a feature map.

What is claimed is: 
     
         1 . A speech recognition model comprising:
 a convolution neural network comprising:
 a first convolution neural network layer configured to generate a first output from a two-dimensional set of input values, the set of input values comprising input values across a first dimension in time and input values across a second dimension in frequency, and the first output comprising a feature map; 
 a second convolution neural network layer different than the first convolution neural network layer, the second convolution neural network layer configured to receive the feature map generated by the first convolution neural network layer and generate a second output using the feature map; and 
 a linear low rank layer configured to receive the second output generated by the second convolution neural network layer and generate a third output using the second output; and 
   a deep neural network configured to receive the third output generated by the linear low rank layer and generate a fourth output using the third output.   
     
     
         2 . The speech recognition model of  claim 1 , further comprising a softmax layer configured to receive the fourth output from the deep neural network and generate a final output for the neural network system. 
     
     
         3 . The speech recognition model of  claim 2 , wherein the deep neural network comprises the softmax layer. 
     
     
         4 . The speech recognition model of  claim 2 , wherein an accuracy of the final output is used to update the convolution neural network. 
     
     
         5 . The speech recognition model of  claim 1 , wherein:
 the feature map comprises a first matrix;   the second output comprises a second matrix; and   the linear low rank layer is configured to generate the third output by:
 creating a vector from the second matrix; and 
 generating the third output using the vector. 
   
     
     
         6 . The speech recognition model of  claim 1 , wherein the first convolution neural network layer is configured to generate the feature map by performing convolution multiplication on the two-dimensional set of input values for a filter that has a time span that extends over all of the input values in the first dimension and a frequency span that extends over less than all of the input values in the second dimension. 
     
     
         7 . The speech recognition model of  claim 6 , wherein performing the convolution multiplication on the two-dimensional set of input values comprises performing the convolution multiplication on the two-dimensional set of input values for the filter using a frequency stride greater than one and a time stride equal to one. 
     
     
         8 . The speech recognition model of  claim 1 , wherein the convolution neural network is configured to:
 receive an audio signal encoding an utterance; and   analyze the audio signal to identify a command included in the utterance.   
     
     
         9 . The neural network system of  claim 1 , wherein the convolution neural network further comprises at least one max-pooling layer configured to remove variability in the input values in the first dimension and the input values in the second dimension. 
     
     
         10 . The speech recognition model of  claim 1 , wherein the first convolution neural network layer comprises a filter size in time that spans two-thirds an overall size of the input values across the first dimension in time. 
     
     
         11 . A method for training a speech recognition model, the method comprising:
 generating, by a first layer of a convolution neural network, a first output from a two-dimensional set of input values, the set of input values comprising input values across a first dimension in time and input values across a second dimension in frequency, and the first output comprising a feature map;   generating, by a second layer of the convolution neural network, a second output using the feature map;   generating, by a linear low rank layer, a third output using the second output;   generating, by a deep neural network, a fourth output using the third output; and   generating, by a softmax layer, a final output of the speech recognition model using the fourth output.   
     
     
         12 . The method of  claim 11 , wherein the deep neural network comprises the softmax layer. 
     
     
         13 . The method of  claim 11 , further comprising using an accuracy of the final output to update the convolution neural network. 
     
     
         14 . The method of  claim 11 , wherein:
 the feature map comprises a first matrix;   the second output comprises a second matrix; and   generating, by the linear low rank layer, the third output using the second output comprises:
 creating a vector from the second matrix; and 
 generating the third output using the vector. 
   
     
     
         15 . The method of  claim 11 , further comprising using the convolution neural network for keyword detection by:
 receiving an audio signal encoding an utterance;   analyzing the audio signal to identify a command included in the utterance; and   performing an action that corresponds to the command.   
     
     
         16 . The method of  claim 11 , wherein generating, by the first layer of the convolution neural network, the first output comprises performing convolution multiplication on the two-dimensional set of input values for a filter that has a time span that extends over all of the input values in the first dimension and a frequency span that extends over less than all of the input values in the second dimension. 
     
     
         17 . The method of  claim 16 , wherein performing the convolution multiplication on the two-dimensional set of input values comprises performing the convolution multiplication on the two-dimensional set of input values for the filter using a frequency stride greater than one and a time stride equal to one. 
     
     
         18 . The method of  claim 11 , further comprising removing, by at least one max-pooling layer of the convolution neural network, variability in the input values in the first dimension and the input values in the second dimension. 
     
     
         19 . The method of  claim 11 , wherein the first layer of the convolution neural network comprises a filter size in time that spans two-thirds an overall size of the input values across the first dimension in time. 
     
     
         20 . The method of  claim 11 , further comprising, after training the speech recognition model, providing the trained speech recognition model to a device for use by the device for keyword detection of one or more key phrases.