Efficient direct convolution using simd instructions

Abstract

A computer comprising one or more processors offering vector instructions may implement a direct convolution on a source data set. The source data set may be one-dimensional or multi-dimensional. For a given vector width, w, of the vector instructions, w consecutive data elements of the output data set are computed in parallel using vector instructions. For multi-dimensional data sets, multiple vectors of the output data set are computed for a single load of a set of vectors from the source data set. New vector instructions are disclosed to improve the performance of the convolution and to enable full utilization of the arithmetic logic units within the one or more processors.

1 .- 20 . (canceled) 
     
     
         21 . A system, comprising:
 a processor configured to execute a vector instruction to generate an output vector from a first input vector and a second input vector, wherein the output vector, the first input vector and the second input vector individually comprise a plurality of data lanes, and wherein to execute a vector instruction the processor is configured to:
 concatenate the first source vector and the second source vector to generate a combined source vector; 
 extract the output vector from the combined source vector starting at an offset greater than zero, wherein the output vector comprises at least one data lane of the first source vector and at least one data lane of the second source vector; and 
 write the extracted output vector to a vector register of the processor. 
   
     
     
         22 . The system of  claim 21 , wherein the first source vector and the second source vector are accessed from respective source vector registers of the processor identified by respective operands of the vector instruction. 
     
     
         23 . The system of  claim 21 , wherein the data lane offset is identified as an operand of the vector instruction. 
     
     
         24 . The system of  claim 21 , wherein to extract the output vector from the combined source vector starting at the data lane offset, the processor is configured to:
 extract a number of data lanes according to an extraction width identified as an operand of the vector instruction, wherein the extraction width is less than a number of data lanes of the vector register; and   add a number of rightmost data lanes containing respective zero values to the output vector to generate an extracted output vector having a number of data lanes matching the number of data lanes of the vector register.   
     
     
         25 . The system of  claim 21 , wherein the vector instruction is executed as part of performing a convolution on a vector, wherein the output vector is a left source vector for the convolution, wherein the second source vector is a center source vector for the convolution, and wherein to perform the convolution, the processor is further configured to:
 load a plurality of source vector registers with respective source vectors of the vector including the first source vector, the second source vector and a third source vector;   concatenate the second source vector and the third source vector to generate another combined source vector;   extract the right source vector from the other combined source vector starting at another data lane offset, wherein the right source vector comprises at least one data lane of the second source vector and at least one data lane of the third source vector; and   write the right source vector to another vector register of the processor;   load a kernel vector comprising a plurality of weighting values including a center weighting value, a left weighting value and a right weighting value; and   generate one or more output vectors respectively comprising weighted sums of a plurality source vectors including the left source vector, the center source vector and the right source vector, wherein to generate a particular output vector of the one or more output vectors, the processor is configured to:
 load a scalar weighting value from a lane of the kernel vector, the lane specified to the particular vector instruction according to the particular output vector and a particular source vector of the plurality of source vectors specified as an operand of the particular vector instruction; 
 scale respective ones of a plurality of data lanes of the particular source vector by the loaded scalar weighting value to generate a scaled vector; 
 add the scaled vector to an accumulator vector stored in a vector accumulator to generate a new output vector; and 
 store the new output vector in the vector accumulator. 
   
     
     
         26 . The system of  claim 25 , wherein the other data lane offset is determined according to the data lane offset and a number of data lanes of the vector register. 
     
     
         27 . The system of  claim 25 , wherein the processor is configured to perform the convolution as part of a convolutional neural network. 
     
     
         28 . A method, comprising:
 executing, by a processor, a vector instruction to generate an output vector from a first input vector and a second input vector, wherein the output vector, the first input vector and the second input vector individually comprise a plurality of data lanes, and wherein the executing comprises:
 concatenating the first source vector and the second source vector to generate a combined source vector; 
 extracting the output vector from the combined source vector starting at a data lane offset, wherein the output vector comprises at least one data lane of the first source vector and at least one data lane of the second source vector; and 
 writing the extracted output vector to a vector register of the processor. 
   
     
     
         29 . The method of  claim 28 , wherein the first source vector and the second source vector are accessed from respective source vector registers of the processor identified by respective operands of the vector instruction. 
     
     
         30 . The method of  claim 28 , wherein the data lane offset is identified as an operand of the vector instruction. 
     
     
         31 . The method of  claim 28 , wherein extracting the output vector from the combined source vector starting at the data lane offset comprises:
 extracting a number of data lanes according to an extraction width identified as an operand of the vector instruction, wherein the extraction width is less than a number of data lanes of the vector register; and   adding a number of rightmost data lanes containing respective zero values to the output vector to generate an extracted output vector having a number of data lanes matching the number of data lanes of the vector register.   
     
     
         32 . The method of  claim 28 , wherein the vector instruction is executed as part of performing a convolution on a vector, wherein the output vector is a left source vector for the convolution, wherein the second source vector is a center source vector for the convolution, and wherein performing the convolution further comprises:
 loading a plurality of source vector registers with respective source vectors of the vector including the first source vector, the second source vector and a third source vector;   executing another vector instruction to generate an right source vector from the second input vector and the third source vector, comprising:
 concatenating the second source vector and the third source vector to generate another combined source vector; 
 extracting the right source vector from the other combined source vector starting at another data lane offset, wherein the right source vector comprises at least one data lane of the second source vector and at least one data lane of the third source vector; and 
 writing the right source vector to another vector register of the processor; 
   loading a kernel vector comprising a plurality of weighting values including a center weighting value, a left weighting value and a right weighting value; and   generating one or more output vectors respectively comprising weighted sums of a plurality source vectors including the left source vector, the center source vector and the right source vector, wherein generating a particular output vector of the one or more output vectors comprises executing a particular vector instruction performing:
 loading a scalar weighting value from a lane of the kernel vector, the lane specified to the particular vector instruction according to the particular output vector and a particular source vector of the plurality of source vectors specified as an operand of the particular vector instruction; 
 scaling respective ones of a plurality of data lanes of the particular source vector by the loaded scalar weighting value to generate a scaled vector; 
 adding the scaled vector to an accumulator vector stored in a vector accumulator to generate a new output vector; and 
 storing the new output vector in the vector accumulator. 
   
     
     
         33 . The method of  claim 32 , wherein the other data lane offset is determined according to the data lane offset and a number of data lanes of the vector register. 
     
     
         34 . The method of  claim 32 , wherein the processor is configured to perform the convolution as part of a convolutional neural network. 
     
     
         35 . One or more non-transitory computer-accessible storage media storing program instructions that when executed on or across one or more computing devices cause the one or more computing devices to perform:
 executing, by a processor, a vector instruction to generate an output vector from a first input vector and a second input vector, wherein the output vector, the first input vector and the second input vector individually comprise a plurality of data lanes, and wherein the executing comprises:
 concatenating the first source vector and the second source vector to generate a combined source vector; 
 extracting the output vector from the combined source vector starting at a data lane offset, wherein the output vector comprises at least one data lane of the first source vector and at least one data lane of the second source vector; and 
 writing the extracted output vector to a vector register of the processor. 
   
     
     
         36 . The one or more non-transitory computer-accessible storage media of  claim 35 , wherein the first source vector and the second source vector are accessed from respective source vector registers of the processor identified by respective operands of the vector instruction. 
     
     
         37 . The one or more non-transitory computer-accessible storage media of  claim 35 , wherein the data lane offset is identified as an operand of the vector instruction. 
     
     
         38 . The one or more non-transitory computer-accessible storage media of  claim 35 , wherein extracting the output vector from the combined source vector starting at the data lane offset comprises:
 extracting a number of data lanes according to an extraction width identified as an operand of the vector instruction, wherein the extraction width is less than a number of data lanes of the vector register; and   adding a number of rightmost data lanes containing respective zero values to the output vector to generate an extracted output vector having a number of data lanes matching the number of data lanes of the vector register.   
     
     
         39 . The one or more non-transitory computer-accessible storage media of  claim 35 , wherein the vector instruction is executed as part of performing a convolution on a vector, wherein the output vector is a left source vector for the convolution, wherein the second source vector is a center source vector for the convolution, and wherein performing the convolution further comprises:
 loading a plurality of source vector registers with respective source vectors of the vector including the first source vector, the second source vector and a third source vector;   executing another vector instruction to generate an right source vector from the second input vector and the third source vector, comprising:
 concatenating the second source vector and the third source vector to generate another combined source vector; 
 extracting the right source vector from the other combined source vector starting at another data lane offset, wherein the right source vector comprises at least one data lane of the second source vector and at least one data lane of the third source vector; and 
 writing the right source vector to another vector register of the processor; 
   loading a kernel vector comprising a plurality of weighting values including a center weighting value, a left weighting value and a right weighting value; and   generating one or more output vectors respectively comprising weighted sums of a plurality source vectors including the left source vector, the center source vector and the right source vector, wherein generating a particular output vector of the one or more output vectors comprises executing a particular vector instruction performing:
 loading a scalar weighting value from a lane of the kernel vector, the lane specified to the particular vector instruction according to the particular output vector and a particular source vector of the plurality of source vectors specified as an operand of the particular vector instruction; 
 scaling respective ones of a plurality of data lanes of the particular source vector by the loaded scalar weighting value to generate a scaled vector; 
 adding the scaled vector to an accumulator vector stored in a vector accumulator to generate a new output vector; and 
 storing the new output vector in the vector accumulator. 
   
     
     
         40 . The one or more non-transitory computer-accessible storage media of  claim 39 , wherein the processor is configured to perform the convolution as part of a convolutional neural network.