Variable format, variable sparsity matrix multiplication instruction

Abstract

Disclosed embodiments relate to a variable format, variable sparsity matrix multiplication (VFVSMM) instruction. In one example, a processor includes fetch and decode circuitry to fetch and decode a VFVSMM instruction specifying locations of A, B, and C matrices having (M×K), (K×N), and (M×N) elements, respectively, execution circuitry, responsive to the decoded VFVSMM instruction, to: route each row of the specified A matrix, staggering subsequent rows, into corresponding rows of a (M×N) processing array, and route each column of the specified B matrix, staggering subsequent columns, into corresponding columns of the processing array, wherein each of the processing units is to generate K products of A-matrix elements and matching B-matrix elements having a same row address as a column address of the A-matrix element, and to accumulate each generated product with a corresponding C-matrix element.

What is claimed is: 
     
         1 . A processor comprising:
 a cache to store data;   a plurality of cores coupled to the cache, a core of the plurality of cores comprising:
 execution circuitry to execute at least one instruction to perform multiply-accumulate operations with a first source matrix and a second source matrix in accordance with a selected operational mode to generate a result matrix, the selected operational mode comprising either a first operational mode in which at least the first source matrix is a sparse matrix having non-zero data elements located at certain positions, and a second operational mode in which both the first and the second source matrices are dense matrices, 
 wherein when in the first operational mode, the first source matrix is to be stored in a compressed format that identifies the positions of the non-zero data elements, the execution circuitry further comprising:
 a plurality of multiply-accumulate circuits to multiply the non-zero data elements of the first source matrix by corresponding data elements of the second source matrix identified based on the positions in the compressed format to generate a plurality of products and to add the plurality of products to accumulated values to generate the result matrix. 
 
   
     
     
         2 . The processor of  claim 1  wherein the first source matrix is to be stored in a packed sparse format if the first source matrix is a sparse matrix, the packed sparse format comprising a specifier accompanying each matrix element and specifying a logical position of the matrix element within the first source matrix. 
     
     
         3 . The processor of  claim 1  wherein a sparse matrix comprises a matrix having a proportion of non-zero data elements which is less than or equal to a threshold. 
     
     
         4 . The processor of  claim 3  wherein the threshold comprises a value of 1. 
     
     
         5 . The processor of  claim 1  wherein the instruction is to indicate the selected operational mode. 
     
     
         6 . The processor of  claim 5  wherein the instruction comprises a sparse matrix multiply instruction to indicate the first operational mode or a dense matrix multiply instruction to indicate the second operational mode. 
     
     
         7 . The processor of  claim 1  wherein when in the second operational mode, the plurality of multiply-accumulate circuits are to multiply the data elements of the first source matrix, including any zero data elements, by data elements of the second source matrix. 
     
     
         8 . The processor of  claim 1  wherein the execution circuitry is operable to perform matrix multiply operations with a plurality of different data types to encode the data elements of the first source matrix, the second source matrix, and the result matrix. 
     
     
         9 . The processor of  claim 1  wherein the data types include one or more of: 16-bit floating point, 32-bit floating point, 8-bit integer, and 16-bit integer. 
     
     
         10 . The processor of  claim 7  wherein the instruction comprises a plurality of fields including a first field to specify an opcode indicating a first operational mode or a second operational mode, a second field to identify the result matrix, a third field to identify the first source matrix, and a fourth field to identify the second source matrix. 
     
     
         11 . The processor of  claim 1  wherein the first source matrix and the second source matrix comprise input values for a machine-learning application. 
     
     
         12 . A processor comprising:
 fetch and decode circuitry to fetch and decode a first matrix multiplication instruction having fields to specify locations where each of A, B, and C matrices having (M×K), (K×N), and (M×N) elements, respectively, is stored;   execution circuitry, when operating in a first mode in response to the decoded first matrix multiplication instruction, to route each row of the specified A matrix, staggering subsequent rows, into a corresponding row of a processing array having (M×N) processing units, and route each column of the specified B matrix, staggering subsequent columns, into a corresponding column of the processing array, and   wherein the execution circuitry, when operating in a second mode in response to the decoded first matrix multiplication instruction, is to route each row of the specified A matrix, staggering subsequent rows, into the corresponding row of the processing array, and to route each column of the specified B matrix into the corresponding column of the processing array, and wherein each of the (M×N) processing units, operating in the second mode, is to:
 determine whether an address match exists between the specified logical row address of the B-matrix element and the column address of the A-matrix element; and 
 when the match exists, generate the product, and when no match exists, hold the A-matrix element and pass the B-matrix element when the column address of the A-matrix element is larger than the specified logical row address of the B-matrix element, and, otherwise, hold the B-matrix element and pass the A-matrix element. 
   
     
     
         13 . The processor of  claim 12  wherein when the execution circuitry is operating in the first mode, each of the (M×N) processing units is to generate K products of matching A-matrix and B-matrix elements received from the specified A and B matrices, respectively, a match to exist when the B-matrix element has a row address matching a column address of the A-matrix element, and to accumulate each generated product with a corresponding element of the specified C-matrix having a same relative position as a position of the processing unit in the processing array. 
     
     
         14 . The processor of  claim 12 , wherein the execution circuitry is to route each row of the specified A matrix and each column of the specified B matrix at a rate of one element per clock cycle and to stagger each subsequent row and subsequent column by one clock cycle, and wherein each of the (M×N) processing units is to infer column and row addresses of each received A-matrix and B-matrix element based on a clock cycle and on a relative position of the processing unit within the processing array.