System to analyze and enhance software based on graph attention networks

Abstract

Systems, apparatuses and methods may provide for technology that generates a dependence graph based on a plurality of intermediate representation (IR) code instructions associated with a compiled program code, generates a set of graph embedding vectors based on the plurality of IR code instructions, and determines, via a neural network, one of an analysis of the compiled program code or an enhancement of the program code based on the dependence graph and the set of graph embedding vectors. The technology may provide a graph attention neural network that includes a recurrent block and at least one task-specific neural network layer, the recurrent block including a graph attention layer and a transition function. The technology may also apply dynamic per-position recurrence-halting to determine a number of recurring steps for each position in the recurrent block based on adaptive computation time.

We claim: 
     
       1. A computing system comprising:
 a memory to store compiled program code, the code including a plurality of intermediate representation (IR) code instructions; 
 a processor coupled to the memory, the processor including one or more substrates and logic coupled to the one or more substrates, wherein the logic is implemented at least partly in one or more of configurable logic or fixed-functionality hardware logic, the logic coupled to the one or more substrates to:
 generate during inference a dependence graph based on the plurality of IR code instructions; 
 generate during inference a set of graph embedding vectors based on the plurality of IR code instructions, wherein the embedding vectors are generated independently of the dependence graph, and wherein each graph embedding vector is to model as a graph an operation, an associated argument and an associated argument type for an instruction of the plurality of IR code instructions; 
 apply dynamic per-position recurrence-halting using adaptive computation time to determine the number of recurring steps for each position in the recurrent block; and 
 perform during inference, via a graph attention neural network, one of an analysis of the compiled program code or an enhancement of the compiled program code based on the dependence graph and the set of graph embedding vectors, wherein the graph attention neural network comprises a recurrent block and a task-specific neural network layer, the recurrent block comprising a graph attention layer and a transition function, and wherein the number of recurring steps for each position in the recurrent block is determined automatically. 
 
 
     
     
       2. The computing system of  claim 1 , wherein the dependence graph is to map dependencies between instructions in the plurality of IR code instructions. 
     
     
       3. The computing system of  claim 1 , wherein the transition function comprises a fully-connected neural network layer. 
     
     
       4. The computing system of  claim 1 , wherein when performed, the analysis of the compiled program code includes one of a software classification analysis, a thread coarsening analysis, or a heterogeneous scheduling analysis, and wherein when performed, the enhancement of the program code includes program modifications to improve performance via at least one of loop vectoring or optimization pass ordering. 
     
     
       5. The computing system of  claim 1 , further comprising a learning module to train the graph attention neural network, the learning module comprising one of a training neural network trained for classification or regression, or a deep reinforcement learning agent trained using a reward mechanism that favors solving a desired task. 
     
     
       6. A semiconductor apparatus comprising:
 one or more substrates; and 
 logic coupled to the one or more substrates, wherein the logic is implemented at least partly in one or more of configurable logic or fixed-functionality hardware logic, the logic coupled to the one or more substrates to:
 generate during inference a dependence graph based on a plurality of intermediate representation (IR) code instructions associated with a compiled program code; 
 generate during inference a set of graph embedding vectors based on the plurality of IR code instructions, wherein the embedding vectors are generated independently of the dependence graph, and wherein each graph embedding vector is to model as a graph an operation, an associated argument and an associated argument type for an instruction of the plurality of IR code instructions; 
 apply dynamic per-position recurrence-halting using adaptive computation time to determine the number of recurring steps for each position in the recurrent block; and 
 perform during inference, via a graph attention neural network, one of an analysis of the compiled program code or an enhancement of the compiled program code based on the dependence graph and the set of graph embedding vectors, wherein the graph attention neural network comprises a recurrent block and a task-specific neural network layer, the recurrent block comprising a graph attention layer and a transition function, and wherein the number of recurring steps for each position in the recurrent block is determined automatically. 
 
 
     
     
       7. The semiconductor apparatus of  claim 6 , wherein the dependence graph is to map dependencies between instructions in the plurality of IR code instructions. 
     
     
       8. The semiconductor apparatus of  claim 6 , wherein the transition function comprises a fully-connected neural network layer. 
     
     
       9. The semiconductor apparatus of  claim 6 , wherein when performed, the analysis of the compiled program code includes one of a software classification analysis, a thread coarsening analysis, or a heterogeneous scheduling analysis, and wherein when performed, the enhancement of the program code includes program modifications to improve performance via at least one of loop vectoring or optimization pass ordering. 
     
     
       10. The semiconductor apparatus of  claim 6 , wherein a learning module is to train the graph attention neural network, the learning module comprising one of a training neural network trained for classification or regression, or a deep reinforcement learning agent trained using a reward mechanism that favors solving a desired task. 
     
     
       11. The semiconductor apparatus of  claim 6 , wherein the logic coupled to the one or more substrates includes transistor channel regions that are positioned within the one or more substrates. 
     
     
       12. At least one non-transitory computer readable storage medium comprising a set of instructions which, when executed by a computing system, cause the computing system to:
 generate during inference a dependence graph based on a plurality of intermediate representation (IR) code instructions associated with a compiled program code; 
 generate during inference a set of graph embedding vectors based on the plurality of IR code instructions, wherein the embedding vectors are generated independently of the dependence graph, and wherein each graph embedding vector is to model as a graph an operation, an associated argument and an associated argument type for an instruction of the plurality of IR code instructions; 
 apply dynamic per-position recurrence-halting using adaptive computation time to determine the number of recurring steps for each position in the recurrent block; and 
 perform during inference, via a graph attention neural network, one of an analysis of the compiled program code or an enhancement of the compiled program code based on the dependence graph and the set of graph embedding vectors, wherein the graph attention neural network comprises a recurrent block and a task-specific neural network layer, the recurrent block comprising a graph attention layer and a transition function, and wherein the number of recurring steps for each position in the recurrent block is determined automatically. 
 
     
     
       13. The at least one non-transitory computer readable storage medium of  claim 12 , wherein the dependence graph is to map dependencies between instructions in the plurality of IR code instructions. 
     
     
       14. The at least one non-transitory computer readable storage medium of  claim 12 , wherein the transition function comprises a fully-connected neural network layer. 
     
     
       15. The at least one non-transitory computer readable storage medium of  claim 12 , wherein when performed, the analysis of the compiled program code includes one of a software classification analysis, a thread coarsening analysis, or a heterogeneous scheduling analysis, and wherein when performed, the enhancement of the program code includes program modifications to improve performance via at least one of loop vectoring or optimization pass ordering. 
     
     
       16. The at least one non-transitory computer readable storage medium of  claim 12 , wherein a learning module is to train the graph attention neural network, the learning module comprising one of a training neural network trained for classification or regression, or a deep reinforcement learning agent trained using a reward mechanism that favors solving a desired task. 
     
     
       17. A method of analyzing and enhancing computer software programs, comprising:
 generating during inference a dependence graph based on a plurality of intermediate representation (IR) code instructions associated with a compiled program code; 
 generating during inference a set of graph embedding vectors based on the plurality of IR code instructions, wherein the embedding vectors are generated independently of the dependence graph, and wherein each graph embedding vector models as a graph an operation, an associated argument and an associated argument type for an instruction of the plurality of IR code instructions; 
 applying dynamic per-position recurrence-halting using adaptive computation time to determine the number of recurring steps for each position in the recurrent block; and 
 performing during inference, via a graph attention neural network, one of an analysis of the compiled program code or an enhancement of the compiled program code based on the dependence graph and the set of graph embedding vectors, wherein the graph attention neural network comprises a recurrent block and a task-specific neural network layer, the recurrent block comprising a graph attention layer and a transition function, and wherein the number of recurring steps for each position in the recurrent block is determined automatically. 
 
     
     
       18. The method of  claim 17 , wherein the dependence graph maps dependencies between instructions in the plurality of IR code instructions. 
     
     
       19. The method of  claim 17 , wherein the transition function comprises a fully-connected neural network layer. 
     
     
       20. The method of  claim 17 , wherein when performed, the analysis of the compiled program code includes one of a software classification analysis, a thread coarsening analysis, or a heterogeneous scheduling analysis, and wherein when performed, the enhancement of the program code includes program modifications to improve performance via at least one of loop vectoring or optimization pass ordering. 
     
     
       21. The method of  claim 17 , wherein a learning module is used to train the graph attention neural network, the learning module comprising one of a training neural network trained for classification or regression, or a deep reinforcement learning agent trained using a reward mechanism that favors solving a desired task.