Training classifiers for program analysis

Abstract

Methods for training a static security analysis classifier include running an initial security analysis on a training codebase to generate a set of vulnerabilities associated with the training codebase; analyzing the program with a feature set that limits a number of detected vulnerabilities to generate a limited set of vulnerabilities associated with the feature set; comparing the limited set of vulnerabilities to a known vulnerability distribution to generate an accuracy score; and iterating the steps of analyzing and comparing using different feature sets to find a feature set having a highest accuracy score.

What is claimed is: 
     
       1. A method for training a classifier, comprising:
 running an initial security analysis on a training codebase to generate a set of vulnerabilities associated with the training codebase; 
 analyzing a program with a feature set that limits a number of detected vulnerabilities to generate a limited set of vulnerabilities associated with the feature set; 
 comparing the limited set of vulnerabilities to a known vulnerability distribution to generate an accuracy score using a processor; and 
 iterating said steps of analyzing and comparing using different feature sets to find a feature set having a highest accuracy score. 
 
     
     
       2. The method of  claim 1 , wherein the feature set includes vulnerability path length. 
     
     
       3. The method of  claim 1 , wherein the feature set includes one or more blocked code locations. 
     
     
       4. The method of  claim 1 , wherein the known vulnerability distribution comprises a statistical representation of vulnerability prevalence. 
     
     
       5. The method of  claim 1 , wherein comparing comprises comparing numbers of vulnerabilities per line of code. 
     
     
       6. The method of  claim 1 , further comprising training a classifier based on the feature set having a highest accuracy score. 
     
     
       7. The method of  claim 1 , wherein the training codebase comprises a first set of applications and wherein the known vulnerability distribution is based on a manual analysis of a second set of applications. 
     
     
       8. The method of  claim 7 , wherein the training codebase further comprises a user's local codebase. 
     
     
       9. The method of  claim 1 , wherein each iteration uses a different feature set based on a Tabu search strategy. 
     
     
       10. A method for training a classifier, comprising:
 running an initial security analysis on a training codebase to generate a set of vulnerabilities associated with the training codebase; 
 analyzing a program with a feature set, comprising vulnerability path length, that limits a number of detected vulnerabilities to generate a limited set of vulnerabilities associated with the feature set; 
 comparing a number of vulnerabilities per line of code found with the limited set of vulnerabilities to a known statistical representation of vulnerability prevalence to generate an accuracy score using a processor; 
 iterating said steps of analyzing and comparing using different feature sets to find a feature set having a highest accuracy score; and 
 generating a classifier based on the feature set having a highest accuracy score. 
 
     
     
       11. The method of  claim 10 , wherein the feature set includes one or more blocked code locations. 
     
     
       12. The method of  claim 10 , wherein the training codebase comprises a first set of applications and wherein the known vulnerability distribution is based on a manual analysis of a second set of applications. 
     
     
       13. The method of  claim 12 , wherein the training codebase further comprises a user's local codebase. 
     
     
       14. A method for security analysis, comprising:
 training a classifier, comprising:
 running an initial security analysis on a training codebase to generate a set of vulnerabilities associated with the training codebase; 
 analyzing a program with a feature set that limits a number of detected vulnerabilities to generate a limited set of vulnerabilities associated with the feature set; 
 comparing the limited set of vulnerabilities to a known vulnerability distribution to generate an accuracy score using a processor; 
 iterating said steps of analyzing and comparing using different feature sets to find a feature set having a highest accuracy score; and 
 generating a classifier based on the feature set having the highest accuracy score; and 
 scanning code using the classifier to locate potential vulnerabilities. 
 
 
     
     
       15. The method of  claim 14 , wherein the feature set includes vulnerability path length. 
     
     
       16. The method of  claim 14 , wherein the feature set includes one or more blocked code locations. 
     
     
       17. The method of  claim 14 , wherein the known vulnerability distribution comprises a statistical representation of vulnerability prevalence. 
     
     
       18. The method of  claim 14 , wherein comparing comprises comparing numbers of vulnerabilities per line of code. 
     
     
       19. The method of  claim 14 , wherein the training codebase comprises a first set of applications and wherein the known vulnerability distribution is based on a manual analysis of a second set of applications. 
     
     
       20. The method of  claim 19 , wherein the training codebase further comprises a user's local codebase. 
     
     
       21. The method of  claim 14 , wherein each iteration uses a different feature set based on a Tabu search strategy. 
     
     
       22. A method for security analysis, comprising:
 training a classifier, comprising:
 running an initial security analysis on a training codebase comprising a first set of applications and a user's local codebase to generate a set of vulnerabilities associated with the training codebase; 
 analyzing a program with a feature set, comprising vulnerability path length, that limits a number of detected vulnerabilities to generate a limited set of vulnerabilities associated with the feature set; 
 comparing the a number of vulnerabilities per line of code found with the limited set of vulnerabilities to a known statistical representation of vulnerability prevalence based on a manual analysis of a second set of applications to generate an accuracy score using a processor; 
 iterating said steps of analyzing and comparing using different feature sets to find a feature set having a highest accuracy score; and 
 generating a classifier based on the feature set having the highest accuracy score; and 
 scanning code using the classifier to locate potential vulnerabilities.