Methods and apparatus for machine learning based malware detection

Abstract

Apparatus and methods describe herein, for example, a process that can include receiving a potentially malicious file, and dividing the potentially malicious file into a set of byte windows. The process can include calculating at least one attribute associated with each byte window from the set of byte windows for the potentially malicious file. In such an instance, the at least one attribute is not dependent on an order of bytes in the potentially malicious file. The process can further include identifying a probability that the potentially malicious file is malicious, based at least in part on the at least one attribute and a trained threat model.

What is claimed is: 
     
       1. A method, comprising:
 receiving a target file; 
 calculating an attribute value associated with the target file, the attribute value being based on at least one of:
 a set of informational entropy values obtained from the target file; 
 a histogram of byte values within the target file; 
 a set of byte standard deviation ranges associated with the target file; or 
 a hash value for each string from a set of strings within the target file; 
 
 identifying a set of Portable Executable (PE) header values associated with the target file; 
 calculating a probability that the target file is malicious based on the attribute value and the set of PE header values; and 
 communicating a threat score based on the probability, the threat score associated with a potential threat to a user, device, or network. 
 
     
     
       2. The method of  claim 1 , wherein the attribute value is a first attribute value, the method further comprising:
 calculating a second attribute value based on the set of PE header values, the calculating the probability that the target file is malicious being based on the first attribute value and the second attribute value. 
 
     
     
       3. The method of  claim 1 , wherein the calculating the probability includes calculating the probability using a trained threat model including at least one of a random forest classifier or a deep neural network. 
     
     
       4. The method of  claim 1 , further comprising:
 based on the threat score, at least one of quarantining the target file, deleting the target file, taking a remedial action, sending a notification to a user regarding the target file, cleaning the target file, or executing the target file within a virtual container. 
 
     
     
       5. The method of  claim 1 , further comprising:
 calculating at least one of a PE import vector or a PE metadata vector based on the set of PE header values, the calculating the probability that the target file is malicious is based on the attribute value and the at least one of the PE import vector or the PE metadata vector. 
 
     
     
       6. The method of  claim 1 , wherein the set of PE header values includes at least one of a name, an age, an author, a source, a file type, or a size. 
     
     
       7. The method of  claim 1 , wherein the threat score is calculated based on the probability and at least one of a security indication associated with a network, a type of business hosting the network, a rate of false positives associated with a trained threat model, or a rate of false negatives associated with the trained threat model. 
     
     
       8. The method of  claim 1 , wherein the calculating the probability includes calculating the probability that the target file is malicious based on user input relating to a nature of the target file. 
     
     
       9. The method of  claim 1 , wherein the set of informational entropy values is based on a frequency of byte values found within the target file. 
     
     
       10. An apparatus, comprising:
 a memory; and 
 a hardware processor operatively coupled to the memory, the hardware processor configured to:
 receive a target file; 
 calculate a set of informational entropy values obtained from the target file; 
 identify a set of Portable Executable (PE) header values associated with the target file; 
 calculate a probability that the target file is malicious based on the set of informational entropy values and the set of PE header values; and 
 perform a remedial action based on the probability. 
 
 
     
     
       11. The apparatus of  claim 10 , wherein the hardware processor is configured to perform the remedial action by at least one of quarantining the target file, deleting the target file, sending a notification to a user regarding the target file, cleaning the target file, or executing the target file within a virtual container. 
     
     
       12. The apparatus of  claim 10 , wherein the set of PE header values includes at least one of a name, an age, an author, a source, a file type, or a size. 
     
     
       13. The apparatus of  claim 10 , wherein the hardware processor is configured to calculate the probability using a trained threat model including at least one of a random forest classifier or a deep neural network. 
     
     
       14. The apparatus of  claim 10 , wherein the hardware processor is configured to calculate the probability that the target file is malicious based on at least one of a security indication associated with a network, a type of business hosting the network, a rate of false positives associated with a trained threat model, or a rate of false negatives associated with the trained threat model. 
     
     
       15. The apparatus of  claim 10 , wherein the hardware processor is configured to calculate the probability that the target file is malicious based on user input relating to a nature of the target file. 
     
     
       16. A non-transitory processor-readable medium storing code representing instructions to be executed by a processor, the instructions comprising code to cause the processor to:
 receive a target file; 
 identify a set of Portable Executable (PE) header values associated with the target file; 
 calculate an attribute value based on a hash value for each string from a set of strings within the target file; 
 provide the set of PE header values and the attribute value as input to a trained threat model to produce an output comprising (1) a threat score and (2) a classification of at least one of a class of malware, a source of malware, or a severity of malware; and 
 perform, based on the output, at least one of quarantining the target file, deleting the target file, taking a remedial action, sending a notification to a user regarding the target file, cleaning the target file, or executing the target file within a virtual container. 
 
     
     
       17. The non-transitory processor-readable medium of  claim 16 , further comprising code to cause the processor to:
 select the trained threat model based on a type of the target file. 
 
     
     
       18. The non-transitory processor-readable medium of  claim 16 , wherein the code to cause the processor to perform includes code to cause the processor to perform, based on the output and user input relating to a nature of the target file, the at least one of quarantining the target file, deleting the target file, taking the remedial action, sending the notification to the user regarding the target file, cleaning the target file, or executing the target file within the virtual container. 
     
     
       19. The non-transitory processor-readable medium of  claim 16 , wherein the set of PE header values includes at least one of a name, an age, an author, a source, a file type, or a size. 
     
     
       20. The non-transitory processor-readable medium of  claim 16 , wherein the trained threat model includes at least one of a random forest classifier or a deep neural network. 
     
     
       21. The non-transitory processor-readable medium of  claim 16 , wherein the code to cause the processor to perform includes code to cause the processor to perform, based on the output and at least one of a security indication associated with a network, a type of business hosting the network, a rate of false positives associated with the trained threat model, or a rate of false negatives associated with the trained threat model, the at least one of quarantining the target file, deleting the target file, taking the remedial action, sending the notification to the user regarding the target file, cleaning the target file, or executing the target file within the virtual container.