Systems and methods for anomaly detection in software-defined networks from observed host metrics

Abstract

Systems and methods for anomaly detection in software-defined networks from observed host metrics are disclosed. A method may include: (1) training a random forest model comprising a plurality of trees with historical metrics from a software defined network, the software defined network comprising a plurality of hosts; (2) receiving metrics for a plurality of features from the hosts in the software defined network; (3) providing the metrics to the trained random forest model; (4) receiving, from the trained random forest model, a prediction of an anomalous hosts for one of the hosts; (5) identifying a subset of the plurality of trees that contributed to the prediction; (6) generating feature scores for the feature from the subset of trees; (7) generating an anomaly score for the feature based on the feature scores and an explanation; and (8) executing an automated action in response to the anomaly score.

What is claimed is: 
     
         1 . A method, comprising:
 training, by a computer program, a random forest model comprising a plurality of trees with historical metrics from a software defined network, the software defined network comprising a plurality of hosts;   receiving, by the computer program, metrics for a plurality of features from the hosts in the software defined network, wherein the historical metrics and the metrics comprise central processing unit (CPU) metrics, disk usage metrics, memory metrics, networking metrics, and system performance metrics;   providing, by the computer program, the metrics to the trained random forest model;   receiving, by the computer program and from the trained random forest model, a prediction of an anomalous host among one of the plurality of hosts;   identifying, by the computer program and for the anomalous host, a subset of the plurality of trees that contributed to the prediction;   generating, by the computer program, feature scores for the plurality of features from the subset of the plurality of trees, wherein the feature scores for the plurality of features from the subset of the plurality of trees are based on a location of each feature the subset of the plurality of trees and one feature score of the feature scores is higher for a feature that is close to a root of a tree of the subset of the plurality of trees;   generating, by the computer program, anomaly scores for the plurality of features based on the feature scores;   ranking, by the computer program, the plurality of features based on the anomaly score;   generating, by the computer program, an explanation based on a highest anomaly score of the anomaly scores;   persisting, by the computer program to a memory in communication with the computer program, feature importance scores for a plurality of anomalous hosts by averaging feature scores of the subset of the plurality of trees that contributed to the prediction, wherein the plurality of anomalous hosts includes the anomalous host;   triggering, by the computer program, a re-training pipeline for an artificial intelligence model based on the persisted feature importance scores;   re-training, by the computer program, the artificial intelligence model based on the re-training pipeline; and   executing, by the computer program, an automated action in response to the anomaly score, the automated action comprising a vMotion that moves virtual machines from the anomalous host to one or more hosts of the plurality of hosts that is not the anomalous host.   
     
     
         2 . The method of  claim 1 , wherein the plurality of hosts comprise hardware in the software defined network. 
     
     
         3 . The method of  claim 1 , wherein the automated action comprises moving virtual machines from anomalous hosts to healthy hosts. 
     
     
         4 . A system, comprising:
 a software defined network comprising a plurality of hosts, the software defined network comprising a physical switch or a router and a storage and a processor; and   an electronic device executing a computer program that:   to train a random forest model comprising a plurality of trees with historical metrics from the software defined network,   to receive metrics for a plurality of features from the hosts in the software defined network, wherein the historical metrics and the metrics comprise central processing unit (CPU) metrics, disk usage metrics, memory metrics, networking metrics, and system performance metrics,   to provide the metrics to the trained random forest model, to receive, from the trained random forest model, a prediction of host among one of the plurality of hosts,   to identify a subset of the plurality of trees that contributed to the prediction, to generate feature scores for the plurality of features from the subset of the plurality of trees, wherein the feature scores for the plurality of features from the subset of the plurality of trees are based on a location of each feature the subset of the plurality of trees and one feature score of the feature scores is higher for a feature that is close to a root of a tree of the subset of the plurality of trees,   to generate an anomaly score for the feature based on the feature scores,   to rank the plurality of features based on the anomaly score;   to generate an explanation based on a highest anomaly score of the anomaly scores;   to persist, in a memory in communication with the processor, feature importance scores for a plurality of anomalous hosts by averaging feature scores of the subset of the plurality of trees that contributed to the prediction, wherein the plurality of anomalous hosts includes the anomalous host;   to trigger a re-training pipeline for an artificial intelligence model based on the persisted feature importance scores;   to re-train the artificial intelligence model based on the re-training pipeline; and   to execute an automated action in response to the anomaly score, the automated action comprising a vMotion that moves virtual machines from the anomalous host to one or more hosts of the plurality of hosts that is not the anomalous host.   
     
     
         5 . The system of  claim 4 , wherein the plurality of hosts comprise hardware in the software defined network. 
     
     
         6 . The system of  claim 4 , wherein the automated action comprises moving virtual machines from anomalous hosts to healthy hosts. 
     
     
         7 . A non-transitory computer readable storage medium, including instructions stored thereon, which when read and executed by one or more computer processors, cause the one or more computer processors to perform steps comprising:
 training a random forest model comprising a plurality of trees with historical metrics from a software defined network, the software defined network comprising a plurality of hosts, the software defined network comprising a physical switch or a router and a storage and a processor, wherein the historical metrics and the metrics comprise central processing unit (CPU) metrics, disk usage metrics, memory metrics, networking metrics, and system performance metrics;   receiving metrics for a plurality of features from hosts in the software defined network;   providing the metrics to the trained random forest model;   receiving, from the trained random forest model, a prediction of an anomalous host among one of the plurality of hosts;   identifying, for the anomalous host, a subset of the plurality of trees that contributed to the prediction;   generating feature scores for the plurality of features from the subset of the plurality of trees, wherein the feature scores for the plurality of features from the subset of the plurality of trees are based on a location of each feature the subset of the plurality of trees and one feature score of the feature scores is higher for a feature that is close to a root of a tree of the subset of the plurality of trees;   generating an anomaly score for the feature based on the feature scores,   ranking the plurality of features based on the anomaly score;   
       to generate an explanation based on a highest anomaly score of the anomaly scores;
 persisting, in the on-transitory computer readable storage medium in communication with the one or more computer processors, feature importance scores for a plurality of anomalous hosts by averaging feature scores of the subset of the plurality of trees that contributed to the prediction, wherein the plurality of anomalous hosts includes the anomalous host; 
 triggering, by the computer program, a re-training pipeline for an artificial intelligence model based on the persisted feature importance scores; 
 re-training, by the computer program, the artificial intelligence model based on the re-training pipeline; and 
 executing an automated action in response to the anomaly score, the automated action comprising a vMotion that moves virtual machines from the anomalous host to one or more hosts of the plurality of hosts that is not the anomalous host. 
 
     
     
         8 . The non-transitory computer readable storage medium of  claim 7 , wherein the plurality of hosts comprise hardware in the software defined network. 
     
     
         9 . The non-transitory computer readable storage medium of  claim 7 , wherein the automated action comprises moving virtual machines from anomalous hosts to healthy hosts.