Computer system and method for detecting anomalies in multivariate data

Abstract

A data analytics platform may be configured to construct an inferential model for a multivariate observation vector using inferential modeling in combination with component analysis, which may enable the data analytics platform to evaluate only a subset of the variables in the observation vector and then output a predicted version of the multivariate observation vector that includes predicted values for the full set of variables that was originally included in the observation vector. In turn, the data analytics platform may use the predicted version of the multivariate observation vector output by the inferential model to determine whether an anomaly has occurred.

The invention claimed is: 
     
       1. A computing system comprising:
 a network interface; 
 at least one processor; 
 a non-transitory computer-readable medium; and 
 program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to:
 obtain a set of training data vectors for a given asset-related data source, wherein the given asset-related data source outputs observation vectors related to asset operation, wherein the observation vectors output by the given asset-related data source comprise a given set of variables that defines an observed full coordinate space, and wherein each training data vector in the set of training data vectors is reflective of normal asset operation and includes a valid value for each variable in the observed full space; 
 represent the set of training data vectors in an observed inferential space that is defined by a given subset of the given set of variables and then transform the training data vectors from the observed inferential space to a transformed inferential space; 
 transform a given observation vector received from the given asset-related data source from the observed inferential space to the transformed inferential space; 
 perform a comparison in the transformed inferential space between the given observation vector and the set of training data vectors; 
 based on the comparison, identify a subset of training data vectors that are closest to the given observation vector in the transformed inferential space; 
 produce a predicted version of the given observation vector in the observed full space that has a valid value for each variable in the observed full space by:
 determining respective representations of the identified subset of training data vectors in a transformed full space corresponding to the observed full space, each identified training data vector being in a different coordinate space than the observed full space, and each identified training data vector determined in the transformed full space based on associative mappings between a given training data vector originally in the transformed inferential space as mapped to the transformed full space corresponding to the observed full space, and 
 using a machine learning process to perform a nonparametric regression analysis on the respective representations of the identified subset of training data vectors in the transformed full space that produces a predicted version of the given observation vector in the transformed full space, wherein the predicted version of the given observation vector includes a predicted value for each variable in the observed full space; 
 
 use the predicted version of the given observation vector to determine whether an anomaly has occurred at the given asset; and 
 implement at least one of: (1) determining whether the anomaly indicates equipment failure of the given asset and controlling the given asset to prevent the equipment failure, or (2) controlling the given asset based on the anomaly that has occurred at the given asset. 
 
 
     
     
       2. The computing system of  claim 1 , wherein the program instructions that are executable to cause the computing system to transform the given observation vector from the observed inferential space to the transformed inferential space comprise program instructions that are executable to cause the computing system to:
 transform the given observation vector from the observed inferential space to the transformed inferential space in response to determining that the given observation vector has at least one variable with an invalid value. 
 
     
     
       3. The computing system of  claim 2 , wherein the given subset of the given set of variables that defines the observed inferential space comprises a subset of the given set of variables that excludes the at least one variable with the invalid value. 
     
     
       4. The computing system of  claim 3 , wherein the program instructions that are executable to cause the computing system to represent the set of training data vectors in the observed inferential space and then transform the training data vectors from the observed inferential space to a transformed inferential space comprise program instructions that are executable to cause the computing system to:
 represent the set of training data vectors in the observed inferential space and then transform the training data vectors from the observed inferential space in response to determining that the given observation vector has at least one variable with an invalid value. 
 
     
     
       5. The computing system of  claim 1 , wherein the program instructions that are executable to cause the computing system to transform the given observation vector from the observed inferential space to the transformed inferential space comprise program instructions that are executable to cause the computing system to:
 transform the given observation vector from the observed inferential space to the transformed inferential space in accordance with a predefined policy to transform every observation vector received from the given asset-related data source from the observed inferential space to the transformed inferential space. 
 
     
     
       6. The computing system of  claim 5 , further comprising program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to preselect the given subset of the given set of variables that defines the observed inferential space before receiving the given observation vector. 
     
     
       7. The computing system of  claim 6 , wherein the program instructions that are executable to cause the computing system to represent the set of training data vectors in the observed inferential space and then transform the training data vectors from the observed inferential space to a transformed inferential space comprise program instructions that are executable to cause the computing system to represent the set of training data vectors in the observed inferential space and then transform the training data vectors from the observed inferential space before receiving the given observation vector, and wherein the computing system further comprises program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to:
 while transforming the training data vectors from the observed inferential space to the transformed inferential space, store a representation of each training data vector in the transformed inferential space; and 
 after receiving the given observation vector, access the stored representation of each training data vector in the transformed inferential space. 
 
     
     
       8. The computing system of  claim 1 , wherein:
 the program instructions that are executable to cause the computing system to transform the set of training data vectors from the observed inferential space to a transformed inferential space comprise program instructions that are executable to cause the computing system to apply Principal Component Analysis (PCA) to the set of training data vectors in the observed inferential space; and 
 the program instructions that are executable to cause the computing system to transform the given observation vector from the observed inferential space to the transformed inferential space comprise program instructions that are executable to cause the computing system to apply PCA to the given observation vector in the observed inferential space. 
 
     
     
       9. The computing system of  claim 1 , wherein the transformed inferential space comprises a number of dimensions that is equal to or less than a number of dimensions in the observed inferential space. 
     
     
       10. The computing system of  claim 1 , wherein the program instructions that are executable to cause the computing system to identify the subset of training data vectors that are closest to the given observation vector in the transformed inferential space comprise program instructions that are executable to cause the computing system to:
 in the transformed inferential space, determine a distance between the given observation vector and each training data vector in the set of training data vectors; and 
 identify the subset of training data vectors that are closest to the given observation vector based on the determined distances. 
 
     
     
       11. The computing system of  claim 1 , wherein the program instructions are executable to further cause the computing system to inversely transform the predicted version of the given observation vector from the transformed full space to the observed full space. 
     
     
       12. A non-transitory computer-readable medium having instructions stored thereon that are executable to cause a computing system to:
 obtain a set of training data vectors for a given asset-related data source, wherein the given asset-related data source outputs observation vectors related to asset operation, wherein the observation vectors output by the given asset-related data source comprise a given set of variables that defines an observed full coordinate space, and wherein each training data vector in the set of training data vectors is reflective of normal asset operation and includes a valid value for each variable in the observed full space; 
 represent the set of training data vectors in an observed inferential space that is defined by a given subset of the given set of variables and then transform the training data vectors from the observed inferential space to a transformed inferential space; 
 transform a given observation vector received from the given asset-related data source from the observed inferential space to the transformed inferential space; 
 perform a comparison in the transformed inferential space between the given observation vector and the set of training data vectors; 
 based on the comparison, identify a subset of training data vectors that are closest to the given observation vector in the transformed inferential space; 
 produce a predicted version of the given observation vector in the observed full space that has a valid value for each variable in the observed full space by:
 determining respective representations of the identified subset of training data vectors in a transformed full space corresponding to the observed full space, each identified training data vector being in a different coordinate space than the observed full space, and each identified training data vector determined in the transformed full space based on associative mappings between a given training data vector originally in the transformed inferential space as mapped to the transformed full space corresponding to the observed full space, and 
 using a machine learning process to perform a nonparametric regression analysis on the respective representations of the identified subset of training data vectors in the transformed full space that produces a predicted version of the given observation vector in the transformed full space, wherein the predicted version of the given observation vector includes a predicted value for each variable in the observed full space; 
 
 use the predicted version of the given observation vector to determine whether an anomaly has occurred at the given asset; and 
 implement at least one of: (1) determining whether the anomaly indicates equipment failure of the given asset and controlling the given asset to prevent the equipment failure, or (2) controlling the given asset based on the anomaly that has occurred at the given asset. 
 
     
     
       13. The non-transitory computer-readable medium of  claim 12 , wherein the program instructions that are executable to cause the computing system to transform the given observation vector from the observed inferential space to the transformed inferential space comprise program instructions that are executable to cause the computing system to:
 transform the given observation vector from the observed inferential space to the transformed inferential space in response to determining that the given observation vector has at least one variable with an invalid value. 
 
     
     
       14. The non-transitory computer-readable medium of  claim 13 , wherein the given subset of the given set of variables that defines the observed inferential space comprises a subset of the given set of variables that excludes the at least one variable with the invalid value. 
     
     
       15. The non-transitory computer-readable medium of  claim 13 , wherein the program instructions that are executable to cause the computing system to transform the given observation vector from the observed inferential space to the transformed inferential space comprise program instructions that are executable to cause the computing system to:
 transform the given observation vector from the observed inferential space to the transformed inferential space in accordance with a predefined policy to transform every observation vector received from the given asset-related data source from the observed inferential space to the transformed inferential space. 
 
     
     
       16. A computer-implemented method comprising:
 obtaining a set of training data vectors for a given asset-related data source, wherein the given asset-related data source outputs observation vectors related to asset operation, wherein the observation vectors output by the given asset-related data source comprise a given set of variables that defines an observed full coordinate space, and wherein each training data vector in the set of training data vectors is reflective of normal asset operation and includes a valid value for each variable in the observed full space; 
 representing the set of training data vectors in an observed inferential space that is defined by a given subset of the given set of variables and then transforming the training data vectors from the observed inferential space to a transformed inferential space; 
 transforming a given observation vector received from the given asset-related data source from the observed inferential space to the transformed inferential space; 
 performing a comparison in the transformed inferential space between the given observation vector and the set of training data vectors; 
 based on the comparison, identifying a subset of training data vectors that are closest to the given observation vector in the transformed inferential space; 
 produce a predicted version of the given observation vector in the observed full space that has a valid value for each variable in the observed full space by:
 obtaining respective representations of the identified subset of training data vectors in a transformed full space corresponding to the observed full space, each identified training data vector being in a different coordinate space than the observed full space, and each identified training data vector determined in the transformed full space based on associative mappings between a given training data vector originally in the transformed inferential space as mapped to the transformed full space corresponding to the observed full space, and 
 using a machine learning process to perform a nonparametric regression analysis on the respective representations of the identified subset of training data vectors in the transformed full space that produces a predicted version of the given observation vector in the transformed full space, wherein the predicted version of the given observation vector includes a predicted value for each variable in the observed full space; 
 
 using the predicted version of the given observation vector to determine whether an anomaly has occurred at the given asset; and 
 implementing at least one of: (1) determining whether the anomaly indicates equipment failure of the given asset and controlling the given asset to prevent the equipment failure, or (2) controlling the given asset based on the anomaly that has occurred at the given asset. 
 
     
     
       17. The computer-implemented method of  claim 16 , wherein transforming the given observation vector from the observed inferential space to the transformed inferential space comprises:
 transforming the given observation vector from the observed inferential space to the transformed inferential space in response to determining that the given observation vector has at least one variable with an invalid value. 
 
     
     
       18. The computer-implemented method of  claim 17 , wherein
 the given subset of the given set of variables that defines the observed inferential space comprises a subset of the given set of variables that excludes the at least one variable with the invalid value. 
 
     
     
       19. The computer-implemented method of  claim 16 , wherein transforming the given observation vector from the observed inferential space to the transformed inferential space comprises:
 transforming the given observation vector from the observed inferential space to the transformed inferential space in accordance with a predefined policy to transform every observation vector received from the given asset-related data source from the observed inferential space to the transformed inferential space.