Method and system for self supervised training of deep learning based time series models

Abstract

This disclosure relates to method and system for training of deep learning based time-series models based on self-supervised learning. The problem of missing data is taken care of by introducing missing-ness masks. The deep learning model for univariate and multivariate time series data is trained with the distorted input data using the self-supervised learning to reconstruct the masked input data. Herein, the one or more distortion techniques include quantization, insertion, deletion, and combination of the one or more such distortion techniques with random subsequence shuffling. Different distortion techniques in the form of reconstruction of masked input data are provided to solve. The deep learning model performs these different distortion techniques, which force the deep learning model to learn better features. It is to be noted that the system uses a lot of unlabeled data available cheaply as compared to the label or annotated data which is very hard to get.

What is claimed is: 
     
       1. A processor-implemented method for self-supervised training of a deep learning based model with un-labelled time-series data comprising:
 receiving, via one or more hardware processors, a plurality of input data from one or more predefined data sources, wherein the plurality of input data is the un-labelled time-series data; 
 preprocessing, via the one or more hardware processors, the received plurality of input data for verification of availability of the received plurality of input data to:
 remove noise and outliers; 
 achieve uniform sampling frequency of the received plurality of input data; and 
 synchronization by incorporating lags and integration of a plurality of variables from one or more databases; 
 
 masking, via the one or more hardware processors, the preprocessed plurality of input data for one or more missing values of the plurality of input data and applying missingness mask corresponding to the one or more missing values, wherein the method handles missing data without need of imputation as the method uses the missingness mask; 
 distorting, via the one or more hardware processors, the masked plurality of input data of a flame detector voltage of an industrial gas turbine combustor using one or more distortion techniques, wherein the one or more distortion techniques include quantization determined using minimum and maximum of each sensor data and the quantization is different for different sensors, insertion, deletion, and combination of the one or more such distortion techniques with random subsequence shuffling, wherein calculating loss considering the missingness mask appropriately and update weights using back propagation, wherein, in case of the deletion, add padding towards an end or in starting of the input data to maintain a length of the input data same before and after the deletion, and the length of the padding in case of the deletion is same as number of deleted instances, wherein, in case of the insertion, deleting the input data from last or starting to maintain length of the input data after the insertion same as before the insertion, and the length of a cropped portion of the input data is same as number of newly inserted instances; and 
 training, via the one or more hardware processors, the deep learning based model with the distorted plurality of input data using self-supervised learning to reconstruct the masked plurality of input data, wherein, after training the deep learning based model using self-supervised learning, the deep learning based model is finetuned either by replacing final layers of the deep learning based model or training the weights of the final layer of the deep learning based model, wherein the trained deep learning based model is used to identify abnormal trends of the flame detector voltage of the industrial gas turbine combustor. 
 
     
     
       2. The processor-implemented method of  claim 1 , wherein the time-series data includes univariate and multi-variate datasets, and wherein in case of the multivariate datasets, performing the insertion and the deletion operations on each of the sensor data independently. 
     
     
       3. The processor-implemented method of  claim 1 , wherein the quantization technique includes discretization of the time series signal. 
     
     
       4. The processor-implemented method of  claim 1 , wherein the insertion technique comprises of inserting one or more values between selected pairs of instances in the time series data. 
     
     
       5. The processor-implemented method of  claim 1 , wherein the deletion technique comprises of deletion of one or more randomly selected instances in the time series data. 
     
     
       6. A system for self-supervised training of a deep learning based model with un-labelled time-series data comprising:
 an input/output interface for receiving a plurality of input data from one or more predefined data sources, wherein the plurality of input data is un-labelled time-series data, wherein the time-series data includes univariate and multi-variate datasets; 
 one or more hardware processors; 
 a memory in communication with the one or more hardware processors, wherein the one or more hardware processors are configured to execute programmed instructions stored in the memory, to:
 preprocess the received plurality of input data for verification of availability of the received plurality of input data to remove noise and one or more outliers from the received plurality of input data, achieve uniform sampling frequency of the time-series data, and synchronizing by incorporating lags and integration of a plurality of variables from one or more databases; 
 mask the preprocessed plurality of input data for one or more missing values of the input data and applying missingness mask corresponding to the one or more missing values, wherein the system is configured to handle missing data without need of imputation as the system uses the missingness mask; 
 
 distort the masked plurality of input data using one or more distortion techniques of a flame detector voltage of an industrial gas turbine combustor, wherein the one or more distortion techniques include quantization determined using minimum and maximum of each sensor data and the quantization is different for different sensors, insertion, deletion, and combination of the one or more distortion techniques with random subsequence shuffling, wherein the system is configured to calculate loss considering the missingness mask appropriately and update weights using back propagation, wherein, in case of the deletion, the system is configured to add padding towards an end or in starting of the input data to maintain a length of the input data same before and after the deletion, and the length of the padding in case of the deletion is same as number of deleted instances, wherein, in case of the insertion, the system deletes the input data from last or starting to maintain length of the input data after the insertion same as before the insertion, and the length of a cropped portion of the input data is same as number of newly inserted instance; and 
 train the deep learning based model with the distorted plurality of input data using the self-supervised learning to reconstruct the masked plurality of input data, wherein, after training the deep learning based model using self-supervised learning, the deep learning based model is finetuned either by replacing final layers of the deep learning based model or training the weights of the final layer of the deep learning based model, wherein the trained deep learning based model is used to identify abnormal trends of the flame detector voltage of the industrial gas turbine combustor. 
 
     
     
       7. A non-transitory computer readable medium storing one or more instructions which when executed by one or more processors on a system, cause the one or more processors to perform method for self-supervised training of a deep learning based model with un-labelled time-series data comprising:
 receiving, via one or more hardware processors, a plurality of input data from one or more predefined data sources, wherein the plurality of input data is un-labelled time-series data; 
 preprocessing, via the one or more hardware processors, for verification of availability of the received plurality of input data the received plurality of input data to:
 remove noise and outliers from the received plurality of input data; 
 achieve uniform sampling frequency of the received plurality of input data; and 
 synchronization by incorporating lags and integration of a plurality of variables from one or more databases; 
 
 masking, via the one or more hardware processors, the preprocessed plurality of input data for one or more missing values of the input data and applying missingness mask corresponding to the one or more missing values, wherein the method handles missing data without need of imputation as the method uses the missingness mask; 
 distorting, via the one or more hardware processors, the masked plurality of input data of a flame detector voltage of an industrial gas turbine combustor using one or more distortion techniques, wherein the one or more distortion techniques include quantization determined using minimum and maximum of each sensor data and the quantization is different for different sensors, insertion, deletion, and combination of the one or more distortion techniques with random subsequence shuffling, wherein calculating loss considering the missingness mask appropriately and update weights using back propagation, wherein, in case of the deletion, add padding towards an end or in starting of the input data to maintain a length of the input data same before and after the deletion, and the length of the padding in case of the deletion is same as number of deleted instances, wherein, in case of the insertion, deleting the input data from last or starting to maintain length of the input data after the insertion same as before the insertion, and the length of a cropped portion of the input data is same as number of newly inserted instances; and 
 training, via the one or more hardware processors, the deep learning based model with the distorted plurality of input data using the self-supervised learning to reconstruct the masked plurality of input of data, wherein, after training the deep learning based model using self-supervised learning, the deep learning based model is finetuned either by replacing final layers of the deep learning based model or training the weights of the final layer of the deep learning based model, wherein the trained deep learning based model is used to identify abnormal trends of the flame detector voltage of the industrial gas turbine combustor.