Data management, reduction and sampling schemes for storage device failure

Abstract

In a method for training a machine learning model, the method includes: segmenting, by a processor, a dataset from a database into one or more datasets based on time period windows; assigning, by the processor, one or more weighted values to the one or more datasets according to the time period windows of the one or more datasets; generating, by the processor, a training dataset from the one or more datasets according to the one or more weighted values; and training, by the processor, the machine learning model using the training dataset.

What is claimed is: 
     
         1 . A method for training a machine learning model, the method comprising:
 segmenting, by a processor, a dataset from a database into one or more datasets based on time period windows;   assigning, by the processor, one or more weighted values to the one or more datasets according to the time period windows of the one or more datasets;   generating, by the processor, a training dataset from the one or more datasets according to the one or more weighted values; and   training, by the processor, the machine learning model using the training dataset.   
     
     
         2 . The method according to  claim 1 , wherein the machine learning model comprises a solid-state drive (SSD) failure prediction model. 
     
     
         3 . The method according to  claim 1 , wherein a most recent dataset from the one or more datasets is assigned a first weighted value and a least recent dataset from the one or more datasets is assigned a second weighted value, wherein the first weighted value is greater than the second weighted value. 
     
     
         4 . The method according to  claim 3 , wherein the one or more weighted values decrease by a set amount from the first weighted value to the second weighted value. 
     
     
         5 . The method according to  claim 1 , the method further comprising:
 identifying, by the processor, anomaly data in the dataset;   retrieving, by the processor, the anomaly data in the dataset; and   adding, by the processor, the anomaly data to the training dataset.   
     
     
         6 . The method according to  claim 5 , wherein the anomaly data comprises SSD failure data. 
     
     
         7 . The method according to  claim 5 , wherein the anomaly data is identified using a rule based method. 
     
     
         8 . The method according to  claim 5 , wherein the anomaly data is identified using a cluster based method. 
     
     
         9 . The method according to  claim 1 , the method further comprising
 generating, by the processor, anomaly data; and   adding, by the processor, the generated anomaly data to the training dataset.   
     
     
         10 . A data system comprising:
 a database;   a processor coupled to the database; and   a memory coupled to the processor, wherein the memory stores instructions that, when executed by the processor, cause the processor to:
 segment a dataset from the database into one or more datasets based on time period windows; 
 assign one or more weighted values to the one or more datasets according to the time period windows of the one or more datasets; 
 generate a training dataset from the one or more datasets according to the one or more weighted values; and 
 train a machine learning model using the training dataset. 
   
     
     
         11 . The data system according to  claim 10 , wherein the machine learning model comprises a solid-state drive (SSD) failure prediction model. 
     
     
         12 . The data system according to  claim 10 , wherein a most recent dataset from the one or more datasets is assigned a first weighted value and a least recent dataset from the one or more datasets is assigned a second weighted value, wherein the first weighted value is greater than the second weighted value. 
     
     
         13 . The data system according to  claim 12 , wherein the one or more weighted values decrease by a set amount from the first weighted value to the second weighted value. 
     
     
         14 . The data system according to  claim 10 , wherein the processor is further configured to:
 identify anomaly data in the dataset;   retrieve the anomaly data in the dataset; and   add the anomaly data to the training dataset.   
     
     
         15 . The data system according to  claim 14 , wherein the anomaly data comprises SSD failure data. 
     
     
         16 . The data system according to  claim 14 , wherein the anomaly data is identified using a rule based method. 
     
     
         17 . The data system according to  claim 14 , wherein the anomaly data is identified using a cluster based method. 
     
     
         18 . The data system according to  claim 10 , wherein the processor is further configured to:
 generate anomaly data; and   add the generated anomaly data to the training dataset.   
     
     
         19 . A method for training a machine learning model, the method comprising:
 identifying, by a processor, anomaly data in a dataset from a database;   generating, by the processor, anomaly data;   adding, by the processor, the generated anomaly data to the dataset;   identifying, by the processor, a training dataset from the dataset;   retrieving, by the processor, the training dataset from dataset; and   training, by the processor, the machine learning model using the training dataset.   
     
     
         20 . The method according to  claim 19 , wherein the machine learning model comprises a solid-state (SSD) failure prediction model.