Shared file system predictive storage techniques

Abstract

Disclosed in some examples are predictive storage techniques for use in a distributed data system. The predictive storage techniques may be used to manage locally stored elements of a shared data collection, such as the storage of files on nodes of the distributed data system that are limited in local storage space. The predictive storage techniques may achieve a balance between consumption of local resources and timely access of important elements in the shared data collection. For example, the predictive storage techniques may be used for keeping or pre-caching certain items of a collection that are determined as likely to be used in local storage for convenient access, and allowing access the remaining items on request over a network.

What is claimed is: 
     
         1 . A method of predictive data storage for a node device in a data distribution system, the method comprising using at least one processor to perform the operations of:
 scoring each particular element of a plurality of file system elements of a collection of the data distribution system, the scoring using a calculated probability that a user of the node device is likely to interact with the particular element;   calculating an on-demand subset of the collection based upon the scores of the respective plurality of file system elements, wherein the on-demand subset includes fewer elements than the collection; and   causing the element to be placed in the local storage of the node device in response to a determination that the at least one of the on-demand subset is not already in the local storage of the node device.   
     
     
         2 . The method of  claim 1 , wherein scoring each particular element of the plurality of file system elements includes utilizing prior usage history corresponding to at least one of the plurality of file system elements. 
     
     
         3 . The method of  claim 2 , wherein the prior usage history includes information describing interactions of all users of the collection with the at least one of the plurality of file system elements. 
     
     
         4 . The method of  claim 3 , wherein a component of the prior usage history corresponding to the user of the node device is weighted greater than a component of the prior usage history corresponding to other users of the collection. 
     
     
         5 . The method of  claim 1 , wherein scoring each particular element of the plurality of file system elements includes utilizing contextual data specifying a context corresponding to a prior usage history. 
     
     
         6 . The method of  claim 5 , wherein the prior usage history indicates an interaction by a particular user of the collection with one of the plurality of elements in the collection and wherein the contextual data signals a particular situation in which the particular user was in when the particular user interacted with the one of the plurality of elements. 
     
     
         7 . The method of  claim 1 , the operations further including:
 building a machine learning model;   wherein scoring includes using the machine learning model.   
     
     
         8 . The method of  claim 1 , wherein determining the on-demand subset includes selecting the highest scoring elements in the collection until a predetermined limit on one of: a local storage size and a number of elements in the on-demand set has been reached. 
     
     
         9 . The method of  claim 1 , wherein determining the on-demand subset includes selecting a combination of the respective plurality of elements that results in the highest combined score of selected elements given one of: a size constraint on the maximum number of elements in the on-demand set and a size constraint on the maximum total size of the elements in the on-demand set. 
     
     
         10 . The method of  claim 1 , wherein causing the element to be placed in local storage on the node device includes:
 requesting the element from a second node device in the data distribution system over a computer network;   receiving the element from the second node device; and responsive to receiving the element from the second node, storing the element in the local storage of the node device.   
     
     
         11 . A system for predictive data storage for a node device in a data distribution system, the system comprising:
 a prediction module comprising circuitry to:   score each particular element of a plurality of file system elements of a collection of the data distribution system, the score based upon a calculated probability that a user of the node device is likely to interact with the respective element;   a control module comprising circuitry to:   calculate an on-demand subset of the collection based upon the scores of the respective plurality of file system elements, wherein the on-demand subset includes fewer elements than the collection; and   cause the element to be placed in local storage of the node responsive to a determination that the at least one of the on-demand subset is not already in the local storage of the node device.   
     
     
         12 . The system of  claim 11 , wherein the prediction module is configured to score each particular element of the plurality of file system elements by at least utilizing prior usage history corresponding to at least one of the plurality of file system elements. 
     
     
         13 . The system of  claim 11 , wherein the prediction module is configured to score each particular element of the plurality of file system elements by at least utilizing contextual data specifying a context corresponding to a prior usage history. 
     
     
         14 . The system of  claim 13 , wherein the prior usage history indicates an interaction by a particular user of the collection with one of the plurality of elements in the collection, and wherein the contextual data signals a particular situation in which the particular user was in when the particular user interacted with the one of the plurality of elements. 
     
     
         15 . The system of  claim 11 , wherein the prediction module is configured to build a machine learning model, and wherein the prediction module is configured to score by at least using the machine learning model. 
     
     
         16 . A non-transitory machine-readable medium, for predictive data storage for a node device in a data distribution system, the machine-readable including instructions, which when performed by the machine, cause the machine to perform the operations of:
 scoring each particular element of a plurality of file system elements of a collection of the data distribution system, the scoring using a calculated probability that a user of the node device is likely to interact with the particular element;   calculating an on-demand subset of the collection based upon the scores of the respective plurality of file system elements, wherein the on-demand subset includes fewer elements than the collection; and   causing the element to be placed in the local storage of the node device in response to a determination that the at least one of the on-demand subset is not already in the local storage of the node device.   
     
     
         17 . The machine-readable medium of  claim 16 , wherein the operations of scoring each particular element includes utilizing prior usage history corresponding to at least one of the plurality of file system elements. 
     
     
         18 . The machine-readable medium of  claim 17 , wherein the prior usage history includes information describing interactions of all users of the collection with the at least one of the plurality of file system elements. 
     
     
         19 . The machine-readable medium of  claim 18 , wherein a component of the prior usage history corresponding to the user of the node is weighted greater than a component of the prior usage history corresponding to other users of the collection. 
     
     
         20 . The machine-readable medium of  claim 16 , wherein the operations of scoring each particular element includes utilizing contextual data specifying a context corresponding to a prior usage history. 
     
     
         21 . The machine-readable medium of  claim 20 , wherein the prior usage history indicates an interaction by a particular user of the collection with one of the plurality of elements in the collection and wherein the contextual data signals a particular situation in which the particular user was in when the particular user interacted with the one of the plurality of elements. 
     
     
         22 . The machine-readable medium of  claim 16 , wherein the operations include building a machine learning model; and wherein the operations of scoring include using the machine learning model. 
     
     
         23 . The machine-readable medium of  claim 16 , wherein the operations of determining the on-demand subset includes selecting the highest scoring elements in the collection until a predetermined limit on one of: a local storage size and a number of elements in the on-demand set has been reached. 
     
     
         24 . The machine-readable medium of  claim 16 , wherein the operations of determining the on-demand subset includes selecting a combination of the respective plurality of elements that results in the highest combined score of selected elements given one of: a size constraint on the maximum number of elements in the on-demand set and a size constraint on the maximum total size of the elements in the on-demand set. 
     
     
         25 . The machine-readable medium of  claim 16 , wherein the operations of causing the element to be placed in local storage on the node includes:
 requesting the element from a second node device in the data distribution system over a computer network;   receiving the element from the second node device; and responsive to receiving the element from the second node device, storing the element in the local storage of the node device.