Systems and methods for efficient distribution of stored data objects

Abstract

A distributed data storage system is provided for offering shared data to one or more clients. In various embodiments, client systems operate on shared data while having a unique writeable copy of the shared data. According to one embodiment, the data storage system can be optimized for various use cases (e.g., read-mostly where writes to shared data are rare or infrequent (although writes to private data may be frequent. Some implementations of the storage system are configured to provide fault tolerance and scalability for the shared storage. For example, read-only data can be stored in (relatively) high latency, low cost, reliable storage (e.g. cloud), with multiple layers of cache supporting faster retrieval. In addition, some implementations of the data storage system offer a low-latency approach to data caching. Other embodiments improve efficiency with access modeling and conditional execution cache hints that can be distributed across the data storage system.

What is claimed is: 
     
         1 . A proxy unit, comprising:
 a copy-on-write layer;   a read cache;   a read overlay;   a hardware processor device that executes programmed instructions that when executed:   provide an electronic client device external to the proxy unit access to common data stored in an electronic storage unit, the common data being accessed by the hardware processor device through a first server, wherein the copy-on-write layer, the read cache, and the read overlay form a layered architecture;   execute remote requests received from an application on the common data and data in the copy-on-write layer;   wherein the application modifies the data in a file system that is external to the proxy unit, and wherein after the application has modified data in the file system, the modifications are stored in the copy-on-write layer and the application subsequently disconnects from the proxy unit;   wherein after the application disconnects from the proxy unit, the copy-on-write layer is saved in the read-overlay;   wherein the read-overlay including the modifications is uploaded to a first server and a second server to be made available to any other proxy unit in any other client device.   
     
     
         2 . The proxy unit of  claim 1 , wherein the proxy unit is included in an electronic client device, and the electronic client device includes the computer application and the file system. 
     
     
         3 . The proxy unit of  claim 2 , wherein the proxy unit executes on the client device. 
     
     
         4 . The proxy unit of  claim 1 , wherein the hardware processor device of the proxy unit is further configured to manage a local cache for pre-fetching data responsive to cache hints. 
     
     
         5 . The proxy unit of  claim 1 , wherein the hardware processor device of the proxy unit is configured to retrieve and store data from the first server or the storage unit in the read cache responsive to data access patterns for respective client devices. 
     
     
         6 . The proxy unit of  claim 1 , wherein the hardware processor device of the proxy unit is configured to:
 interact with the copy-on-write layer; and   request data from the copy-on-write layer for respective clients.   
     
     
         7 . The proxy unit of  claim 1 , wherein the hardware processor device of the proxy unit is configured to:
 host at least a portion of data managed in a copy-on-write layer; and   store any data written by a respective client associated with at least the portion within the copy-on-write layer.   
     
     
         8 . The proxy unit of  claim 1 , wherein the common data is configured to be available only in read-only form. 
     
     
         9 . The proxy unit of  claim 1 , wherein the hardware processor device of the proxy unit is configured to load the data read from the first server or the storage unit into the read cache based on at least one predicted request. 
     
     
         10 . A computer implemented method, comprising:
 providing a copy-on-write layer, a read cache, and a read overlay;   by a hardware processor device that executes programmed instructions, providing an electronic client device external to the proxy unit access to common data stored in an electronic storage unit, the common data being accessed by the hardware processor device through a first server, wherein the copy-on-write layer, the read cache, and the read overlay form a layered architecture;   by the hardware processor device, executing remote requests received from an application on the common data and data in the copy-on-write layer;   wherein the application modifies the data in a file system that is external to the proxy unit, and wherein after the application has modified data in the file system, the modifications are stored in the copy-on-write layer and the application subsequently disconnects from the proxy unit;   wherein after the application disconnects from the proxy unit, the copy-on-write layer is saved in the read-overlay;   wherein the read-overlay including the modifications is uploaded to a first server and a second server to be made available to any other proxy unit in any other client device.   
     
     
         11 . The method of  claim 10 , wherein the method is executed at an electronic client device, and the electronic client device includes the computer application and the file system. 
     
     
         12 . The method of  claim 10 , wherein the hardware processor device is further configured to manage a local cache for pre-fetching data responsive to cache hints. 
     
     
         13 . The method of  claim 10 , wherein the hardware processor device is configured to retrieve and store data from the first server or the storage unit in the read cache responsive to data access patterns for respective client devices. 
     
     
         14 . The method of  claim 10 , wherein the hardware processor device is configured to:
 interact with the copy-on-write layer; and   request data from the copy-on-write layer for respective clients.   
     
     
         15 . The method of  claim 10 , wherein the hardware processor device is configured to:
 host at least a portion of data managed in a copy-on-write layer; and   store any data written by a respective client associated with at least the portion within the copy-on-write layer.   
     
     
         16 . The method of  claim 10 , wherein the common data is configured to be available only in read-only form. 
     
     
         17 . The method of  claim 10 , wherein the hardware processor device is configured to load the data read from the first server or the storage unit into the read cache based on at least one predicted request.