Full allocation volume to deduplication volume migration in a storage system

Abstract

A method, computer program product, and computer system for full allocation volume to deduplication volume migration in a storage system. The method includes moving a physical allocation of stored data associated with a full allocation volume into a virtual address range in a deduplication domain and setting up the deduplication metadata to be a passthrough. The method then performs a background deduplication process on the virtual address range once populated with the physical allocations using a drive query hash interface to perform hash calculations on physical drives at which the data is stored.

What is claimed is: 
     
       1. A computer-implemented method for full allocation volume to deduplication volume migration in a storage system, said method carried out by one or more processors of a computer system and comprising:
 moving a physical allocation of stored data associated with a full allocation volume into a virtual address range in a deduplication domain and setting up the deduplication metadata to be a passthrough, wherein setting up the deduplication metadata to be a passthrough includes:
 creating a passthrough forward lookup structure for assimilation of the fully allocated volume to the deduplicated volume; and creating a stub backward lookup structure to be updated with deduplication metadata as the deduplicated volume is deduplicated; and 
 
 performing a background deduplication process on the virtual address range once populated with the physical allocations using a drive query hash interface to perform hash calculations on physical drives at which the data is stored, wherein the virtual address range becomes a deduplicated volume for the data. 
 
     
     
       2. The method of  claim 1 , including refraining from moving data stored on the physical drives during the method. 
     
     
       3. The method of  claim 1 , wherein the physical allocation of stored data is a managed disk of data extents associated with physical data extents stored across a plurality of physical drives having excess compute capability at which hash calculations for the deduplication process are performed. 
     
     
       4. The method of  claim 1 , including pausing input/output operations whilst moving a physical allocation of stored data associated with a full allocation volume into a virtual address range in a deduplication domain. 
     
     
       5. The method of  claim 1 , including reserving, by a deduplication layer, the virtual address range in the deduplication domain to which the physical allocations associated with the fully allocated volume are moved. 
     
     
       6. The method of  claim 1 , wherein moving a physical allocation of stored data associated with a full allocation volume into a virtual address range in a deduplication domain is carried out by a virtualization layer and includes marking the volume as a deduplicated volume and activating a deduplication layer and the passthrough metadata. 
     
     
       7. The method of  claim 1 , wherein performing a background deduplication process includes iterating over grains of the deduplication volume by a deduplication layer with locking of a grain to an input/output operation during a grain deduplication. 
     
     
       8. The method of  claim 7 , wherein iterating over grains includes:
 when a grain is a deduplication hit, creating referrer metadata on a passthrough forward lookup structure to point to source metadata and marking the grain as invalid; 
 when a grain is a deduplication miss, storing a hash for the grain in a deduplication fingerprint database, creating source metadata on the passthrough forward lookup structure to point to grain on data disk; and 
 for all grains, updating a backward lookup structure with invalidations or virtual addresses. 
 
     
     
       9. The method of  claim 1 , including carrying out hash calculations at a Flash Core Module (FCM) on the drive layer without reading the data from outside the FCM. 
     
     
       10. A system for full allocation volume to deduplication volume migration in a storage system including a processor and a memory configured to provide computer program instructions to the processor to execute the function of the components, the system comprising:
 a physical allocation moving component in a virtualization layer for moving a physical allocation of stored data associated with a full allocation volume into a virtual address range in a deduplication domain; a deduplication metadata setting up component for setting up the deduplication metadata to be a passthrough, wherein setting up the deduplication metadata to be a passthrough includes:
 creating a passthrough forward lookup structure for assimilation of the fully allocated volume to the deduplicated volume; and creating a stub backward lookup structure to be updated with deduplication metadata as the deduplicated volume is deduplicated; and 
 
 a deduplication component of a deduplication layer for performing a background deduplication process on the virtual address range once populated with the physical allocations using a drive query hash interface to perform hash calculations on physical drives at which the data is stored, wherein the virtual address range becomes a deduplicated volume for the data. 
 
     
     
       11. The system of  claim 10 , wherein the physical allocation of stored data is a managed disk of data extents associated with physical data extents stored across a plurality of physical drives having excess compute capability at which hash calculations for the deduplication process are performed. 
     
     
       12. The system of  claim 10 , including a pausing component for pausing input/output operations whilst moving a physical allocation of stored data associated with a full allocation volume into a virtual address range in a deduplication domain. 
     
     
       13. The system of  claim 10 , including a reserving component in the deduplication layer for reserving the virtual address range in the deduplication domain to which the physical allocations associated with the fully allocated volume are moved. 
     
     
       14. The system of  claim 10 , wherein the physical allocation moving component includes marking the volume as a deduplicated volume and activating a deduplication layer and the passthrough metadata. 
     
     
       15. The system of  claim 10 , wherein the deduplication component includes a grain iterating component for iterating over grains of the deduplication volume by a deduplication layer with locking of a grain to an input/output operation during a grain deduplication. 
     
     
       16. The system of  claim 15 , wherein the grain iterating component includes:
 a hit component for, when a grain is a deduplication hit, creating referrer metadata on a passthrough forward lookup structure to point to source metadata and marking the grain as invalid; 
 a miss component for, when a grain is a deduplication miss, storing a hash for the grain in a deduplication fingerprint database, creating source metadata on the passthrough forward lookup structure to point to grain on data disk; and 
 a backward lookup update component for updating a backward lookup structure with invalidations or virtual addresses. 
 
     
     
       17. The system of  claim 10 , wherein the plurality of physical drives are Flash Core Modules (FCM). 
     
     
       18. A computer program product for full allocation volume to deduplication volume migration in a storage system, the non-transitory computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to:
 move a physical allocation of stored data associated with a full allocation volume into a virtual address range in a deduplication domain and set up the deduplication metadata to be a passthrough, wherein setting up the deduplication metadata to be a passthrough includes:
 creating a passthrough forward lookup structure for assimilation of the fully allocated volume to the deduplicated volume; and creating a stub backward lookup structure to be updated with deduplication metadata as the deduplicated volume is deduplicated; and 
 
 perform a background deduplication process on the virtual address range once populated with the physical allocations using a drive query hash interface to perform hash calculations on physical drives at which the data is stored, wherein the virtual address range becomes a deduplicated volume for the data.