Auxiliary information for reconstructing digital images processed through print-scan channels

Abstract

Systems and methods of generating and using auxiliary information for reconstructing digital images that have processed through print-scan channels are described. Auxiliary information, including values of reference pixels, is extracted from an input image. Output image data containing a representation of the input image is generated. The auxiliary information is stored in association with the output image data. The auxiliary data structure is decoded to produce decoded auxiliary information. Scanned image data that is obtained from a hard copy of the output image data is registered and color-corrected based on the decoded auxiliary information to obtain an output image corresponding to a high-quality reconstruction of the input image. The auxiliary information may include coded transform coefficient data from which the input image may be reconstructed using the registered and color-corrected output image as side information.

1 . An image processing method, comprising:
 extracting auxiliary information, including values of reference pixels, from an input image;   encoding the auxiliary information into an auxiliary data structure;   generating output image data containing a representation of the input image; and   in at least one physical storage medium, storing the auxiliary data structure in association with the output image data.   
   
   
       2 . The method of  claim 1 , wherein the extracting comprises identifying pixels at regularly spaced sample locations across the input image as ones of the reference pixels. 
   
   
       3 . The method of  claim 1 , wherein the extracting comprises identifying salient regions of the input image and selecting pixels in the salient regions as ones of the reference pixels. 
   
   
       4 . The method of  claim 3 , wherein the identifying comprises identifying corner regions of the input image as ones of the salient regions. 
   
   
       5 . The method of  claim 3 , wherein the identifying comprises applying to the input image at least one of an autocorrelation feature extractor, a variance feature extractor, and a scale-invariant feature transform to identify ones of the salient regions. 
   
   
       6 . The method of  claim 3 , wherein the extracting comprises selecting blocks of pixels corresponding to the salient regions as registration reference pixel blocks, and the encoding comprises encoding luminance values of the pixels of the registration reference pixels blocks into the auxiliary data structure. 
   
   
       7 . The method of  claim 1 , wherein the extracting comprises identifying regions of the input image characterized by respective texture measures that meet a specified low texture threshold and selecting pixels in the identified regions as color reference pixels, and the encoding comprises encoding color values of the color reference pixels into the auxiliary data structure. 
   
   
       8 . The method of  claim 1 , wherein the extracting comprises extracting as part of the auxiliary information at least one of: pixel dimensions of the input image; and a specified print size of the input image. 
   
   
       9 . The method of  claim 1 , further comprising:
 generating a sequence of quantized frequency domain vectors from a sequence of blocks of the input image, wherein each of the quantized frequency domain vectors comprises a set of quantized forward transform coefficients derived from a respective block of the input image; and   calculating a respective set of coset indices from each of the frequency domain vectors;   wherein the encoding comprises encoding a respective subset of each of the sets of coset indices into the auxiliary data structure.   
   
   
       10 . The method of  claim 1 , further comprising reconstructing the input image from scanned image data obtained from a hard copy of the output image data, wherein the reconstructing comprises:
 decoding the auxiliary data structure associated with the output image data to produce decoded auxiliary information;   estimating locations of corners of the input image in the scanned image data;   warping the scanned image data within the estimated corner locations to obtain sample points in the scanned image data; and   registering the portion of the scanned image data within the estimated corner locations based on correspondences between the reference pixel values in the decoded auxiliary information and values of sample points at corresponding locations in the scanned image data.   
   
   
       11 . The method of  claim 10 , wherein the reconstructing additionally comprises deriving a color transform between ones of the reference pixel values in the auxiliary data structure and values of corresponding ones of the sample points of the registered image, and applying the color transform to the registered image to obtain a color-corrected image. 
   
   
       12 . The method of  claim 11 , further comprising generating a sequence of quantized frequency domain vectors from a sequence of blocks of the input image, wherein each of the quantized frequency domain vectors comprises a set of quantized forward transform coefficients derived from a respective block of the input image, and calculating a respective set of coset indices from each of the frequency domain vectors;
 wherein the encoding comprises encoding a respective subset of each of the sets of coset indices into the auxiliary data structure, and the reconstructing comprises generating an output image from the subsets of the coset indices in the decoded auxiliary information using the color-corrected image as side information.   
   
   
       13 . The method of  claim 1 , wherein the generating comprises merging the input image and the auxiliary data structure into the output image data representing a layout of the input image and the auxiliary data structure. 
   
   
       14 . The method of  claim 13 , wherein the storing comprising printing the output image data onto at least one page of print media. 
   
   
       15 . An image processing method, comprising:
 obtaining scanned image data from a hard copy of output image data containing an input image;   decoding an auxiliary data structure associated with the output image data to produce decoded auxiliary information;   estimating locations of corners of the input image in the scanned image data;   warping the scanned image data within the estimated corner locations to obtain sample points in the scanned image data;   registering the portion of the scanned image data within the estimated corner locations based on correspondences between the reference pixel values in the decoded auxiliary information and values of sample points at corresponding locations in the scanned image data;   deriving a color transform between ones of the reference pixel values in the auxiliary data structure and values of corresponding ones of the sample points of the registered image; and   applying the color transform to the registered image to obtain a color-corrected image.   
   
   
       16 . The method of  claim 15 , wherein the decoding comprises decoding the auxiliary data structure to obtain estimates of transform coefficients extracted from the input image, and further comprising generating an output image from the transform coefficient estimates using the color-corrected image as side information. 
   
   
       17 . An image processing system, comprising:
 an auxiliary information processing component that extracts auxiliary information, including values of reference pixels, from an input image;   an encoding processing component that encodes the auxiliary information into an auxiliary data structure; and   an output image processing component that generates output image data containing a representation of the input image;   wherein at least one of the encoding processing component and the output image processing component stores the auxiliary data structure in association with the output image data in at least one physical storage medium.   
   
   
       18 . The system of  claim 17 , further comprising a scanned image processing system that reconstructs the input image from scanned image data obtained from a hard copy of the output image data, wherein the scanned image processing system comprises:
 a decoding processing component that decodes the auxiliary data structure associated with the output image data to produce decoded auxiliary information;   a preprocessing stage that estimates locations of corners of the input image in the scanned image data; and   an image reconstruction processing component that warps the scanned image data within the estimated corner locations to obtain sample points in the scanned image data, and registers the portion of the scanned image data within the estimated corner locations based on correspondences between the reference pixel values in the decoded auxiliary information and values of sample points at corresponding locations in the scanned image data.   
   
   
       19 . The system of  claim 18 , wherein the image reconstruction processing component additionally derives a color transform between ones of the reference pixel values in the auxiliary data structure and values of corresponding ones of the sample points of the registered image, and applies the color transform to the registered image to obtain a color-corrected image. 
   
   
       20 . The system of  claim 19 , wherein:
 the auxiliary information processing component generates a sequence of quantized frequency domain vectors from a sequence of blocks of the input image, each of the quantized frequency domain vectors comprising a set of quantized forward transform coefficients derived from a respective block of the input image, and calculating a respective set of coset indices from each of the frequency domain vectors;   the encoding processing component encodes a respective subset of each of the sets of coset indices into the auxiliary data structure; and   the reconstruction processing component generates an output image from the subsets of the coset indices in the decoded auxiliary information using the color-corrected image as side information.