Representing and reconstructing high dynamic range images

Abstract

A high dynamic range image can be recovered from a full-resolution lower-dynamic-range image and a reduced-resolution higher-dynamic-range image. Information regarding higher spatial frequencies may be obtained by extracting high spatial frequencies from the lower-dynamic-range image. In some embodiments an approximate impulse-response function is determined by comparing the higher- and lower-dynamic range images. A scaling image obtained by applying the impulse-response function to a high-frequency band of the lower-dynamic range image is combined with an upsampled higher-dynamic range image to yield a reconstructed image.

1. Apparatus for reconstructing a higher-dynamic-range image from a full resolution, lower dynamic range (FRLDR) representation of an image and a lower resolution, higher-dynamic-range (LRHDR) representation of the image, the LRHDR representation having a lower resolution and a higher dynamic range than the FRLDR representation, the apparatus comprising:
 a spatial filtering unit configured to generate a scaling image comprising information relating to high spatial frequency components of the FRLDR representation; 
 an upsampling unit configured to generate an upsampled image from the LRHDR representation; and, 
 a combining unit configured to combine the scaling image and the upsampled image to obtain a reconstructed higher-dynamic-range image, 
 wherein each of the spatial filtering unit, upsampling unit and combining unit comprises image-processing hardware, software stored on a non-transitory computer-readable medium and executable by a data processor, or a combination thereof. 
 
     
     
       2. Apparatus according to  claim 1  comprising a display wherein the apparatus is configured to display the reconstructed higher-dynamic range image on the display. 
     
     
       3. Apparatus according to  claim 1  wherein the FRLDR representation is created from an original full resolution, high-dynamic-range representation of the image using a tone-mapping operation and wherein the spatial filtering unit is configured to perform an inverse tone-mapping operation. 
     
     
       4. Apparatus according to  claim 1  wherein the spatial filtering unit is configured to:
 downsample the FRLDR representation to obtain a downsampled image having a resolution sufficiently low to comprise only spatial frequency components below a desired spatial frequency level; 
 upsample the downsampled image to obtain a resampled image; and 
 combine the resampled image and the FRLDR representation to generate the scaling image. 
 
     
     
       5. Apparatus according to  claim 4  wherein the spatial filtering unit is configured to divide each pixel value of the FRLDR representation by a corresponding pixel value of the resampled image. 
     
     
       6. Apparatus according to  claim 4  wherein an average of pixel values in the scaling image is 1. 
     
     
       7. Apparatus according to  claim 1  wherein the combining unit is configured to multiply each pixel value of the scaling image with a corresponding pixel value of the upsampled image. 
     
     
       8. Apparatus according to  claim 1  wherein the spatial filtering unit is configured to apply a convolution filter to the FRLDR representation. 
     
     
       9. Apparatus according to  claim 1  wherein the spatial filtering unit is configured to convert a color space of the FRLDR representation into a color space of the LRHDR representation. 
     
     
       10. Apparatus according to  claim 1  wherein the spatial filtering unit is configured to:
 determine an impulse-response function from the FRLDR representation and the LRHDR representation; 
 extract high spatial frequency components of the FRLDR representation; and 
 multiply each pixel value of the extracted high spatial frequency components by a corresponding value of the impulse-response function to generate the scaling image. 
 
     
     
       11. Apparatus according to  claim 10  wherein the spatial filtering unit is configured to apply a quantization threshold to the high spatial frequency components. 
     
     
       12. Apparatus according to  claim 11  wherein the spatial filtering unit is configured to:
 extract high spatial frequency components of the LRHDR representation to obtain a high frequency band of the LRHDR representation; 
 reduce a resolution of the FRLDR representation to obtain a reduced resolution low-dynamic-range image; 
 extract high spatial frequency components of the reduced resolution low-dynamic-range image to obtain a high frequency band of the reduced resolution low-dynamic-range image; and 
 process the high frequency band of the LRHDR representation and the high frequency band of the reduced resolution low-dynamic-range image to determine the impulse-response function. 
 
     
     
       13. Apparatus according to  claim 12  wherein the spatial filtering unit is configured to:
 select a group of pixels from within a spatial region of the high frequency band of the LRHDR representation and selecting a corresponding group of pixels from within the spatial region of the high frequency band of the reduced resolution low-dynamic-range image; 
 sort the group of pixels and the corresponding group of pixels by pixel value to obtain a pair of sorted arrays defining a regional impulse-response function corresponding to the spatial region; and, 
 combine one or more regional impulse-response functions to obtain the impulse-response function. 
 
     
     
       14. Apparatus according to  claim 13  wherein the spatial filtering unit is configured to select and sort groups of pixels for a plurality of spatial regions. 
     
     
       15. Apparatus according to  claim 13  wherein the spatial filtering unit is configured to:
 select a plurality of pairs of pixel values from the pair of sorted arrays, each pair of pixel values comprising pixel values having a common index within the pair of sorted arrays; and, 
 interpolate between the plurality of pairs of pixel values. 
 
     
     
       16. Apparatus according to  claim 13  wherein the spatial filtering unit is configured to:
 determine a regional impulse-response function value for each of a plurality of regions that include the pixel; and, 
 computing a weighted combination of the regional impulse-response function values. 
 
     
     
       17. Apparatus according to  claim 16  wherein the spatial filtering unit is configured to weight the regional impulse-response function values based on distances of the pixel from central points in the regions to which the regional impulse-response function values correspond. 
     
     
       18. Apparatus according to  claim 15  wherein the spatial filtering unit is configured to extrapolate the regional impulse-response function past at least one of the smallest and largest ones of the pairs of pixel values. 
     
     
       19. A non-transitory computer-readable medium with a set of instructions stored thereon, that, when loaded into an electronic controller, cause the controller to perform the steps of:
 receiving a full resolution, lower dynamic range (FRLDR) representation of the image and a lower resolution, higher-dynamic-range (LRHDR) representation of the image, the LRHDR representation having a lower resolution and a higher dynamic range than the FRLDR representation; 
 processing at least the FRLDR representation to obtain a scaling image comprising information relating to high spatial frequency components of the FRLDR representation; 
 upsampling the LRHDR representation to obtain an upsampled image; and 
 combining the scaling image and the upsampled image to obtain a reconstructed higher-dynamic-range image. 
 
     
     
       20. Apparatus according to  claim 1  wherein the apparatus is integrated with a DVD player. 
     
     
       21. Apparatus according to  claim 1  wherein the apparatus is integrated with a video player. 
     
     
       22. Apparatus according to  claim 1  wherein the apparatus is integrated with a computer display, the apparatus configured to display the reconstructed higher-dynamic range image on the computer display. 
     
     
       23. Apparatus according to  claim 1  wherein the apparatus is integrated with a television having a display, the apparatus configured to display the reconstructed higher-dynamic range image on the display. 
     
     
       24. Apparatus according to  claim 1  wherein the apparatus is integrated with an image-displaying device having a display, the apparatus configured to display the reconstructed higher-dynamic range image on the display.