Generation of high dynamic range images from low dynamic range images

Abstract

An approach is provided for generating a high dynamic range image from a low dynamic range image. The generation is performed using a mapping relating input data in the form of input sets of image spatial positions and a combination of color coordinates of low dynamic range pixel values associated with the image spatial positions to output data in the form of high dynamic range pixel values. The mapping is generated from a reference low dynamic range image and a corresponding reference high dynamic range image. Thus, a mapping from the low dynamic range image to a high dynamic range image is generated on the basis of corresponding reference images. The approach may be used for prediction of high dynamic range images from low dynamic range images in an encoder and decoder. A residual image may be generated and used to provide improved high dynamic range image quality.

1 . A method of encoding an input image, the method comprising:
 receiving the input image;   generating a mapping relating input data in the form of input sets of image spatial positions and a combination of color coordinates of low dynamic range pixel values associated with the image spatial positions to output data in the form of high dynamic range pixel values in response to a reference low dynamic range image and a corresponding reference high dynamic range image; and   generating an output encoded data stream by encoding the input image in response to the mapping.   
     
     
         2 . The method of  claim 1  wherein the input image is an input high dynamic range image; and the method further comprises:
 receiving an input low dynamic range image corresponding to the input high dynamic range image; 
 generating a prediction base image from the input low dynamic range image 
 predicting a predicted high dynamic range image from the prediction base image in response to the mapping; 
 encoding a residual high dynamic range image in response to the predicted high dynamic range image and the input high dynamic range image to generate encoded high dynamic range data; and 
 including the encoded high dynamic range data in the output encoded data stream. 
 
     
     
         3 . The method of  claim 1  wherein each input set corresponds to a spatial interval for each spatial image dimension and at least one value interval for the combination, and the generation of the mapping comprises for each image position of at least a group of image positions of the reference low dynamic range image:
 determining at least one matching input set having spatial intervals corresponding to the each image position and a value interval for the combination corresponding to a combination value for the each image position in the reference low dynamic range image; and 
 determining an output high dynamic range pixel value for the matching input set in response to a high dynamic range pixel value for the each image position in the reference high dynamic range image. 
 
     
     
         4 . The method of  claim 1  wherein the mapping is at least one of:
 a spatially subsampled mapping; 
 a temporally subsampled mapping; and 
 a combination value subsampled mapping. 
 
     
     
         5 . The method of  claim 1  wherein the input image is an input high dynamic range image; and the method further comprises:
 receiving an input low dynamic range image corresponding to the input high dynamic range image; 
 generating a prediction base image from the input low dynamic range image 
 predicting a predicted high dynamic range image from the prediction base image in response to the mapping; and 
 adapting at least one of the mapping and a residual high dynamic range image for the predicted high dynamic range image in response to a comparison of the input high dynamic range image and the predicted high dynamic range image. 
 
     
     
         6 . The method of  claim 1  wherein the input image is the reference high dynamic range image and the reference low dynamic range image is an input low dynamic range image corresponding to the input image. 
     
     
         7 . The method of  claim 1  wherein the input sets for the mapping further comprises depth indications associated with image spatial positions and the mapping further reflects a relationship between depth and high dynamic range pixel values. 
     
     
         8 . The method of  claim 1 , wherein the generating an output encoded data stream comprises adding a derived mapping specification to the output encoded data stream on the basis of at least parts of the mapping. 
     
     
         9 . A method of generating a high dynamic range image from a low dynamic range image, the method comprising:
 receiving the low dynamic range image;   providing a mapping relating input data in the form of input sets of image spatial positions and a combination of color coordinates of low dynamic range pixel values associated with the image spatial positions to output data in the form of high dynamic range pixel values, the mapping reflecting a dynamic range relationship between a reference low dynamic range image and a corresponding reference high dynamic range image; and   generating the high dynamic range image in response to the low dynamic range image and the mapping.   
     
     
         10 . The method of  claim 9  wherein generating the high dynamic range image comprises determining at least part of a predicted high dynamic range image by for each position of at least part of the predicted dynamic range image:
 determining at least one matching input set matching the each position and a first combination of color coordinates of low dynamic range pixel values associated with the each position; 
 retrieving from the mapping at least one output high dynamic range pixel value for the at least one matching input set; 
 determining a high dynamic range pixel value for the each position in the predicted high dynamic range image in response to the at least one output high dynamic range pixel value; and 
 determining the high dynamic range image in response to the at least part of the predicted high dynamic range image. 
 
     
     
         11 . The method of  claim 9  wherein the low dynamic range image is an image of a low dynamic range video sequence and the method comprises generating the mapping using a previous low dynamic range image of the low dynamic range video sequence as the reference low dynamic range image and a previous high dynamic range image generated for the previous low dynamic range image as the reference high dynamic range image. 
     
     
         12 . The method of  claim 11  wherein the previous high dynamic range image is further generated in response to residual image data for the previous low dynamic range image relative to predicted image data for the previous low dynamic range image. 
     
     
         13 . The method of  claim 9  wherein the low dynamic range image is an image of a low dynamic range video sequence, and the method further comprises using a nominal mapping for at least some low dynamic range images of the low dynamic range video sequence. 
     
     
         14 . The method of  claim 9  wherein the combination is indicative of at least one of a texture, gradient, and spatial pixel value variation for the image spatial positions. 
     
     
         15 . The method of  claim 9  wherein the input sets for the mapping further comprises depth indications associated with image spatial positions, and the mapping further reflects a relationship between depth and high dynamic range pixel values. 
     
     
         16 . A device for encoding an input image, the device comprising:
 a receiver for receiving the input image;   a mapping generator for generating a mapping relating input data in the form of input sets of image spatial positions and a combination of color coordinates of low dynamic range pixel values associated with the image spatial positions to output data in the form of high dynamic range pixel values for a reference low dynamic range image and a corresponding reference high dynamic range image; and   an output processor for generating output encoded data by encoding the input image in response to the mapping.   
     
     
         17 . A device as claimed in  claim 16 , in which the output processor is arranged to include in the output encoded data at least one of a derived mapping and a residual high dynamic range image. 
     
     
         18 . An apparatus comprising
 the device of  claim 17 ;   input connection means for receiving a signal comprising the input image and feeding it to the device of  claim 17 ; and   output connection means for outputting the output encoded data stream from the device of  claim 17 .   
     
     
         19 . A device for generating a high dynamic range image from a low dynamic range image, the device comprising:
 a receiver for receiving the low dynamic range image;   a mapping processor for providing a mapping relating input data in the form of input sets of image spatial positions and a combination of color coordinates of low dynamic range pixel values associated with the image spatial positions to output data in the form of high dynamic range pixel values, the mapping reflecting a dynamic range relationship between a reference low dynamic range image and a corresponding reference high dynamic range image; and   an image generator for generating the high dynamic range image in response to the low dynamic range image and the mapping.   
     
     
         20 . A device for generating a high dynamic range image from a low dynamic range image as claimed in  claim 19 , comprising means to receive a residual high dynamic range image and a correction unit to apply e.g. by addition the residual high dynamic range image to the high dynamic range image in response to the low dynamic range image from the mapping. 
     
     
         21 . A device for generating a high dynamic range image from a low dynamic range image as claimed in  claim 19 , in which the mapping processor is further arranged to determine the mapping at least partially on the basis of a received derived mapping. 
     
     
         22 . An apparatus comprising:
 the device of  claim 19 ;   input connection means for receiving the low dynamic range image and feeding it to the device of  claim 19 ;   output connection means for outputting a signal comprising the high dynamic range image from the device of  claim 19 .   
     
     
         23 . An encoded signal comprising:
 an encoded low dynamic range image; and   residual image data for the low dynamic range image, at least part of the residual image data being indicative of a difference between a desired high dynamic range image corresponding to the low dynamic range image and a predicted high dynamic range image resulting from application of a mapping to the encoded low dynamic range image, where the mapping relates input data in the form of input sets of image spatial positions and a combination of color coordinates of low dynamic range pixel values associated with the image spatial positions to output data in the form of high dynamic range pixel values, the mapping reflecting a dynamic range relationship between a reference low dynamic range image and a corresponding reference high dynamic range image.   
     
     
         24 . An encoded signal as claimed in  claim 23  further comprising at least one of a further information for specifying or modifying the mapping or information specifying properties of images such as spatial parts of images from which to determine the mapping. 
     
     
         25 . A storage medium comprising the encoded signal of  claim 23 .