Image processing apparatus, image processing method, and program

Abstract

There is provided an image processing apparatus including: an exposure compensation unit carrying out exposure compensation of a long exposure image and a short exposure image in keeping with an exposure ratio; a blur information measuring unit using the exposure-compensated images to calculate blur information showing a degree of discrepancy between pixel values at a corresponding pixel position in the long exposure image and the short exposure image; a blending coefficient calculating unit using the blur information to decide a blending coefficient to be used in a blending process on pixel values at the corresponding pixel position in the long exposure image and the short exposure image; and a blending unit carrying out the blending process on the pixel values at the corresponding pixel position in the long exposure image and the short exposure image using the calculated blending coefficient to decide the pixel value of an output image.

What is claimed is: 
     
         1 . An image processing apparatus comprising:
 an image processing unit configured to generate at least two pixel values having different exposure times respectively,   wherein the image processing unit includes:
 a compensation unit configured to carry out exposure compensation of the at least two pixel values; 
 an information measuring unit configured to use the at least two pixel values after the exposure compensation to calculate blur information that is an index value showing a degree of discrepancy between the at least two pixel values; 
 a coefficient calculating unit configured to use the blur information to decide a blending coefficient as a blending ratio; and 
 a blending unit configured to carry out a pixel value blending process on the at least two pixel values at a corresponding pixel position. 
   
     
     
         2 . The image processing apparatus according to  claim 1 , wherein the image processing unit is further configured to receive one image containing at least some pixel values corresponding to long exposure pixels and at least some pixel values corresponding to short exposure pixels, and the image processing unit further includes:
 an interpolation unit configured to carry out an interpolation process on the one image to generate a long exposure image, in which all pixels of the long exposure image are set at pixel values of long exposure pixels of an image pickup element, and a short exposure image, in which all pixels are set at pixel values of short exposure pixels of the image pickup element, wherein:   the compensation unit is further configured to carry out the exposure compensation in keeping with an exposure ratio between the long exposure pixels and the short exposure pixels on the long exposure image and the short exposure image generated by the interpolation unit, and   the information measuring unit is further configured to calculate the blur information using the compensated long exposure image and the compensated short exposure image generated by the compensation unit.   
     
     
         3 . The image processing apparatus according to  claim 2 , wherein a first pixel value of the at least two pixel values corresponds to the long exposure image and a second pixel value of the at least two pixel values corresponds to the short exposure image. 
     
     
         4 . The image processing apparatus according to  claim 1 , wherein the information measuring unit is configured to calculate the blur information for each color of a plurality of colors using the at least two pixel values that have been subjected to the exposure compensation and to output a maximum value of the blur information for each color at the corresponding pixel position as the blur information corresponding to a pixel located at the corresponding pixel position. 
     
     
         5 . The image processing apparatus according to  claim 1 , wherein the information measuring unit includes an estimating unit configured to estimate noise included in the at least two pixel values and to calculate the blur information corresponding to a pixel located at the corresponding pixel position by subtracting the noise estimated by the estimating unit from a value calculated based on a difference between the at least two pixel values that have been subjected to the exposure compensation. 
     
     
         6 . The image processing apparatus according to  claim 1 , wherein for a pixel value of the at least two pixel values that corresponds to a long exposure time and is at a saturation level, the coefficient calculating unit is configured to calculate a blending coefficient that suppresses the blending ratio of the pixel value that corresponds to the long exposure time. 
     
     
         7 . The image processing apparatus according to  claim 1 , wherein the blending unit is configured to carry out the pixel value blending process on the at least two pixel values at the corresponding pixel position to generate a high dynamic range image as an output image. 
     
     
         8 . An image processing method comprising:
 compensating for exposure of at least two pixel values having different exposure times respectively;   after compensating, calculating blur information based on the at least two pixel values, wherein the blur information is an index value showing a degree of discrepancy between the at least two pixel values;   deciding a blending coefficient, the blending coefficient representing a blending ratio, based on the blur information; and   performing a pixel value blending process on the at least two pixel values at a corresponding pixel position.   
     
     
         9 . The image processing method according to  claim 8 , further comprising:
 receiving one image containing at least some pixel values corresponding to long exposure pixels and at least some pixel values corresponding to short exposure pixels; and   performing an interpolation process on the one image to generate a long exposure image, in which all pixels of the long exposure image are set at pixel values of long exposure pixels of an image pickup element, and a short exposure image, in which all pixels are set at pixel values of short exposure pixels of the image pickup element, wherein:
 compensating for exposure of the at least two pixel values having the different exposure times respectively comprises compensating for exposure of the at least two pixel values having the different exposure times respectively in keeping with an exposure ratio between the long exposure pixels and the short exposure pixels on the long exposure image and the short exposure image, and 
 calculating the blur information comprises using the compensated long exposure image and the compensated short exposure image. 
   
     
     
         10 . The image processing method according to  claim 9 , wherein a first pixel value of the at least two pixel values corresponds to the long exposure image and a second pixel value of the at least two pixel values corresponds to the short exposure image. 
     
     
         11 . The image processing method according to  claim 8 , further comprising calculating the blur information for each color of a plurality of colors using the at least two pixel values that have been subjected to the exposure compensation and outputting a maximum value of the blur information for each color at the corresponding pixel position as the blur information corresponding to a pixel located at the corresponding pixel position. 
     
     
         12 . The image processing method according to  claim 8 , further comprising estimating noise included in the at least two pixel values and calculating the blur information corresponding to a pixel located at the corresponding pixel position by subtracting the estimated noise from a value calculated based on a difference between the at least two pixel values after compensating. 
     
     
         13 . The image processing method according to  claim 8 , further comprising for a pixel value of the at least two pixel values that corresponds to a long exposure time and is at a saturation level, calculating a blending coefficient that suppresses the blending ratio of the pixel value that corresponds to the long exposure time. 
     
     
         14 . The image processing method according to  claim 8 , wherein performing the pixel value blending process on the at least two pixel values at the corresponding pixel position comprises performing the pixel value blending process on the at least two pixel values at the corresponding pixel position to generate a high dynamic range image as an output image. 
     
     
         15 . A memory having instructions that, when executed, cause performance of a method, the method comprising:
 compensating for exposure of at least two pixel values having different exposure times respectively;   after compensating, calculating blur information based on the at least two pixel values, wherein the blur information is an index value showing a degree of discrepancy between the at least two pixel values;   deciding a blending coefficient, the blending coefficient representing a blending ratio, based on the blur information; and   performing a pixel value blending process on the at least two pixel values at a corresponding pixel position.   
     
     
         16 . The memory according to  claim 15 , wherein the method further comprises:
 receiving one image containing at least some pixel values corresponding to long exposure pixels and at least some pixel values corresponding to short exposure pixels; and   performing an interpolation process on the one image to generate a long exposure image, in which all pixels of the long exposure image are set at pixel values of long exposure pixels of an image pickup element, and a short exposure image, in which all pixels are set at pixel values of short exposure pixels of the image pickup element, wherein:
 compensating for exposure of the at least two pixel values having the different exposure times respectively comprises compensating for exposure of the at least two pixel values having the different exposure times respectively in keeping with an exposure ratio between the long exposure pixels and the short exposure pixels on the long exposure image and the short exposure image, and 
 calculating the blur information comprises using the compensated long exposure image and the compensated short exposure image. 
   
     
     
         17 . The memory according to  claim 16 , wherein a first pixel value of the at least two pixel values corresponds to the long exposure image and a second pixel value of the at least two pixel values corresponds to the short exposure image. 
     
     
         18 . The memory according to  claim 15 , wherein the method further comprises calculating the blur information for each color of a plurality of colors using the at least two pixel values that have been subjected to the exposure compensation and outputting a maximum value of the blur information for each color at the corresponding pixel position as the blur information corresponding to a pixel located at the corresponding pixel position. 
     
     
         19 . The memory according to  claim 15 , wherein the method further comprises estimating noise included in the at least two pixel values and calculating the blur information corresponding to a pixel located at the corresponding pixel position by subtracting the estimated noise from a value calculated based on a difference between the at least two pixel values after compensating. 
     
     
         20 . The memory according to  claim 15 , wherein the method further comprises for a pixel value of the at least two pixel values that corresponds to a long exposure time and is at a saturation level, calculating a blending coefficient that suppresses the blending ratio of the pixel value that corresponds to the long exposure time. 
     
     
         21 . The memory according to  claim 15 , wherein performing the pixel value blending process on the at least two pixel values at the corresponding pixel position comprises performing the pixel value blending process on the at least two pixel values at the corresponding pixel position to generate a high dynamic range image as an output image.