Image processing method, and image processing device

Abstract

The image processing device includes: a specular reflection detector ( 1110 ) configured to detect, in a first image, an enlarged region including a saturated region and an adjacent region, the saturated region being a pixel region having a pixel value equal to or larger than a predetermined threshold, and the adjacent region at least partially surrounding the saturated region; and a replacement unit ( 1130 ) configured to replace a pixel value of a pixel included at least in the adjacent region, with one of (i) a pixel value of a pixel in a region of a second image that is located at a position corresponding to a position of the enlarged region in the first image and (ii) a pixel value of a pixel adjacent to the pixel in the second image, the adjacent region being included in the enlarged region detected by the specular reflection detector ( 1110 ).

1 . An image processing method, comprising:
 detecting, in a first image, an enlarged region including a saturated region and an adjacent region, the saturated region being a pixel region having a pixel value equal to or larger than a predetermined threshold, and the adjacent region at least partially surrounding the saturated region; and   replacing a pixel value of a pixel included at least in the adjacent region, with one of (i) a pixel value of a pixel in a region of a second image that is located at a position corresponding to a position of the enlarged region in the first image and (ii) a pixel value of a pixel adjacent to the pixel in the second image, the adjacent region being included in the enlarged region detected in said detecting.   
     
     
         2 . The image processing method according to  claim 1 , further comprising
 estimating the corresponding position of the region in the second image by searching the second image for an image region that matches a surrounding region that at least partially surrounds the enlarged region detected in said detecting,   wherein in said replacing, the pixel value of the pixel included at least in the adjacent region is replaced with a pixel value of a pixel at the corresponding position of the region in the second image estimated in said estimating.   
     
     
         3 . The image processing method according to  claim 2 ,
 wherein said estimating includes:   determining the surrounding region that at least partially surrounds the enlarged region detected in said detecting;   searching, based on predetermined criteria, the second image for an appropriate image region to be replaced with the enlarged region in the first image, the image region matching the surrounding region and satisfying the predetermined criteria; and   selecting a position that best matches a predetermined measure from the image region obtained in said searching, as the corresponding position of the region in the second image.   
     
     
         4 . The image processing method according to  claim 3 ,
 wherein the predetermined criteria include at least one of an amount of saturated pixels in the image region, mean brightness of the image region, an amount of spatial information of the image region, frequency information of the image region, an amount of structure of the image region, an amount of edges in the image region, an amount of texture of the image region, an amount of noise of the image region, smoothness of a transition region between the image region and the determined surrounding region, a temporal distance between the first image and the second image, and a spatial distance between the enlarged region in the first image and the appropriate image region in the second image to be replaced with the enlarged region in the first image, the image region being obtained in said searching.   
     
     
         5 . The image processing method according to  claim 2 ,
 wherein said estimating is performed using, as the second image, one of images of a video sequence or a plurality of images including images captured by a plurality of cameras.   
     
     
         6 . The image processing method according to  claim 2 ,
 wherein further in said replacing, a pixel value of a pixel included in a transition region between the enlarged region and the surrounding region in the first image is smoothed, the enlarged region and the surrounding region being obtained in said replacing.   
     
     
         7 . The image processing method according to  claim 1 ,
 wherein in said detecting, a pixel region with saturated pixel values is determined as the saturated region in the first image.   
     
     
         8 . The image processing method according to  claim 1 ,
 wherein said detecting includes:   classifying, in the first image, pixels each having the pixel value equal to or larger than the predetermined threshold, as saturated pixels; and   identifying the saturated region by grouping the pixels classified as the saturated pixels in said classifying, the pixels being adjacent to each other.   
     
     
         9 . The image processing method according to  claim 1 ,
 wherein the adjacent region is a false color region that is a region in which a false color appears around the saturated region.   
     
     
         10 . The image processing method according to  claim 9 ,
 wherein in said replacing, a pixel value of a pixel in the saturated region is replaced with a pixel value of a pixel in a region of the second image that is located at a position corresponding to a position of the saturated region in the first image, and a pixel value of a blue component of a pixel in the false color region is replaced with a pixel value of a blue component of a pixel in a region of the second image that is located at a position corresponding to a position of the false color region in the first image, the saturated region and the false color region being included in the enlarged region detected in said detecting.   
     
     
         11 . The image processing method according to  claim 9 ,
 wherein in said replacing, a pixel value of a blue component of a pixel in the false color region is replaced with a pixel value of a blue component of a pixel in a region of the second image that is located at a position corresponding to a position of the false color region in the first image, and after the replacing, a pixel value of a pixel in the saturated region is replaced with a pixel value of a pixel in a region of the second image that is located at a position corresponding to a position of the saturated region in the first image, the false color region and the saturated region being included in the enlarged region detected in said detecting, and   further in said detecting, the enlarged region including the saturated region and the adjacent region is detected in the first image after replacing the pixel value of the blue component of the pixel in the false color region and before replacing the pixel value of the pixel in the saturated region in said replacing, the saturated region being the pixel region having the pixel value equal to or larger than the predetermined threshold, and the adjacent region at least partially surrounding the saturated region.   
     
     
         12 . The image processing method according to  claim 1 ,
 wherein the adjacent region is a false color region that is a region in which a false color appears around the saturated region,   the first image is identical to the second image, and   in said replacing, a pixel value of a blue component of a pixel in the adjacent region included in the enlarged region is replaced by performing interpolation using a pixel value of a blue component outside of a region in the second image that corresponds to the adjacent region in the first image, the enlarged region being detected in said detecting.   
     
     
         13 . The image processing method according to  claim 12 , further comprising
 estimating a position of a region in a third image that is located at a position corresponding to the position of the enlarged region in the first image by searching the third image for an image region that matches a surrounding region that at least partially surrounds the enlarged region detected in said detecting,   wherein in said replacing, a pixel value of a pixel included in the saturated region is replaced with a pixel value of a pixel in the region of the third image that is located at the corresponding position estimated in said estimating, the saturated region being included in the enlarged region detected in said detecting.   
     
     
         14 . The image processing method according to  claim 9 ,
 wherein further in said detecting, when a pixel value of a blue component in each pixel included in a region around the saturated region is tracked in a direction distant from a boundary of the saturated region and after the pixel value becomes local maximum, the false color region is determined by determining a position at which the pixel value is equal to or smaller than a predetermined value, as a boundary position of the false color region.   
     
     
         15 . An image processing device, comprising:
 a detecting unit configured to detect, in a first image, an enlarged region including a saturated region and an adjacent region, the saturated region being a pixel region having a pixel value equal to or larger than a predetermined threshold, and the adjacent region at least partially surrounding the saturated region; and   a replacement unit configured to replace a pixel value of a pixel included at least in the adjacent region, with one of (i) a pixel value of a pixel in a region of a second image that is located at a position corresponding to a position of the enlarged region in the first image and (ii) a pixel value of a pixel adjacent to the pixel in the second image, the adjacent region being included in the enlarged region detected by said detecting unit.   
     
     
         16 . A non-transitory computer-readable recording medium on which a program causing a computer to execute the image processing method according to  claim 1  is recorded.