Image processing apparatus, image pickup apparatus, image pickup system, image processing method, and non-transitory computer-readable storage medium

Abstract

An image processing apparatus includes a calculator configured to calculate coordinate conversion information for associating a position of a first image with a position of a second image, a region setting portion configured to set a first region in the first image and set a second region associated with the first region in the second image based on the coordinate conversion information, and an evaluation value obtaining portion configured to compare the first region with the second region to obtain an evaluation value.

What is claimed is: 
     
       1. An image processing apparatus, which generates a background-blurred image from two images captured by changing an in-focus position, comprising:
 a calculator configured to calculate coordinate conversion information for associating a position of a first image with a position of a second image; 
 a region setting portion configured to set a first region in the first image and set a second region associated with the first region in the second image based on the coordinate conversion information; 
 an evaluation value obtaining portion configured to compare the first region with the second region to obtain an evaluation value, and to determine whether the first region in the first image is a foreground region or a background region based on the evaluation value; and 
 an image generating portion configured to combine the first image and a blurred image, which is obtained by blurring the first image, based on the evaluation value and to generate the background-blurred image, wherein the image generating portion is configured to use the first image for the foreground region and use the blurred image for the background region. 
 
     
     
       2. The image processing apparatus according to  claim 1 , wherein the calculator calculates the coordinate conversion information based on a motion vector between the first image and the second image. 
     
     
       3. The image processing apparatus according to  claim 1 , wherein the coordinate conversion information is a projection transform coefficient indicating a deformation of an object image. 
     
     
       4. The image processing apparatus according to  claim 1 , wherein the evaluation value obtaining portion obtains the evaluation value without interpolating pixels included in the first region and the second region. 
     
     
       5. The image processing apparatus according to  claim 1 , wherein the evaluation value obtaining portion normalizes the evaluation value in accordance with sizes of the first region and the second region. 
     
     
       6. The image processing apparatus according to  claim 1 , wherein the evaluation value obtaining portion changes a weight for each pixel included in the second region to obtain the evaluation value. 
     
     
       7. The image processing apparatus according to  claim 1 , wherein the evaluation value obtaining portion compares edge integral values of the first region and the second region with each other to obtain the evaluation value. 
     
     
       8. The image processing apparatus according to  claim 1 , wherein the evaluation value obtaining portion compares differences between signal values of the first region and the second region to obtain the evaluation value. 
     
     
       9. The image processing apparatus according to  claim 1 , wherein the region setting portion sets a region, which has each vertex at points obtained using the coordinate conversion information with respect to each vertex of the first region, as the second region. 
     
     
       10. The image processing apparatus according to  claim 1 , wherein the region setting portion sets a region, which includes a pixel obtained using the coordinate conversion information with respect to a pixel included in the first region, as the second region. 
     
     
       11. The image processing apparatus according to  claim 1 , wherein the evaluation value is used to obtain distance information. 
     
     
       12. The image processing apparatus according to  claim 1 , wherein the evaluation value is used to determine a moving object region. 
     
     
       13. An image pickup apparatus comprising:
 an image pickup element configured to perform photoelectric conversion of an object image to obtain a first image and a second image; and 
 an image processing apparatus according to  claim 1 . 
 
     
     
       14. An image pickup system comprising:
 an image pickup optical system; and 
 an image pickup apparatus according to  claim 13 , configured to obtain the object image via the image pickup optical system. 
 
     
     
       15. An image processing method for generating a background-blurred image from two images captured by changing an in-focus position comprising the steps of:
 calculating coordinate conversion information for associating a position of a first image with a position of a second image; 
 setting a first region in the first image and setting a second region associated with the first region in the second image based on the coordinate conversion information; 
 comparing the first region with the second region to obtain an evaluation value; 
 determining whether the first region in the first image is a foreground region or a background region based on the evaluation value; and 
 combining the first image and a blurred image, which is obtained by blurring the first image, based on the evaluation value and generating the background-blurred image, wherein the first image is used for the foreground region and the blurred image is used for the background image. 
 
     
     
       16. A non-transitory computer-readable storage medium which stores an image processing program for causing an image processing apparatus, which generates a background-blurred image from two images captured by changing an in-focus position, to execute the steps of:
 calculating coordinate conversion information for associating a position of a first image with a position of a second image; 
 setting a first region in the first image and setting a second region associated with the first region in the second image based on the coordinate conversion information; 
 comparing the first region with the second region to obtain an evaluation value; 
 determining whether the first region in the first image is a foreground region or a background region based on the evaluation value; and 
 combining the first image and a blurred image, which is obtained by blurring the first image, based on the evaluation value and generating the background-blurred image, wherein the first image is used for the foreground region and the blurred image is used for the background region.