Image processing apparatus, image capturing apparatus, information processing apparatus, image processing method, and computer-readable storage medium

Abstract

An acquisition unit acquires light source information specifying a virtual light source, normal line information indicating a normal line distribution in a shooting range, and normal line error information indicating a normal line error region in which an error has occurred in the normal line distribution. A first generation unit generates first gain information indicating a first gain amount distribution, based on the light source information and the normal line information. A second generation unit generates second gain information indicating a second gain amount distribution, by performing smoothing processing on a gain amount of a region, which corresponds to the normal line error region, in the first gain amount distribution. An addition unit adds an effect of the virtual light source to an image of the shooting range based on the second gain amount distribution.

What is claimed is: 
     
       1. An image processing apparatus comprising at least one processor and/or at least one circuit which functions as:
 an acquisition unit configured to acquire light source information specifying a virtual light source, normal line information indicating a normal line distribution in a shooting range, and normal line error information indicating a normal line error region in which an error has occurred in the normal line distribution; 
 a first generation unit configured to generate first gain information indicating a first gain amount distribution, based on the light source information and the normal line information; 
 a second generation unit configured to generate second gain information indicating a second gain amount distribution, by performing smoothing processing on a gain amount of a region, which corresponds to the normal line error region, in the first gain amount distribution; and 
 an addition unit configured to add an effect of the virtual light source to an image of the shooting range, by applying a gain conforming to the second gain amount distribution to the image. 
 
     
     
       2. The image processing apparatus according to  claim 1 ,
 wherein the second generation unit determines, based on a difference in gain amount between positions in the region, which corresponds to the normal line error region, in the first gain amount distribution, a smoothing range of each of the positions in the smoothing processing, and 
 the smoothing range determined for a second position for which the difference in gain amount is a second value is larger than the smoothing range determined for a first position for which the difference in gain amount is a first value smaller than the second value. 
 
     
     
       3. The image processing apparatus according to  claim 1 ,
 wherein the acquisition unit further acquires boundary information indicating a boundary position between subjects in the shooting range, and 
 in a case where the normal line error region includes the boundary position, the second generation unit generates the second gain information indicating the second gain amount distribution, by replacing the gain amount of the region, which corresponds to the normal line error region, in the first gain amount distribution with a gain amount of a position adjacent to the normal line error region, after performing the smoothing processing. 
 
     
     
       4. The image processing apparatus according to  claim 3 ,
 wherein the acquisition unit acquires, as the boundary information, information indicating a change in diffuse reflectance per pixel position, information indicating a change in object distance per pixel position, object detection information, information indicating a change in color per pixel position, or information indicating a change in luminance per pixel position. 
 
     
     
       5. The image processing apparatus according to  claim 1 ,
 wherein the acquisition unit further acquires boundary information indicating a boundary position between subjects in the shooting range, and 
 in a case where the normal line error region includes the boundary position, the second generation unit
 determines whether a distance from the boundary position to a position adjacent to the normal line error region is less than a threshold value, 
 in a case where the distance is less than the threshold value, generates the second gain information indicating the second gain amount distribution, by replacing the gain amount of the region, which corresponds to the normal line error region, in the first gain amount distribution with a gain amount of the position adjacent to the normal line error region, after performing the smoothing processing, and 
 in a case where the distance is not less than the threshold value, generates the second gain information indicating the second gain amount distribution, by replacing a gain amount of a region included in a distance range less than or equal to the threshold value from the boundary position in the first gain amount distribution with a gain amount of a position adjacent to the region included in the distance range, after performing the smoothing processing. 
 
 
     
     
       6. An image capturing apparatus comprising:
 the image processing apparatus according to  claim 1 ; and 
 an image sensor configured to generate the image. 
 
     
     
       7. An information processing apparatus comprising at least one processor and/or at least one circuit which functions as:
 an acquisition unit configured to acquire first normal line information indicating a first normal line distribution which is a normal line distribution in a shooting range and normal line error information indicating a normal line error region in which an error has occurred in the first normal line distribution; and 
 a generation unit configured to generate second normal line information indicating a second normal line distribution, by performing smoothing processing on a normal line of the normal line error region in the first normal line distribution. 
 
     
     
       8. An image processing method executed by an image processing apparatus, comprising:
 acquiring light source information specifying a virtual light source, normal line information indicating a normal line distribution in a shooting range, and normal line error information indicating a normal line error region in which an error has occurred in the normal line distribution; 
 generating first gain information indicating a first gain amount distribution, based on the light source information and the normal line information; 
 generating second gain information indicating a second gain amount distribution, by performing smoothing processing on a gain amount of a region, which corresponds to the normal line error region, in the first gain amount distribution; and 
 adding an effect of the virtual light source to an image of the shooting range, by applying a gain conforming to the second gain amount distribution to the image. 
 
     
     
       9. A non-transitory computer-readable storage medium which stores a program for causing a computer to execute an image processing method comprising:
 acquiring light source information specifying a virtual light source, normal line information indicating a normal line distribution in a shooting range, and normal line error information indicating a normal line error region in which an error has occurred in the normal line distribution; 
 generating first gain information indicating a first gain amount distribution, based on the light source information and the normal line information; 
 generating second gain information indicating a second gain amount distribution, by performing smoothing processing on a gain amount of a region, which corresponds to the normal line error region, in the first gain amount distribution; and 
 adding an effect of the virtual light source to an image of the shooting range, by applying a gain conforming to the second gain amount distribution to the image.