Image processing apparatus, image display apparatus and imaging apparatus having the same, image processing method, and computer-readable medium storing image processing program

Abstract

An image processing apparatus receives an image signal and generates a display image having an image range associated with a display area of a display unit from an image based on the image signal. The image processing apparatus includes a position-of-interest-calculating-unit, a frame-out-accuracy-calculation-unit, an alteration-variable-decision-unit and an image-alteration-unit. The position-of-interest-calculating-unit calculates a position of interest as a position of an object of interest in the image. The frame-out-accuracy-calculation-unit calculates a frame-out accuracy representing an accuracy that the position of interest deviates from the image range based on the position of interest and the image range. The alteration-variable-decision-unit decides a processing variable of alteration processing performed with respect to the image in conformity with the frame-out accuracy. The image-alteration-unit performs the alteration processing with respect to the image in conformity with the decided processing variable to generate a signal of the display image.

1 . An image processing apparatus which receives an image signal and generates a display image having an image range associated with a display area of a display unit from an image based on the image signal, the apparatus comprising:
 a position-of-interest calculating unit which calculates a position of interest as a position of an object of interest in the image based on the image signal;   a frame-out accuracy calculation unit which calculates a frame-out accuracy representing an accuracy that the position of interest deviates from the image range based on the position of interest and the image range;   an alteration variable decision unit which decides a processing variable of alteration processing performed with respect to the image based on the image signal in conformity with the frame-out accuracy; and   an image alteration unit which performs the alteration processing with respect to the image based on the image signal in conformity with the decided processing variable to generate a signal of the display image.   
     
     
         2 . The apparatus according to  claim 1 ,
 wherein the alteration variable decision unit includes a change processing decision unit which decides a change region as a region where change processing is carried out with respect to the image based on the image signal in conformity with the frame-out accuracy and a level of change of the change processing, and   the image alteration unit performs the change processing with the decided level of the change with respect to the decided change region of the image based on the image signal.   
     
     
         3 . The apparatus according to  claim 2 ,
 wherein the change region is a display image region concerning a position opposing the position of interest with respect to a center of the image range.   
     
     
         4 . The apparatus according to  claim 2 ,
 wherein the change region is a display image region concerning a position apart from the position of interest by a predetermined distance.   
     
     
         5 . The apparatus according to  claim 2 ,
 wherein the change processing decision unit changes an area of the change region in accordance with the frame-out accuracy.   
     
     
         6 . The apparatus according to  claim 5 ,
 wherein a type of the change processing is at least one of filling an image with color, increase/decrease in resolution, increase/decrease in sharpness, increase/decrease in contrast, increase/decrease in luminance, increase/decrease in chroma, and a change in a hue.   
     
     
         7 . The apparatus according to  claim 2 ,
 wherein a type of the change processing is at least one of increase/decrease in resolution, increase/decrease in sharpness, increase/decrease in contrast, increase/decrease in luminance, increase/decrease in chroma, and a change in a hue, and   the change processing decision unit changes an amount of the increase/decrease or an amount of the change as the level of the change in accordance with the frame-out accuracy.   
     
     
         8 . The apparatus according to  claim 1 ,
 wherein the alteration variable decision unit includes a processing decision unit which sets a change region around a representative point of the object of interest in the image based on the image signal in conformity with the frame-out accuracy and decides processing for an image performed with respect to the change region, and   the image alteration unit performs the processing with respect to the change region of the image based on the image signal.   
     
     
         9 . The apparatus according to  claim 8 ,
 wherein the position-of-interest calculation unit includes a contour extraction unit which extracts a contour of the object of interest, and   the processing decision unit decides the change region based on the contour.   
     
     
         10 . The apparatus according to  claim 8 ,
 wherein the processing decision unit changes an area of the change region and/or a level of the processing in accordance with the frame-out accuracy.   
     
     
         11 . The apparatus according to  claim 8 ,
 wherein the processing decision unit changes an area of a portion surrounded by the change region in the image based on the image signal in conformity with the frame-out accuracy.   
     
     
         12 . The apparatus according to  claim 8 ,
 wherein the change region and/or the processing changes with time.   
     
     
         13 . The apparatus according to  claim 1 ,
 wherein the alteration variable decision unit includes a deformation parameter decision unit which decides a deformation parameter for performing geometric conversion with respect to the image based on the image signal in conformity with the frame-out accuracy, and   the image alteration unit executes deformation processing including the geometric conversion with respect to the image based on the image signal in conformity with the deformation parameter.   
     
     
         14 . The apparatus according to  claim 13 ,
 wherein the geometric conversion includes changing a shape of a frame peripheral of the image in the image range.   
     
     
         15 . The apparatus according to  claim 14 ,
 wherein the geometric conversion is configured to shorten and deform a part of the frame of the image in the image range, and   the deformation parameter decision unit decides a ratio of shortening the part of the frame in accordance with the frame-out accuracy.   
     
     
         16 . The apparatus according to  claim 13 ,
 wherein the geometric conversion is conversion including shrinking a part of the image in the image range and expanding the other part of the image.   
     
     
         17 . The apparatus according to  claim 16 ,
 wherein the deformation parameter decision unit decides a ratio of effecting the shrinking and the expanding in accordance with the frame-out accuracy.   
     
     
         18 . The apparatus according to  claim 1 ,
 wherein the frame-out accuracy calculation unit calculates the frame-out accuracy in accordance with a positional relationship of the position of interest with respect to the image range.   
     
     
         19 . The apparatus according to  claim 1 ,
 wherein the frame-out accuracy calculation unit calculates the frame-out accuracy in accordance with a change in a positional relationship of the position of interest with respect to the image range.   
     
     
         20 . The apparatus according to  claim 1 ,
 wherein the frame-out accuracy calculation unit calculates the frame-out accuracy in accordance with a positional relationship of the position of interest with respect to the image range and a change in the positional relationship of the position of interest with respect to the image range.   
     
     
         21 . The apparatus according to  claim 2 ,
 wherein the frame-out accuracy calculation unit executes:   (i) calculating a first frame-out accuracy in accordance with a positional relationship of the position of interest with respect to the image range; and   (ii) calculating a second frame-out accuracy in accordance with a change in the positional relationship of the position of interest with respect to the image range,   the change processing decision unit executes:   (i) deciding in accordance with the first frame-out accuracy a first change region which is a region where the image based on the image signal is subjected to first change processing and whose area differs depending on the first frame-out accuracy;   (ii) deciding in accordance with the second frame-out accuracy a second change region which is a region where the image based on the image signal is subjected to second change processing and whose area differs depending on the second frame-out accuracy;   (iii) deciding the level of the change of the first change processing which is at least one of filling an image with a color, increase/decrease in resolution, increase/decrease in sharpness, increase/decrease in contrast, increase/decrease in luminance, increase/decrease in chroma, and a change in a hue; and   (iv) deciding the level of the change of the second change processing which is at least one of the filling an image with a color, the increase/decrease in resolution, the increase/decrease in sharpness, the increase/decrease in contrast, the increase/decrease in luminance, the increase/decrease in chroma, and a change in a hue, and   the image alteration unit executes:   (i) performing the first change processing having the decided level of the change with respect to the decided first change-region; and   (ii) performing the second change processing having the decided level of the change with respect to the decided second change region.   
     
     
         22 . The apparatus according to  claim 2 ,
 wherein the frame-out accuracy calculation unit executes:   (i) calculating a first frame-out accuracy in accordance with a positional relationship of the position of interest with respect to the image range; and   (ii) calculating a second frame-out accuracy in accordance with a change in the positional relationship of the position of interest with respect to the image range,   the change processing decision unit executes:   (i) deciding in accordance with the first frame-out accuracy the level of the change, which differs depending on the first frame-out accuracy, of first change processing which is at least one of increase/decrease in resolution, increase/decrease in sharpness, increase/decrease in contrast, increase/decrease in luminance, increase/decrease in chroma, and a change in a hue;   (ii) deciding in accordance with the second frame-out accuracy the level of the change, which differs depending on the second frame-out accuracy, of second change processing which is at least one of the increase/decrease in resolution, the increase/decrease in sharpness, the increase/decrease in contrast, the increase/decrease in luminance, the increase/decrease in chroma, and the change in a hue;   (iii) deciding a first change region where the first change processing is performed with respect to the image based on the image signal; and   (iv) deciding a second change region where the second change processing is performed with respect to the image based on the image signal, and   the image alteration unit executes:   (i) performing the first change processing having the decided level of the change with respect to the decided first change region; and   (ii) performing the second change processing having the decided level of the change with respect to the decided second change region.   
     
     
         23 . The apparatus according to  claim 2 ,
 wherein the frame-out accuracy calculation unit executes:   (i) calculating a first frame-out accuracy in accordance with one of a positional relationship of the position of interest with respect to the image range and a change in the positional relationship of the position of interest with respect to the image range; and   (ii) calculating a second frame-out accuracy in accordance with the other of the positional relationship of the position of interest with respect to the image range and the change in the positional relationship of the position of interest with respect to the image range,   the change processing decision unit executes:   (i) deciding in accordance with one of the first frame-out accuracy and the second frame-out accuracy a first change region which is a region where the image based on the image signal is subjected to first change processing and whose area differs depending on one of the first frame-out accuracy and the second frame-out accuracy;   (ii) deciding the level of the change of the first change processing which is at least one of filling an image with a color, increase/decrease in resolution, increase/decrease in sharpness, increase/decrease in contrast, increase/decrease in luminance, increase/decrease in chroma, and a change in a hue;   (iii) deciding the level of the change, which differs depending on the other of the first frame-out accuracy and the second frame-out accuracy, of the second change processing which is at least one of the increase/decrease in resolution, the increase/decrease in sharpness, the increase/decrease in contrast, the increase/decrease in luminance, the increase/decrease in chroma, and the change in a hue; and   (iv) deciding a second change region which is a region where the image based on the image signal is subjected to the second change processing;   the image alteration unit executes:   (i) performing the first change processing having the decided level of the change with respect to the decided first change region; and   (ii) performing the second change processing having the decided level of the change with respect to the decided second change region.   
     
     
         24 . The apparatus according to  claim 8 ,
 wherein the frame-out accuracy calculation unit executes:   (i) calculating a first frame-out accuracy in accordance with one of a positional relationship of the position of interest with respect to the image range and a change in the positional relationship of the position of interest with respect to the image range; and   (ii) calculating a second frame-out accuracy in accordance with the other of the positional relationship of the position of interest with respect to the image range and the change in the positional relationship of the position of interest with respect to the image range,   the processing decision unit executes:   (i) setting a first change region around a representative point of the object of interest in the image based on the image signal in conformity with the first frame-out accuracy and decides first processing for an image performed with respect to the first change region; and   (ii) setting a second change region around a representative point of the object of interest in the image based on the image signal in conformity with the second frame-out accuracy and decides second processing for an image performed with respect to the second change region, and   the image alteration unit executes:   (i) performing the first processing with respect to the first change region of the image based on the image signal; and   (ii) performing the second processing with respect to the second change region of the image based on the image signal.   
     
     
         25 . The apparatus according to  claim 8 ,
 wherein the frame-out accuracy calculation unit executes:   (i) calculating a first frame-out accuracy in accordance with one of a positional relationship of the position of interest with respect to the image range and a change in the positional relationship of the position of interest with respect to the image range; and   (ii) calculating a second frame-out accuracy in accordance with the other of the positional relationship of the position of interest with respect to the image range and the change in the positional relationship of the position of interest with respect to the image range,   the processing decision unit executes:   (i) deciding the change region to change an area of the change region and/or a level of the processing in accordance with the first frame-out accuracy, and to change an area of a portion surrounded by the change region in accordance with the second frame-out accuracy; and   (ii) deciding the processing for the processing region based on the first frame-out accuracy and the second frame-out accuracy; and   the image alteration unit performs the processing with respect to the change region of the image based on the image signal.   
     
     
         26 . The apparatus according to  claim 8 ,
 wherein the frame-out accuracy calculation unit calculates the frame-out accuracy in accordance with a positional relationship of the position of interest with respect to the image range, and   the processing decision unit sets a region indicative of the positional relationship between the position of interest and the image range as the change region.   
     
     
         27 . The apparatus according to  claim 8 ,
 wherein the frame-out accuracy calculation unit calculates the frame-out accuracy in accordance with a change in a positional relationship of the position of interest with respect to the image range, and   the processing decision unit sets a region indicative of a direction of the change in the positional relationship between the position of interest and the image range as the change region.   
     
     
         28 . The apparatus according to  claim 13 ,
 wherein the frame-out accuracy calculation unit executes:   (i) calculating a first frame-out accuracy in accordance with one of a positional relationship of the position of interest with respect to the image range and a change in the positional relationship of the position of interest with respect to the image range; and   (ii) calculating a second frame-out accuracy in accordance with the other of the positional relationship of the position of interest with respect to the image range and the change in the positional relationship of the position of interest with respect to the image range,   the deformation parameter decision unit executes:   (i) deciding a first parameter for the geometric conversion including changing a shape of a frame peripheral of the image based on the image signal with a direction and a ratio associated with the first frame-out accuracy; and   (ii) deciding a second parameter for the geometric conversion including shrinking a part of the image based on the image signal and expanding the other part of the image with a direction and a ratio associated with the second frame-out accuracy, and   the image alteration unit performs the deformation processing including the geometric conversion with respect to the image based on the image signal in conformity with the first deformation parameter and the second deformation parameter.   
     
     
         29 . The apparatus according to  claim 1 ,
 wherein the alteration variable decision unit includes a change processing decision unit and a processing decision unit,   the frame-out accuracy calculation unit executes:   (i) calculating a first frame-out accuracy in accordance with one of a positional relationship of the position of interest with respect to the image range and a change in the positional relationship of the position of interest with respect to the image range; and   (ii) calculating a second frame-out accuracy in accordance with the other of the positional relationship of the position of interest with respect to the image range and the change in the positional relationship of the position of interest with respect to the image range,   the change processing decision unit decides a first change region in the image based on the image signal and change processing performed with respect to the first change region based on the first frame-out accuracy,   the processing decision unit decides a second change region around a representative point of the object of interest in the image based on the image signal and processing for an image performed with respect to the second image region based on the second frame-out accuracy, and   the image alteration unit executes the change processing with respect to the first change region of the image based on the image signal and performs the processing with respect to the second change region of the image based on the image signal to generate a signal of the display image.   
     
     
         30 . The apparatus according to  claim 29 ,
 wherein the change processing decision unit changes an area of the first change region in accordance with the first frame-out accuracy,   a type of the change processing is at least one of filling an image with color, increase/decrease in resolution, increase/decrease in sharpness, increase/decrease in contrast, increase/decrease in luminance, increase/decrease in chroma, and a change in a hue.   
     
     
         31 . The apparatus according to  claim 29 ,
 wherein a type of the change processing is at least one of increase/decrease in resolution, increase/decrease in sharpness, increase/decrease in contrast, increase/decrease in luminance, increase/decrease in chroma, and a change in a hue, and   the change processing decision unit changes an amount of the increase/decrease or an amount of the change which is the level of the change in accordance with the frame-out accuracy.   
     
     
         32 . The apparatus according to  claim 29 ,
 wherein the processing decision unit changes an area of the second change region and/or a level of the processing in accordance with the second frame-out accuracy.   
     
     
         33 . The apparatus according to  claim 29 ,
 wherein the processing decision unit changes an area of a portion surrounded by the change region in the image in accordance with the image signal in conformity with the second frame-out accuracy.   
     
     
         34 . The apparatus according to  claim 1 ,
 wherein the alteration variable decision unit includes a change processing decision unit and a deformation parameter decision unit,   the frame-out accuracy calculation unit executes:   (i) calculating a first frame-out accuracy in accordance with one of a positional relationship of the position of interest with respect to the image range and a change in the positional relationship of the position of interest with respect to the image range; and   (ii) calculating a second frame-out accuracy in accordance with the other of the positional relationship of the position of interest with respect to the image range and the change in the positional relationship of the position of interest with respect to the image range,   the change processing decision unit decides a first change region in the image based on the image signal and change processing performed with respect to the first change region based on the first frame-out accuracy,   the deformation parameter decision unit decides a deformation parameter for performing geometric conversion with respect to the image based on the image signal in conformity with the second frame-out accuracy, and   the image alteration unit executes the change processing with respect to the first change region of the image based on the image signal and performs the deformation processing with respect to the image based on the image signal in conformity with the deformation parameter to generate a signal of the display image.   
     
     
         35 . The apparatus according to  claim 34 ,
 wherein the change processing decision unit changes an area of the first change region in accordance with the first frame-out accuracy, a type of the change processing is at least one of filling an image with color, increase/decrease in resolution, increase/decrease in sharpness, increase/decrease in contrast, increase/decrease in luminance, increase/decrease in chroma, and a change in a hue.   
     
     
         36 . The apparatus according to  claim 34 ,
 wherein a type of the change processing is at least one of increase/decrease in resolution, increase/decrease in sharpness, increase/decrease in contrast, increase/decrease in luminance, increase/decrease in chroma, and a change in a hue, and   the change processing decision unit changes an amount of the increase/decrease or an amount of the change which is the level of the change in accordance with the frame-out accuracy.   
     
     
         37 . The apparatus according to  claim 34 ,
 wherein the geometric conversion includes deforming a frame peripheral of the image by shortening a part of the frame of the image in the image range, and   the deformation parameter decision unit decides a ratio of shortening the part of the frame in accordance with the second frame-out accuracy.   
     
     
         38 . The apparatus according to  claim 34 ,
 wherein the geometric conversion is conversion including shrinking a part of the image in the image range and expanding the other part of the image, and   the deformation parameter decision unit decides a ratio of effecting the shrinking and the expanding in accordance with the second frame-out accuracy.   
     
     
         39 . The apparatus according to  claim 1 ,
 wherein the alteration variable decision unit includes a processing decision unit and a deformation parameter decision unit,   the frame-out accuracy calculation unit executes:   (i) calculating a first frame-out accuracy in accordance with one of a positional relationship of the position of interest with respect to the image range and a change in the positional relationship of the position of interest with respect to the image range; and   (ii) calculating a second frame-out accuracy in accordance with the other of the positional relationship of the position of interest with respect to the image range and the change in the positional relationship of the position of interest with respect to the image range,   the processing decision unit decides a change region around a representative point of the object of interest in the image based on the image signal and processing for an image performed with respect to the image region based on the first frame-out accuracy,   the deformation parameter decision unit decides a deformation parameter for performing geometric conversion with respect to the image based on the image signal in conformity with the second frame-out accuracy, and   the image alteration unit performs the processing with respect to the change region of the image based on the image signal and executes deformation processing with respect to the image based on the image signal in conformity with the deformation parameter to generate a signal of the display image.   
     
     
         40 . The apparatus according to  claim 39 ,
 wherein the processing decision unit changes an area of the change region and/or a level of the processing in accordance with the first frame-out accuracy.   
     
     
         41 . The apparatus according to  claim 39 ,
 wherein the processing decision unit changes an area of a portion surrounded by the change region in the image based on the image signal in conformity with the first frame-out accuracy.   
     
     
         42 . The apparatus according to  claim 39 ,
 wherein the geometric conversion includes deforming a frame peripheral of the image by shortening a part of the frame of the image in the image range, and   the deformation parameter decision unit decides a ratio of shortening the part of the frame in accordance with the second frame-out accuracy.   
     
     
         43 . The apparatus according to  claim 39 ,
 wherein the geometric conversion is conversion including shrinking a part of the image in the image range and expanding the other part of the image, and   the deformation parameter decision unit decides a ratio of effecting the shrinking and the expanding in accordance with the second frame-out accuracy.   
     
     
         44 . An image display apparatus comprising:
 an image processing apparatus according to  claim 1 ; and   the display unit which displays the display image generated by the image processing apparatus.   
     
     
         45 . An imaging apparatus comprising:
 an image processing apparatus according to  claim 1 ; and   an imaging unit which performs imaging to generate the image signal that is input to the image processing apparatus.   
     
     
         46 . An imaging apparatus comprising:
 an image processing apparatus according to  claim 1 ;   an imaging unit which performs imaging to generate the image signal that is input to the image processing apparatus; and   the display unit which displays the display image generated by the image processing apparatus.   
     
     
         47 . The apparatus according to  claim 45 ,
 wherein the imaging unit has a zoom function.   
     
     
         48 . The apparatus according to  claim 47 , further comprising a control unit configured to judge whether the imaging unit has started to exercise the zoom function and allow the image processing apparatus to execute the alteration processing when the zoom function has started to be exercised. 
     
     
         49 . An image processing method of generating a display image having an image range associated with a display area of a display unit from an image based on an input image signal, the method comprising:
 calculating a position of interest which is a position of an object of interest in the image based on the image signal;   calculating a frame-out accuracy representing an accuracy that the position of interest deviates from the image range based on the position of interest and the image range;   deciding a processing variable of alteration processing performed with respect to the image based on the image signal in conformity with the frame-out accuracy; and   performing the alteration processing with respect to the image based on the image signal in conformity with the decided processing variable to generate a signal of the display image.   
     
     
         50 . The method according to  claim 49 ,
 wherein the processing variable includes a change region which is a region where change processing is performed with respect to the image based on the image signal and a level of change of the change processing, and   the alteration processing includes processing of performing the change processing having the decided level of the change with respect to the decided change region of the image based on the image signal.   
     
     
         51 . The method according to  claim 49 ,
 wherein the processing variable includes a change region around a representative point of the object of interest in the image based on the image signal and a process for an image performed with respect to the change region, and   the alteration processing includes processing of carrying out the process with respect to the change region of the image based on the image signal.   
     
     
         52 . The method according to  claim 49 ,
 wherein the processing variable includes a deformation parameter used for performing geometric conversion with respect to the image based on the image signal, and   the alteration processing includes processing of carrying out deformation processing with respect to the image based on the image signal in conformity with the deformation parameter.   
     
     
         53 . A computer-readable medium storing an image processing program being executed by a computer and configured to generate a display image having an image range associated with a display area of a display unit from an image based on an input image signal, the program controlling the computer to execute functions comprising:
 calculating a position of interest which is a position of an object of interest in the image based on the image signal;   calculating a frame-out accuracy representing an accuracy that the position of interest deviates from the image range based on the position of interest and the image range;   deciding a processing variable of alteration processing performed with respect to the image based on the image signal in conformity with the frame-out accuracy; and   performing the alteration processing with respect to the image based on the image signal in conformity with the decided processing variable to generate a signal of the display image.   
     
     
         54 . The computer-readable medium according to  claim 53 ,
 wherein deciding the processing variable comprises deciding a change region which is a region where change processing is performed with respect to the image based on the image signal and a level of change of the change processing in conformity with the frame-out accuracy, and   generating the signal of the display image comprises performing the change processing having the decided level of the change with respect to the decided change region of the image based on the image signal to generate the signal of the display image.   
     
     
         55 . The computer-readable medium according to  claim 53 ,
 wherein deciding the processing variable comprises setting a change region around a representative point of the object of interest in the image based on the image signal in conformity with the frame-out accuracy and deciding a process for an image performed with respect to the change region, and   generating the signal of the display image comprises performing the process with respect to the change region of the image based on the image signal to generate the signal of the display image.   
     
     
         56 . The computer-readable medium according to  claim 53 ,
 wherein deciding the processing variable comprises deciding a deformation parameter representing information of deformation performed with respect to the image based on the image signal in conformity with the frame-out accuracy, and   generating the signal of the display image comprises performing deformation processing with respect to the image based on the image signal in conformity with the deformation parameter to generate the signal of the display image.