Method and/or architecture for motion estimation using integrated information from camera isp

Abstract

A camera comprising a first circuit and a second circuit. The first circuit may be configured to perform image signal processing using encoding related information. The second circuit may be configured to encode image data using image signal processing related information. The first circuit may be further configured to pass the image signal processing related information to the second circuit. The second circuit may be further configured to pass the encoding related information to the first circuit. The second circuit may be further configured to modify one or more motion estimation processes based upon the information from the first circuit.

1 . A camera comprising:
 a first circuit configured to receive an input signal and perform image signal processing using encoding related information and at least one limitation of the camera; and   a second circuit configured to encode image data using image signal processing related information, wherein (i) said first circuit is further configured to pass said image signal processing related information to said second circuit and (ii) said second circuit is further configured to pass said encoding related information to said first circuit and modify one or more motion estimation processes based upon the information from said first circuit.   
     
     
         2 . The camera according to  claim 1 , wherein said first and said second circuits are integrated in a single integrated circuit. 
     
     
         3 . The camera according to  claim 1 , wherein said second circuit modifies said one or more motion estimation processes to favor small motion vectors as long as distortion is below a predetermined threshold when a zoom process is active. 
     
     
         4 . The camera according to  claim 1 , wherein said second circuit modifies said one or more motion estimation processes to favor co-located motion prediction when a zoom process is active. 
     
     
         5 . The camera according to  claim 1 , wherein said second circuit modifies said one or more motion estimation processes to favor small blocks in areas containing edges for medium zoom speeds. 
     
     
         6 . The camera according to  claim 1 , wherein said second circuit modifies said one or more motion estimation processes to favor intra coding modes to reduce distortion for zoom speeds ranging from medium to high. 
     
     
         7 . The camera according to  claim 1 , wherein said second circuit modifies said one or more motion estimation processes to enable reference picture resampling with a direction of the resampling being determined by a direction of zoom. 
     
     
         8 . The camera according to  claim 1 , wherein said second circuit modifies said one or more motion estimation processes to enable reference picture resampling with a resampling ratio being determined by a zoom ratio. 
     
     
         9 . The camera according to  claim 1 , wherein said second circuit modifies said one or more motion estimation processes to favor small motion vectors and intra blocks at edges of a picture while zooming out. 
     
     
         10 . The camera according to  claim 1 , wherein said second circuit modifies said one or more motion estimation processes to favor multiple reference pictures when zoom is varied between zoom-in and zoom-out. 
     
     
         11 . The camera according to  claim 1 , wherein said second circuit modifies said one or more motion estimation processes to favor co-located motion prediction and small motion vectors when an automatic exposure process is active. 
     
     
         12 . The camera according to  claim 1 , wherein said second circuit modifies said one or more motion estimation processes to favor a weighted prediction mode when a substantially linear luminance change is detected. 
     
     
         13 . The camera according to  claim 1 , wherein said second circuit modifies said one or more motion estimation processes to control weighted prediction factors based on exposure times when an auto-exposure process is active. 
     
     
         14 . The camera according to  claim 13 , wherein weighted prediction factors comprise one or more factors selected from the group consisting of offset factors and multiplicative factors. 
     
     
         15 . The camera according to  claim 1 , wherein said second circuit modifies said one or more motion estimation processes such that when small global motion changes are reported in coding statistics the efficiency of Long Term Picture coding is evaluated and pictures designated as Long Term are used for prediction instead of a current picture when global changes between pictures are less than a predetermined percentage of total brightness. 
     
     
         16 . The camera according to  claim 1 , wherein said second circuit modifies said one or more motion estimation processes such that when small global motion changes are reported in coding statistics and automatic exposure is active either a current picture is skipped or skip mode is favored. 
     
     
         17 . The camera according to  claim 1 , wherein said second circuit modifies said one or more motion estimation processes such that small motion co-located motion vectors are emphasized when an automatic focus process is active. 
     
     
         18 . The camera according to  claim 1 , wherein said second circuit modifies said one or more motion estimation processes to favor Skip or Long Term Picture prediction modes when global motion is low and an automatic focus process is in a hunting mode. 
     
     
         19 . The camera according to  claim 1 , wherein said second circuit modifies said one or more motion estimation processes to favor multiple reference pictures using small search ranges when an automatic focus process of said first circuit is hunting for optimal focus. 
     
     
         20 . The camera according to  claim 1 , wherein said second circuit modifies said one or more motion estimation processes such that (i) when a trend in average picture brightness from one picture to another picture exceeds a predetermined threshold the motion estimation processes operate as though automatic exposure is active and (ii) when a sudden increase in brightness or over-exposure is detected the motion estimation processes use either a Long Term Picture prediction mode, zero motion vectors or insertion of an intra frame. 
     
     
         21 . A camera comprising:
 means for receiving an input signal and performing image signal processing using encoding related information and at least one limitation of the camera; and   means for performing image encoding using image signal processing related information and said camera settings information, wherein (i) the image signal processing means is further configured to pass the image signal processing related information to the image encoding means and (ii) the image encoding means is further configured to pass the encoding related information to the image signal processing means and modify one or more motion estimation processes based upon the information from the image signal processing means.   
     
     
         22 . A method for control of motion estimation comprising the steps of:
 receiving an input signal;   performing an image signal processing operation on the input signal using encoding related information and at least one limitation of the camera; and   performing an image encoding operation using image signal processing related information and said camera settings information, wherein (i) the image signal processing operation is further configured to pass the image signal processing related information to the image encoding operation and (ii) the image encoding operation is further configured to pass the encoding related information to the image signal processing operation and modify one or more motion estimation processes based upon the image signal processing related information.   
     
     
         23 . The method according to  claim 20 , wherein:
 when zoom is active said one or more motion estimation processes are modified to favor one or more criteria selected from the group consisting of (i) small motion vectors with distortion below a predetermined threshold, (ii) co-located motion vectors, (iii) small blocks in areas containing edges for medium zoom speeds, (iv) intra coding modes to reduce distortion for zoom speeds ranging from medium to high, (v) reference picture resampling with a resampling direction determined by a direction of zoom, (vi) reference picture resampling with a resampling ratio determined by a ratio of zoom, (vii) small motion vectors and intra blocks at edges of a picture while zooming out and (viii) multiple reference pictures when zoom is varied between zooming-in and zooming-out;   when automatic exposure is active said one or more motion estimation processes are modified to favor one or more criteria selected from the group consisting of (i) co-located motion prediction and small motion vectors, (ii) weighted prediction in response to detection of a substantially linear luminance change, (iii) controlling weighted prediction factors based on exposure time, (iv) controlling offset and multiplicative factors based on exposure time, (v) skipping a current picture when small global motion changes are reported in coding statistics, (vi) favoring skip mode when small global motion changes are reported in coding statistics and (vii) using a Long Term Picture for prediction instead of a current picture when global changes between pictures are less than a predetermined percentage of total brightness; and   when automatic focus is active said one or more motion estimation processes are modified to favor one or more criteria selected from the group consisting of (i) small motion co-located motion vectors, (ii) Skip or Long Term Picture prediction when global motion is low and the automatic focus is in a hunting mode and (iii) multiple reference pictures using small search ranges when the automatic focus is hunting for optimal focus.