Dynamic pixel-wise multi-gain readout for high dynamic range imaging

Abstract

Systems and methods are disclosed to enable fast readout from an image sensor to support pixel-wise conversion gain selection for high dynamic range imaging. In embodiments, an image sensor integrated circuit performs the pixel-wise gain selection with its readout circuitry, so that the image sensor outputs pixels with only the selected gain option. In this manner, the image sensor is able to achieve faster frame rates and lower power consumption. Depending on the embodiment, the conversion gain may be selected by the readout logic, an image signal processor, or an auto-exposure engine. The gain selection may be made based on a previous image captured by the camera or other pixels in the same image. The image signal processor may interpolate a high-gain and a low-gain portion of the image to obtain full resolution images in the two gain options, and merge the two to obtain the final image.

1 - 20 . (canceled) 
     
     
         21 . An image sensor integrated circuit, comprising:
 a plurality of photodiodes that implement pixels in a pixel array, configured to capture a frame; and   readout logic for a group of pixels configured to:
 receive a gain selection signal indicating respective conversion gains for reading pixel values of the frame from individual ones of the pixels, including a high conversion gain (HCG) and a low conversion gain (LCG); 
 read at least one first pixel value of the frame from one or more pixels in the group of pixels with the HCG as indicated by the gain selection signal; 
 read, after the read of the at least one first pixel value, pixel values of the frame from remaining pixels in the group of pixels with LCG as indicated by the gain selection signal in response to determining that the at least one first pixel value satisfies a threshold; 
 read, after the read of the at least one first pixel value, pixel values of the frame from the remaining pixels in the group of pixels with HCG as indicated by the gain selection signal in response to determining that the at least one first pixel value fails to satisfy the threshold; and 
 output the pixel values of the frame via an output interface of the image sensor integrated circuit. 
   
     
     
         22 . The image sensor integrated circuit of  claim 21 , wherein the gain selection signal is received from another circuit distinct from the image sensor integrated circuit. 
     
     
         23 . The image sensor integrated circuit of  claim 21 , wherein the threshold is based on a saturation limit for the one or more pixels in the group of pixels. 
     
     
         24 . The image sensor integrated circuit of  claim 23 , wherein the at least one first pixel value satisfies the threshold when a specified number of the one or more pixels have a pixel value that exceeds the saturation limit. 
     
     
         25 . The image sensor integrated circuit of  claim 23 , wherein the at least one first pixel value satisfies the threshold when an average saturation limit for the one or more pixels exceeds the saturation limit. 
     
     
         26 . The image sensor integrated circuit of  claim 21 , wherein the group of pixels are positioned in a local region of the pixel array. 
     
     
         27 . The image sensor integrated circuit of  claim 26 , wherein the pixel array includes a second group of pixels positioned in the local region of the pixel array, and wherein second readout logic for the second group of pixels is configured to:
 read pixel values of the frame from pixels in the second group of pixels with LCG as indicated by the gain selection signal in response to determining that the at least one first pixel value satisfies the threshold;   read pixel values of the frame from the pixels in the second group of pixels with HCG as indicated by the gain selection signal in response to determining that the at least one first pixel value fails to satisfy the threshold; and   output the pixel values of the frame via the output interface of the image sensor integrated circuit.   
     
     
         28 . The image sensor integrated circuit of  claim 27 , further comprising a regional gain control circuit coupled to the group of pixels and the second group of pixels in the local region of the pixel array, wherein the regional gain control circuit is configured to:
 determine the gain selection signal based on the read of the at least one first pixel value of the frame from the one or more pixels in the group of pixels; and   provide the gain selection signal to the readout logic and the second readout logic for reading pixel values of the frame from the remaining pixels in the group of pixels and the pixels in the second group of pixels.   
     
     
         29 . The image sensor integrated circuit of  claim 28 , wherein the regional gain control circuit is implemented on an application-specific integrated circuit (ASIC) layer coupled to a pixel layer that implements the pixel array. 
     
     
         30 . The image sensor integrated circuit of  claim 26 , further comprising a color filter array (CFA) configured to filter light received by the pixels for different colors in the pixel array, wherein the CFA uses one color for pixels in the local region of the pixel array. 
     
     
         31 . A camera, comprising:
 one or more lenses configured to receive light;   an image sensor comprising:
 a plurality of photodiodes that implement pixels in a pixel array, configured to capture a frame according to the light; 
 a regional gain control circuit asserting a gain selection signal indicating respective conversion gains for reading pixel values of the frame from individual ones of the pixels in a local region of the pixel array, including a high conversion gain (HCG) and a low conversion gain (LCG); 
 readout logic for a group of pixels configured to:
 receive the gain selection signal from the regional gain control circuit indicating the respective conversion gains for reading the pixel values of the frame from the individual ones of the pixels in the group of pixels; 
 
 wherein the regional gain control circuit is configured to:
 read at least one first pixel value of the frame from one or more pixels in the local region of the pixel array with the HCG as indicated by the asserted gain selection signal; 
 assert, after the read of the at least one first pixel value, LCG for the gain selection signal for readout of pixel values of the frame from remaining pixels in the local region of the pixel array in response to determining that the at least one first pixel value satisfies a threshold; 
 assert, after the read of the at least one first pixel value, HCG for the gain selection signal for readout of pixel values of the frame from the remaining pixels in the local region of the pixel array in response to determining that the at least one first pixel value fails to satisfy the threshold; and 
 output the pixel values of the frame via an output interface of the image sensor. 
 
   
     
     
         32 . The camera of  claim 31 , wherein the regional gain control circuit is implemented on an application-specific integrated circuit (ASIC) layer coupled to a pixel layer that implements the pixel array. 
     
     
         33 . The camera of  claim 31 , wherein the threshold is based on a saturation limit for the one or more pixels in the group of pixels. 
     
     
         34 . The camera of  claim 31 , wherein the regional gain control circuit is configured to assert the gain selection signal to an additional group of pixels in the local region of the pixel array after the read of the at least one first pixel value. 
     
     
         35 . The camera of  claim 31 , further comprising an additional regional gain control circuit asserting an additional gain selection signal indicating respective conversion gains for reading pixel values of the frame from individual ones of the pixels in an additional local region of the pixel array, including the high conversion gain (HCG) and the low conversion gain (LCG), wherein the additional regional gain control circuit asserts its additional gain selection signal independent of the regional gain control circuit's assertion of its gain selection signal. 
     
     
         36 . The camera of  claim 35 , further comprising:
 at least one additional readout logic for at least one additional group of pixels configured to:
 receive the additional gain selection signal from the additional regional gain control circuit indicating the respective conversion gains for reading the pixel values of the frame from the individual ones of the pixels in the at least one additional group of pixels. 
   
     
     
         37 . The camera of  claim 31 , further comprising a color filter array (CFA) configured to filter light received by the pixels for different colors in the pixel array, wherein the CFA uses one color for pixels in the local region of the pixel array. 
     
     
         38 . A camera, comprising:
 one or more lenses configured to receive light;   an image sensor comprising:
 a plurality of photodiodes that implement pixels in a pixel array, configured to capture a frame according to the light; 
 a regional gain control circuit asserting a gain selection signal indicating respective conversion gains for reading pixel values of the frame from individual ones of the pixels in a local region of the pixel array, including a high conversion gain (HCG) and a low conversion gain (LCG); 
 readout logic for a group of pixels configured to:
 receive the gain selection signal from the regional gain control circuit indicating the respective conversion gains for reading the pixel values of the frame from the individual ones of the pixels in the group of pixels; 
 
 wherein the regional gain control circuit is configured to:
 read at least one first pixel value of the frame from one or more pixels in the local region of the pixel array with the HCG as indicated by the asserted gain selection signal; 
 assert, after the read of the at least one first pixel value, LCG or HCG for the gain selection signal for readout of individual pixel values of the frame from remaining pixels in the group of pixels according to a pixel gain configuration pattern, the pixel gain configuration pattern being determined based on the at least one first pixel value; and 
 output the pixel values of the frame via an output interface of the image sensor. 
 
   
     
     
         39 . The camera of  claim 38 , wherein the group of pixels includes a 2×2 group of four pixels, and wherein the pixel gain configuration pattern is one of:
 a high gain only pattern that asserts HCG for all four pixels; 
 a low-light focused pattern that asserts HCG for three of the four pixels; 
 a balanced pattern that asserts HCG for two of the four pixels; 
 a high-light focused pattern that asserts HCG for one of the four pixels; and 
 a low-gain only pattern that asserts LCG for all four pixels. 
 
     
     
         40 . The camera of  claim 38 , wherein the regional gain control circuit is configured to assert the pixel gain configuration pattern to multiple groups of pixels in the local region of the pixel array.