High resolution thin multi-aperture imaging systems

Abstract

A multi-aperture imaging system comprising a first camera with a first sensor that captures a first image and a second camera with a second sensor that captures a second image, the two cameras having either identical or different FOVs. The first sensor may have a standard color filter array (CFA) covering one sensor section and a non-standard color CFA covering another. The second sensor may have either Clear or standard CFA covered sections. Either image may be chosen to be a primary or an auxiliary image, based on a zoom factor. An output image with a point of view determined by the primary image is obtained by registering the auxiliary image to the primary image.

What is claimed is: 
     
       1. A multi-aperture imaging system comprising:
 a) a first camera that provides a first camera image, the first camera having a first sensor with a first plurality of sensor pixels covered at least in part with a non-standard first color filter array (CFA) used to increase a specific color sampling rate relative to a same color sampling rate in a standard CFA, wherein the non-standard first CFA includes a repetition of a n×n micro-cell where n=4 and wherein each micro-cell includes a BBRR-RBBR-RRBB-BRRB color filter order; 
 b) a second camera that provides a second camera image, the second camera having a second sensor with a second plurality of sensor pixels, the second plurality of sensor pixels being either Clear or covered with a standard second CFA, wherein the second camera image has an overlap area with the first camera image, wherein the second CFA is one of RGB (Bayer), RGBE, CYYM, CYGM, RGBW#1, RGBW#2 or RGBW#3; and 
 c) a processor configured to process the first and second camera images into a fused output image, wherein in the overlap area pixels of the second camera image are registered with corresponding pixels of the first camera image. 
 
     
     
       2. A multi-aperture imaging system comprising:
 a) a first camera that provides a first camera image, the first camera having a first sensor with a first plurality of sensor pixels covered at least in part with a non-standard first color filter array (CFA) used to increase a specific color sampling rate relative to a same color sampling rate in a standard CFA, wherein the non-standard first CFA includes a repetition of a n×n micro-cell where n=6 and wherein each micro-cell includes a color filter order selected from the group consisting of RBBRRB-RWRBWB-BBRBRR-RRBRBB-BWBRWR-BRRBBR, BBGRRG-RGRBGB-GBRGRB-RRGBBG-BGBRGR-GRBGBR, RBBRRB-RGRBGB-BBRBRR-RRBRBB-BGBRGR-BRRBBR and RBRBRB-BGBRGR-RBRBRB-BRBRBR-RGRBGB-BRBRBR; 
 b) a second camera that provides a second camera image, the second camera having a second sensor with a second plurality of sensor pixels, the second plurality of sensor pixels being either Clear or covered with a standard second CFA, wherein the second camera image has an overlap area with the first camera image, wherein the second CFA is one of RGB (Bayer), RGBE, CYYM, CYGM, RGBW#1, RGBW#2 or RGBW#3; and 
 c) a processor configured to process the first and second camera images into a fused output image, wherein in the overlap area pixels of the second camera image are registered with corresponding pixels of the first camera image. 
 
     
     
       3. The multi-aperture imaging system of  claim 1 , wherein the first camera is a Wide camera with a field of view FOV w  and wherein the second camera is a Tele camera with a field of view FOV T  smaller than FOV w . 
     
     
       4. A method of acquiring images by a multi-aperture imaging system, the method comprising:
 a) providing a first image generated by a first camera of the imaging system, the first camera having a first field of view (FOV 1 ); 
 b) providing a second image generated by a second camera of the imaging system, the second camera having a second field of view (FOV 2 ) such that FOV 2 <FOV 1 , the second image having an overlap area with the first image; and 
 c) fusing the first and second images into a fused image, wherein the fusing includes applying a registration process between the first and second images, the registration process including:
 i. extracting a first Luma image from the first image 
 ii. extracting a second Luma image from the second image, 
 iii. applying low-pass filtering on the second Luma image in order to match its spatial frequency content to that of the first Luma image and to generate a low-pass second Luma image, and 
 iv. applying registration on the low-pass second Luma image and the first Luma image, 
 
 wherein the non-standard a CFA includes a repetition of a n×n micro-cell where n=4 and 
 wherein each micro-cell includes a BBRR-RBBR-RRBB-BRRB color filter order. 
 
     
     
       5. The method of  claim 4 , wherein n=6 instead of n=4 and wherein instead of each micro-cell including a BBRR-RBBR-RRBB-BRRB color filter order, each micro-cell includes a color filter order selected from the group consisting of RBBRRB-RWRBWB-BBRBRR-RRBRBB-BWBRWR-BRRBBR, BBGRRG-RGRBGB-GBRGRB-RRGBBG-BGBRGR-GRBGBR, RBBRRB-RGRBGB-BBRBRR-RRBRBB-BGBRGR-BRRBBR and RBRBRB-BGBRGR-RBRBRB-BRBRBR-RGRBGB-BRBRBR. 
     
     
       6. A multi-aperture imaging system comprising:
 a) a first camera that provides a first image, the first camera having a first field of view (FOV 1 ) and a first sensor with a first color filter array; and   b) a second camera that provides a second image, the second camera having a second field of view (FOV 2 ) such that FOV 2 <FOV 1  and a second sensor with a second color filter array different from the first color filter array;
 wherein the first color filter array has a first red color pixel and a second red color pixel adjacent to the first red color pixel, and a first blue color pixel and a second blue color pixel adjacent to the first blue color pixel. 
   
     
     
       7. The multi-aperture imaging system of claim 6, wherein the first red color pixel is disposed adjacent to the first blue color pixel in a diagonal direction. 
     
     
       8. The multi-aperture imaging system of claim 7, wherein the second color filter array is one of an RGB (Bayer), RGBE, CYYM, CYGM, RGBW#1, RGBW#2 or RGBW#3 color filter array. 
     
     
       9. The multi-aperture imaging system of claim 7, wherein the first color filter array is one of an RGBE, CYYM, CYGM, RGBW#1, RGBW#2 or RGBW#3 color filter array. 
     
     
       10. The multi-aperture imaging system of claim 9, wherein the first color filter array has a first 2×2 pixel group with three blue color pixels and a fourth pixel that is not a blue color pixel. 
     
     
       11. The multi-aperture imaging system of claim 10, wherein the first color filter array has a second 2×2 pixel group with three red color pixels and a fourth pixel that is not a red color pixel. 
     
     
       12. A multi-aperture imaging system comprising:
 a) a first camera that provides a first image, the first camera having a first sensor with a first color filter array; and   b) a second camera that provides a second image, the second camera having a second sensor with a second color filter array different from the first color filter array,
 wherein the first color filter array has a first red color pixel group comprising a first red color pixel and a second red color pixel adjacent to the first red color pixel in a first direction, and a first blue color pixel group comprising a first blue color pixel and a second blue color pixel adjacent to the first blue color pixel in a second direction perpendicular to the first direction. 
   
     
     
       13. The multi-aperture imaging system of claim 12, wherein the first camera has a first field of view (FOV 1 ) and wherein the second camera has a second field of view (FOV 2 ) such that FOV 2 <FOV 1 . 
     
     
       14. The multi-aperture imaging system of claim 12, wherein the first red color pixel is disposed adjacent to the first blue color pixel in a diagonal direction. 
     
     
       15. The multi-aperture imaging system of claim 12, wherein the first color filter array has a first 2×2 pixel group with three blue color pixels and a fourth pixel that is not a blue pixel. 
     
     
       16. The multi-aperture imaging system of claim 12, wherein the first color filter array has a second 2×2 pixel group with three red color pixels and a fourth pixel that is not a red color pixel. 
     
     
       17. The multi-aperture imaging system of claim 12, wherein the first color filter array is a non-Bayer color filter array. 
     
     
       18. The multi-aperture imaging system of claim 17, wherein the second color filter array is a standard color filter array. 
     
     
       19. The multi-aperture imaging system of claim 12, wherein the first color filter array has a first red and green color pixel group with a first red color pixel and a first green color pixel adjacent to the first red color pixel in the first direction. 
     
     
       20. A multi-aperture imaging system comprising:
 a) a first camera that provides a first image, the first camera having a sensor with a first color filter array and a first resolution; and   b) a second camera that provides a second image, the second camera having a second sensor with a second color filter array and a second resolution,
 wherein the first color filter array has a first red color pixel and a second red color pixel adjacent to the first red color pixel, and wherein the output image has a third resolution higher than the first resolution. 
   
     
     
       21. The multi-aperture imaging system of claim 20, wherein the first sensor has a first blue color pixel group with a first blue color pixel and a second blue color pixel adjacent to the first blue color pixel. 
     
     
       22. The multi-aperture imaging system of claim 21, wherein the first red color pixel is disposed adjacent to the first blue color pixel in a diagonal direction. 
     
     
       23. The multi-aperture imaging system of claim 22, wherein the first camera has a first field of view (FOV1) and the second camera has a second field of view (FOV2) different from FOV1. 
     
     
       24. The multi-aperture imaging system of claim 23, wherein the FOV1 is greater than the FOV2.