High resolution thin multi-aperture imaging systems

Abstract

A multi-aperture imaging system comprising a first camera with a first sensor that captures a first image and a second camera with a second sensor that captures a second image, the two cameras having either identical or different FOVs. The first sensor may have a standard color filter array (CFA) covering one sensor section and a non-standard color CFA covering another. The second sensor may have either Clear or standard CFA covered sections. Either image may be chosen to be a primary or an auxiliary image, based on a zoom factor. An output image with a point of view determined by the primary image is obtained by registering the auxiliary image to the primary image.

What is claimed is: 
     
       1. A multi-aperture imaging system comprising:
 a) a first camera that provides a first camera image, the first camera having a first sensor with a first plurality of sensor pixels covered at least in part with a non-standard color filter array (CFA) used to increase a specific color sampling rate relative to a same color sampling rate in a standard CFA, wherein the non-standard CFA includes a repetition of a n×n micro-cell where n=4 and wherein each micro-cell includes a BBRR-RBBR-RRBB-BRRB color filter order;   b) a second camera that provides a second camera image, the second camera having a second sensor with a second plurality of sensor pixels, the second plurality of sensor pixels being either Clear or covered with a standard CFA, wherein the second camera image has an overlap area with the first camera image; and   c) a processor configured to process the first and second camera images into a fused output image, wherein in the overlap area pixels of the second camera image are registered with corresponding pixels of the first camera image.   
     
     
       2. A multi-aperture imaging system comprising:
 a) a first camera that provides a first camera image, the first camera having a first sensor with a first plurality of sensor pixels covered at least in part with a non-standard color filter array (CFA) used to increase a specific color sampling rate relative to a same color sampling rate in a standard CFA, wherein the non-standard CFA includes a repetition of a n×n micro-cell where n=6 and wherein each micro-cell includes a color filter order selected from the group consisting of RBBRRB-RWRBWB-BBRBRR-RRBRBB-BWBRWR-BRRBBR, BBGRRG-RGRBGB-GBRGRB-RRGBBG-BGBRGR-GRBGBR, RBBRRB-RGRBGB-BBRBRR-RRBRBB-BGBRGR-BRRBBR and RBRBRB-BGBRGR-RBRBRB-BRBRBR-RGRBGB-BRBRBR;   b) a second camera that provides a second camera image, the second camera having a second sensor with a second plurality of sensor pixels, the second plurality of sensor pixels being either Clear or covered with a standard CFA, wherein the second camera image has an overlap area with the first camera image; and   c) a processor configured to process the first and second camera images into a fused output image, wherein in the overlap area pixels of the second camera image are registered with corresponding pixels of the first camera image.   
     
     
       3. The multi-aperture imaging system of  claim 1 , wherein the first camera is a Wide camera with a field of view FOV w  and wherein the second camera is a Tele camera with a field of view FOV T  smaller than FOV w . 
     
     
       4. A method of acquiring images by a multi-aperture imaging system, the method comprising:
 a) providing a first image generated by a first camera of the imaging system, the first camera having a first field of view (FOV 1 );   b) providing a second image generated by a second camera of the imaging system, the second camera having a second field of view (FOV 2 ) such that FOV 2 <FOV 1 , the second image having an overlap area with the first image; and   c) fusing the first and second images into a fused image, wherein the fusing includes applying a registration process between the first and second images, the registration process including:
 i. extracting a first Luma image from the first image 
 ii. extracting a second Luma image from the second image, 
 iii. applying low-pass filtering on the second Luma image in order to match its spatial frequency content to that of the first Luma image and to generate a low-pass second Luma image, and 
 iv. applying registration on the low-pass second Luma image and the first Luma image, 
   wherein the non-standard CFA includes a repetition of a n×n micro-cell where n=4 and   wherein each micro-cell includes a BBRR-RBBR-RRBB-BRRB color filter order.   
     
     
       5. The method of  claim 4 , wherein n=6 instead of n=4 and wherein instead of each micro-cell including a BBRR-RBBR-RRBB-BRRB color filter order, each micro-cell includes a color filter order selected from the group consisting of RBBRRB-RWRBWB-BBRBRR-RRBRBB-BWBRWR-BRRBBR, BBGRRG-RGRBGB-GBRGRB-RRGBBG-BGBRGR-GRBGBR, RBBRRB-RGRBGB-BBRBRR-RRBRBB-BGBRGR-BRRBBR and RBRBRB-BGBRGR-RBRBRB-BRBRBR-RGRBGB-BRBRBR. 
     
     
       6. A multi-aperture imaging system, comprising:
 a) a first camera that provides a first image, the first camera having a fixed first field of view (FOV1) and a first sensor with a first plurality of sensor pixels covered at least in part with a first color filter array (CFA);   b) a second camera that provides a second image, the second camera having a fixed second field of view (FOV2) such that FOV2<FOV1 and a second sensor with a second plurality of sensor pixels, the second image having an overlap area with the first image, the first sensor having a sensor overlap area with the second sensor and a sensor non-overlap area; and   c) a third camera that provides a third image, the third camera having a fixed third field of view (FOV3) such that FOV3<FOV2, and a third sensor with a third plurality of sensor pixels, the third image having an overlap area with the second image; and   d) a processor configured to provide an output image based on a zoom factor (ZF) input that defines a respective field of view (FOVZF), such that for FOV2<FOVZF<FOV1 the processor is further configured to select the first image as a primary image, to register the overlap area of the second image to the first image to obtain a fused image, and to provide the fused image as the output image from the point of view of the first camera,   wherein a CFA pattern of the first CFA in the sensor overlap area differs from a CFA pattern of the first CFA in the sensor non-overlap area, and wherein a demosaicing process applied to the first CFA in the sensor overlap area differs from a demosaicing process applied to the CFA pattern of the first CFA in the sensor non-overlap area.   
     
     
       7. The multi-aperture imaging system of claim 6, wherein if FOV3>FOVZF, then the processor is further configured to provide an output image from the point of view of the third camera. 
     
     
       8. A multi-aperture imaging system, comprising:
 a) a first camera that provides a first image, the first camera having a fixed first field of view (FOV1) and a first sensor with a first plurality of sensor pixels covered at least in part with a first color filter array (CFA);   b) a second camera that provides a second image, the second camera having a fixed second field of view (FOV2) such that FOV2<FOV1 and a second sensor with a second plurality of sensor pixels, the second image having an overlap area with the first image, the first sensor having a sensor overlap area with the second sensor and a sensor non-overlap area; and   c) a third camera that provides a third image, the third camera having a fixed third field of view (FOV3) such that FOV3<FOV2, and a third sensor with a third plurality of sensor pixels, the third image having an overlap area with the second image; and   d) a processor configured to provide an output image based on a zoom factor (ZF) input that defines a respective field of view (FOVZF), such that for FOV3<FOVZF<FOV2 the processor is further configured to select the second image as a primary image, to register the overlap area of the third image to the second image to obtain a fused image, and to provide the fused image as the output image from the point of view of the second camera,   wherein a CFA pattern of the first CFA in the sensor overlap area differs from a CFA pattern of the first CFA in the sensor non-overlap area, and wherein a demosaicing process applied to the first CFA in the sensor overlap area differs from a demosaicing process applied to the CFA pattern of the first CFA in the sensor non-overlap area.   
     
     
       9. The multi-aperture imaging system of claim 8, wherein if FOV3>FOVZF, then the processor is further configured to provide the output image from the point of view of the third camera. 
     
     
       10. A method of acquiring images by a multi-aperture imaging system, comprising:
 a) providing a first image generated by a first camera of the imaging system, the first camera having a fixed first field of view (FOV1) and a first sensor with a first plurality of sensor pixels covered at least in part with a first color filter array (CFA);   b) providing a second image generated by a second camera of the imaging system, the second camera having a fixed second field of view (FOV2) such that FOV2<FOV1, and a second sensor with a second plurality of sensor pixels, the second image having an overlap area with the first image, the first sensor having a sensor overlap area with the second sensor and a sensor non-overlap area;   c) providing a third image generated by a third camera of the imaging system, the third camera having a fixed third field of view (FOV3) such that FOV3<FOV2, and a third sensor with a third plurality of sensor pixels, the third image having an overlap area with the second image; and   d) using a processor to provide an output image from a point of view of the first camera, the second camera, or the third camera, based on a zoom factor (ZF) input that defines a respective field of view (FOVZF) such that:
 if FOV2<FOVZF<FOV1, then the processor is configured to select the first image as a primary image, to register the overlap area of the second image to the first image to obtain a fused image and to provide the fused image as the output image from the point of view of the first camera, 
 if FOV3<FOVZF<FOV2, then the processor is configured to select the second image as a primary image, to register the overlap area of the third image to the second image to obtain a fused image and to provide the fused image as the output image from the point of view of the second camera, and 
   if FOV3>FOVZF, then the processor is configured to provide an output image from the point of view of the third camera,
 wherein a CFA pattern of the first CFA in the sensor overlap area differs from a CFA pattern of the first CFA in the sensor non-overlap area, and wherein a demosaicing process applied to the first CFA in the sensor overlap area differs from a demosaicing process applied to the CFA pattern of the first CFA in the sensor non-overlap area.