System and method for lightfield capture

Abstract

A system for generating holographic images or videos comprising a camera array, a plurality of processors, and a central computing system. A method for generating holographic images can include receiving a set of images and processing the images.

We claim: 
     
         1 . A method for generating a three-dimensional image of a scene, comprising:
 acquiring a primary image of the scene using a primary camera;   acquiring a set of images of the scene, wherein each image of the set of images is acquired using a camera of a plurality of cameras;   cropping the primary image to a region of interest, wherein the region of interest is associated with a feature from the scene;   for each image of the set of images:
 determining an image region of interest; 
 shifting the image region of interest to correct for a relative yaw or a relative pitch between the primary camera and the camera that acquired the respective image; 
 shifting the image region of interest to correct for a relative translation between the primary camera and the camera that acquired the respective image; 
 cropping the respective image to the shifted image region of interest; and 
   generating the three-dimensional image of the scene using the cropped primary image and the set of cropped images.   
     
     
         2 . The method of  claim 1  further comprising:
 after generating the holographic image, acquiring a second primary image and a second set of images; 
 cropping the second primary image to a second region of interest associated with the feature of the scene; 
 cropping each image of the second set of images to a second shifted region of interest associated with a respective image, wherein the second shifted region of interest is determined based on the second region of interest; and 
 generating a second three-dimensional image comprising the cropped second primary image and the second set of cropped images. 
 
     
     
         3 . The method of  claim 2 , wherein the second region of interest is determined by tracking a change in a position of the feature between the primary image and the second primary image. 
     
     
         4 . The method of  claim 3 , wherein the second region of interest is translated from the region of interest based on the change in the position of the feature. 
     
     
         5 . The method of  claim 1 , wherein a size of the region of interest depends on a zoom of the three-dimensional image. 
     
     
         6 . The method of  claim 5 , wherein the size of the region of interest and a size of each of the image region of interests is the same, wherein the plurality of cameras and the primary camera are arranged in a planar array. 
     
     
         7 . The method of  claim 1 , wherein generating the holographic image further comprises generating a synthetic image between two images of the set of images or between the primary image and an image of the set of images, wherein the holographic image further comprises the synthetic image. 
     
     
         8 . The method of  claim 1 , wherein the primary image and the set of images are processed using a distributed computing network. 
     
     
         9 . The method of  claim 8 , wherein the primary camera comprises a primary camera processor and each camera of the plurality of cameras comprises a unique camera processor, wherein the primary camera processor crops the primary image and wherein each image of the set of images is processed using the respective unique camera processor associated with the respective image. 
     
     
         10 . The method of  claim 1 , wherein generating the three-dimensional image comprises arranging the primary image and the set of images in a quilt image. 
     
     
         11 . A system comprising:
 a first camera;   a second camera with a known pose relative to the reference camera; wherein the first camera and the second camera are configured to capture images of a scene with different perspectives;   a first camera processor, connected to the first camera, configured to:
 determine a region of interest associated with a feature in a first image captured by the first camera; 
 crop the first image to the region of interest; 
 transmit the region of interest to a second camera processor connected to the second camera; and 
 transmit the cropped first image to a display; and 
   wherein the second camera processor is configured to:
 determine an auxiliary region of interest in a second image captured using the second camera, wherein the auxiliary region of interest is determined based on the region of interest and the known pose; 
 crop the second image to the auxiliary region of interest; and 
 transmit the cropped second image to the display. 
   
     
     
         12 . The system of  claim 11 , wherein the display is configured to present the cropped reference image and the cropped auxiliary image as a three-dimensional image. 
     
     
         13 . The system of  claim 11 , wherein the first camera processor comprises a microprocessor. 
     
     
         14 . The system of  claim 11 , wherein the central computing system is configured to generate the holographic image before a camera of the plurality of cameras acquires a second image. 
     
     
         15 . The system of  claim 11 , further comprising a central computing system configured to generate one or more interpolated images between the cropped reference image and the cropped auxiliary image, wherein the one or more interpolated images are transmitted to the display. 
     
     
         16 . The system of  claim 11 , wherein the second camera processor determines the auxiliary region of interest by translating the auxiliary region of interest to correct for a translation between the first and the second camera. 
     
     
         17 . The system of  claim 16 , wherein the second camera processor further determines the auxiliary region of interest by rotating the auxiliary region of interest to correct for a pitch or a yaw of the second camera relative to the first camera. 
     
     
         18 . The system of  claim 11 , wherein the first camera processor is further configured to rotate the first image based on calibration data associated with the first camera. 
     
     
         19 . The system of  claim 18 , wherein the second camera processor is further configured to rotate the second image based on calibration data associated with the second camera. 
     
     
         20 . The system of  claim 11 , wherein the cropped first image and the cropped second image are transmitted to a display before the first camera or the second camera acquires a subsequent frame.