Light field offset rendering

Abstract

A method for light field rendering of a 3D scene data. The rendering method comprises capturing a light field image at a retracted plane, parallel to the display plane, decoding a light field camera to produce a hogel camera for each hogel at the retraction plane to produce an integral image, and applying a pixel remapping technique to the pixels of the integral image for display on a light field display. The 3D scene is captured and remapped to adjust the perspective giving the illusion the image was taken at the display plane, allowing captured images to be displayed in both the inner frustum and outer frustum of a light field display, creating an immersive experience.

We claim: 
     
       1. A computer-implemented light field rendering method comprising:
 defining a display plane for a light field display with a spatial resolution and a directional resolution, the light field display comprising an inner frustum volume bounded by the display plane and a far clip plane and an outer frustum volume bounded by the display plane and a near clip plane; 
 defining a retraction plane parallel to and at an integral offset distance from the display plane, the retraction plane comprising a plurality of light field cameras, each camera having a focal length and spaced at a sample gap, wherein the integral offset distance is calculated from the focal length, directional resolution of the light field display, and an offset integer, N≥1; 
 capturing a view of the 3D scene as a source image at each of the plurality of light field cameras; 
 decoding each source image to generate a plurality of hogel cameras on the retraction plane, each hogel camera providing an elemental image; 
 generating an integral image comprising a plurality of pixels from the elemental images at the retraction plane; and 
 executing a pixel remapping technique on individual pixels in the integral image to create a rendered light field image at the display plane. 
 
     
     
       2. The method of  claim 1 , wherein the view of the 3D scene captured at the plurality of light field cameras at the retraction plane comprises image information from the inner frustum volume and the outer frustum volume. 
     
     
       3. The method of  claim 1 , wherein the captured view of the 3D scene comprises all of the image information in the outer frustum volume. 
     
     
       4. The method of  claim 1 , wherein the retraction plane is positioned at the near clip plane. 
     
     
       5. The method of  claim 1 , further comprising displaying the rendered light field image on a light field display. 
     
     
       6. The method of  claim 1 , wherein the retraction plane surface area is greater than the display plane surface area. 
     
     
       7. The method of  claim 1 , wherein the plurality of light field cameras have optical properties comprising orientation, lens pitch, directional resolution, and field of view. 
     
     
       8. The method of  claim 1 , further comprising generating a plurality of integral images at a plurality of retraction planes. 
     
     
       9. The method of  claim 8 , further comprising compositing the plurality of integral images to create a composited rendered light field image at the display plane. 
     
     
       10. The method of  claim 9 , wherein compositing incorporates transparency data. 
     
     
       11. The method of  claim 1 , wherein each light field camera is one of a digital single reflex mirror (DSLR) camera, pinhole camera, plenoptic camera, compact camera, and mirrorless camera. 
     
     
       12. The method of  claim 1 , wherein each light field camera is a computer-generated camera. 
     
     
       13. The method of  claim 1 , wherein the pixel remapping technique causes the pixels to change their hogel index (H x , H y ) from the retraction plane to the display plane. 
     
     
       14. The method of  claim 1 , wherein the retraction plane is outside of the outer frustum volume. 
     
     
       15. A computer-implemented method of displaying a light field image comprising:
 capturing a first light field at a retraction plane relative to a light field display plane using a light field camera, the first light field comprising an array of retraction plane hogels, each hogel having a plurality of pixels, the retraction plane positioned at an integral offset distance from the display plane, the integral offset distance calculated as an offset integer N, where N≥1; 
 assigning a hogel index (H x , H y ) and a pixel index (P x , P y ) to each pixel in each retraction plane hogel to indicate its position in the light field display by applying a pixel remapping technique to select a single pixel from each retraction plane hogel; 
 loading each pixel from the light field at the retraction plane (LF r ) and storing each pixel to the light field at the display plane (LF d ) using a compositing function; and 
 generating a light field image at the display plane comprising remapped pixels. 
 
     
     
       16. The method of  claim 15 , wherein one pixel from the retraction plane produces one pixel at the display plane. 
     
     
       17. The method of  claim 15 , wherein applying a pixel remapping technique changes the hogel index (H x , H y ) of each pixel in each retraction plane hogel and the pixel index (P x , P y ) remains constant. 
     
     
       18. The method of  claim 17 , wherein the pixel remapping technique is based on the equation LF r [H x +(DR x *N)−(N*P x ), H y +(DR y *N)−(N*P y ), P x , P y ]⇒LF d [H x , H y , P x , P y ]. 
     
     
       19. The method of  claim 15 , wherein the pixel remapping technique is a function of the directional resolution of the light field display and the offset integer, N. 
     
     
       20. The method of  claim 15 , wherein the retraction plane is comprised of a sufficient number of hogels to provide the number of pixels to achieve the required directional resolution of light field display at the display plane. 
     
     
       21. The method of  claim 15 , wherein the size of the display plane is defined by the directional resolution and spatial resolution of the light field display. 
     
     
       22. The method of  claim 15 , wherein the light field camera is a mirror (DSLR) camera, pinhole camera, plenoptic camera, compact camera, or mirrorless camera.