Methods, systems, and media for generating an immersive light field video with a layered mesh representation

Abstract

Mechanisms for generating compressed images are provided. More particularly, methods, systems, and media for capturing, reconstructing, compressing, and rendering view-dependent immersive light field video with a layered mesh representation are provided.

What is claimed is: 
     
         1 . A method for generating compressed videos, the method comprising:
 receiving an image;   converting the received image into a layered mesh;   generating a texture-atlased image that consolidates the layered mesh;   generating a compressed image by compressing the texture-atlased image; and   storing the compressed image in association with an identifier of the received image.   
     
     
         2 . The method of  claim 1 , wherein the image is a multi-sphere image that is generated by obtaining a sequence of images from a plurality of camera devices and generating, for each of the images in the sequence of images, the multi-sphere image that includes a series of concentric spherical each having an RGBA texture map that is generated based on the images obtained from the plurality of camera devices. 
     
     
         3 . The method of  claim 2 , further comprising generating the layered mesh by converting the multi-sphere image into the layered mesh, wherein the layered mesh is a polygonal mesh with a corresponding RGBA texture. 
     
     
         4 . The method of  claim 1 , wherein the image is received from a plurality of camera devices mounted on a portion of a spherical structure having a plurality of vertices and wherein a portion of the spherical structure at which a lens of a camera device is located is removed such that the camera device captures an image of an object located outside the portion of the sphere. 
     
     
         5 . The method of  claim 4 , wherein one camera device of the plurality of camera devices is designated a leader camera device such that the leader camera device triggers capture of the sequence of images by the remaining camera devices in the plurality of camera devices. 
     
     
         6 . The method of  claim 1 , further comprising converting the received image to the layered mesh by subdividing layers of the received image into a plurality of layer groups. 
     
     
         7 . The method of  claim 6 , wherein each layer group in the plurality of layer groups has a same number of layers. 
     
     
         8 . The method of  claim 6 , further comprising computing layer depth maps within each layer group of the plurality of layer groups based on layer disparity. 
     
     
         9 . The method of  claim 8 , further comprising converting each of the layer depth maps to the layered mesh. 
     
     
         10 . The method of  claim 9 , further comprising projecting RGBA values from layers of the received image onto the layered mesh. 
     
     
         11 . A system for generating compressed videos, the system comprising:
 a hardware processor that is configured to:
 receive an image; 
 convert the received image into a layered mesh; 
 generate a texture-atlased image that consolidates the layered mesh; 
 generate a compressed image by compressing the texture-atlased image; and 
 store the compressed image in association with an identifier of the received image. 
   
     
     
         12 . The system of  claim 11 , wherein the image is a multi-sphere image that is generated by obtaining a sequence of images from a plurality of camera devices and generating, for each of the images in the sequence of images, the multi-sphere image that includes a series of concentric spherical each having an RGBA texture map that is generated based on the images obtained from the plurality of camera devices. 
     
     
         13 . The system of  claim 12 , wherein the hardware processor is further configured to generate the layered mesh by converting the multi-sphere image into the layered mesh, wherein the layered mesh is a polygonal mesh with a corresponding RGBA texture. 
     
     
         14 . The system of  claim 11 , wherein the image is received from a plurality of camera devices mounted on a portion of a spherical structure having a plurality of vertices and wherein a portion of the spherical structure at which a lens of a camera device is located is removed such that the camera device captures an image of an object located outside the portion of the sphere. 
     
     
         15 . The system of  claim 14 , wherein one camera device of the plurality of camera devices is designated a leader camera device such that the leader camera device triggers capture of the sequence of images by the remaining camera devices in the plurality of camera devices. 
     
     
         16 . The system of  claim 11 , wherein the hardware processor is further configured to convert the received image to the layered mesh by subdividing layers of the received image into a plurality of layer groups. 
     
     
         17 . The system of  claim 16 , wherein each layer group in the plurality of layer groups has a same number of layers. 
     
     
         18 . The system of  claim 16 , wherein the hardware processor is further configured to compute layer depth maps within each layer group of the plurality of layer groups based on layer disparity. 
     
     
         19 . The system of  claim 18 , wherein the hardware processor is further configured to convert each of the layer depth maps to the layered mesh. 
     
     
         20 . The system of  claim 19 , wherein the hardware processor is further configured to project RGBA values from layers of the received image onto the layered mesh. 
     
     
         21 . A non-transitory computer-readable medium containing computer executable instructions that, when executed by a processor, cause the processor to perform a method for generating compressed images, the method comprising:
 receiving an image;   converting the received image into a layered mesh;   generating a texture-atlased image that consolidates the layered mesh;   generating a compressed image by compressing the texture-atlased image; and   storing the compressed image in association with an identifier of the received image.