Lightweight View Dependent Rendering System for Mobile Devices

Abstract

An example system for lightweight view dependent rendering is described herein. The system includes a processor configured to determine a moving region of each camera view, wherein the moving region is defined by a bounding box and track each moving region to obtain a plurality of cropped videos for each camera view. The system may also be configured to generate a billboard for each camera view and render each billboard.

What is claimed is: 
     
         1 . A system for lightweight view dependent rendering, comprising:
 a plurality of cameras configured to capture a volumetric scene;   a memory;   a processor configured to
 determine a moving region of a camera view, wherein the moving region is defined by a bounding box; 
 track the moving region to obtain a cropped video from the camera view; 
 segment a foreground region from the cropped video to produce a matte; 
 generate a billboard for the cropped video; and 
 render the billboard, wherein the matte is applied to the billboard. 
   
     
     
         2 . The system of  claim 1 , wherein the billboard is rendered via an augmented reality toolkit. 
     
     
         3 . The system of  claim 1 , wherein the billboard is rendered on top of computer-generated objects and a background by replacing the matte with the computer-generated objects and the background. 
     
     
         4 . The system of  claim 1 , wherein the moving region is extracted from a three-dimensional data representation. 
     
     
         5 . The system of  claim 1 , wherein the moving region is a person. 
     
     
         6 . The system of  claim 1 , wherein the moving region is tracked in the camera view via machine learning. 
     
     
         7 . The system of  claim 1 , wherein the billboard is encoded into a single video. 
     
     
         8 . The system of  claim 1 , comprising a plurality of billboards from the camera view, wherein each billboard of the plurality of billboards is encoded and transmitted independently. 
     
     
         9 . The system of  claim 1 , comprising a plurality of billboards from the camera view, wherein the plurality of billboards is encoded and transmitted via a single video file. 
     
     
         10 . The system of  claim 1 , wherein the billboard is scaled to render the billboard into an augmented reality. 
     
     
         11 . A method for lightweight view dependent rendering, comprising:
 determining a moving region of a camera view, wherein the moving region is defined by a bounding box;   tracking the moving region to obtain a cropped video from the camera view;   segmenting a foreground region from the cropped video to produce a matte;   generating a billboard for the cropped video; and   rendering the billboard, wherein the matte is applied to the billboard.   
     
     
         12 . The method of  claim 11 , wherein the billboard is rendered via an augmented reality toolkit. 
     
     
         13 . The method of  claim 11 , wherein the billboard is rendered on top of computer-generated objects and a background by replacing the matte with the computer-generated objects and the background. 
     
     
         14 . The method of  claim 11 , wherein the moving region is extracted from a three-dimensional data representation. 
     
     
         15 . The method of  claim 11 , wherein the moving region is sporting equipment manipulated during an athletic event. 
     
     
         16 . The method of  claim 11 , wherein the moving region is tracked the camera view via machine learning. 
     
     
         17 . The method of  claim 11 , wherein the billboard for each camera view is encoded into a single video. 
     
     
         18 . The method of  claim 11 , comprising a plurality of billboards from the camera view, wherein each billboard of the plurality of billboards is encoded and transmitted independently. 
     
     
         19 . The method of  claim 11 , comprising a plurality of billboards from the camera view, wherein the plurality of billboards is encoded and transmitted via a single video file. 
     
     
         20 . The method of  claim 11 , wherein the billboard is scaled to render the billboard into an augmented reality. 
     
     
         21 . An apparatus, comprising:
 a plurality of cameras configured to capture a volumetric scene;   a processor to generate a 3D data representation of the volumetric scene;   the processor configured to
 determine a moving region from the 3D data representation, wherein the moving region is defined by a bounding box; 
 track the moving region to obtain a cropped video from the 3D data representation; 
 segment a foreground region from the cropped video to produce a matte; 
 generate a billboard for the cropped video; and 
 render the billboard, wherein the matte is applied to the billboard. 
   
     
     
         22 . The apparatus of  claim 21 , wherein the billboard is rendered via an augmented reality toolkit. 
     
     
         23 . The apparatus of  claim 21 , wherein the billboard is rendered on top of computer-generated objects and a background by replacing the matte with the computer-generated objects and the background. 
     
     
         24 . The apparatus of  claim 21 , wherein the 3D data representation is a point cloud. 
     
     
         25 . The apparatus of  claim 21 , wherein the moving region is a person.