Systems and method for gpu based virtual reality video streaming server

Abstract

Systems and methods of processing and streaming a virtual reality video using a graphics processing unit (GPU) are provided. A video server is configured to cause a processor to read, from a video data source, source video data including multiple spherical image frame data and store the source video data in a first memory. The video server is further configured to cause the GPU to convert, in response to storing first spherical image frame data in a first frame buffer of a second memory, the first spherical image frame data to first equirectangular image frame data that correspond to a portion of spherical image represented by the first spherical image frame data, encode the converted first equirectangular image frame data and store the encoded first equirectangular image frame data in an encoded frame buffer of the second memory.

What is claimed is: 
     
         1 . A method comprising:
 initiating, by one or more processors, provision of a first non-spherical image converted from a first spherical image associated with a first field of view;   detecting, by the one or more processors, use of a second field of view before complete provision of the first non-spherical image; and   initiating, by the one or more processors, conversion of the first spherical image into a second non-spherical image associated with the second field of view.   
     
     
         2 . The method of  claim 1 , wherein:
 the first non-spherical image is a first equirectangular image corresponding to the first field of view;   the detecting of the use of the second field of view includes detecting a change from the first field of view to the second field of view; and   the second non-spherical image is a second equirectangular image corresponding to the second field of view.   
     
     
         3 . The method of  claim 1 , further comprising:
 initiating a first conversion of the first spherical image associated with the first field of view into the first non-spherical image; and   initiating storage of the first non-spherical image; and wherein:   the detecting of the use of the second field of view occurs after the initiating of the first conversion of the first spherical image into the first non-spherical image but before completion of the initiated storage of the first non-spherical image.   
     
     
         4 . The method of  claim 1 , further comprising:
 omitting conversion of a second spherical image associated with the second field of view into the second non-spherical image associated with the second field of view in response to the detecting of the use of second field of view occurring before completion of the initiated provision of the first non-spherical image.   
     
     
         5 . The method of  claim 1 , wherein:
 the initiated conversion of the first spherical image associated with the first field of view into the second non-spherical image associated with the second field of view is performed in lieu of converting a second spherical image associated with the second field of view into the second non-spherical image associated with the second field of view.   
     
     
         6 . The method of  claim 1 , further comprising:
 accessing the first spherical image associated with the first field of view from a source from which a second spherical image associated with the second field of view is accessible; and   omitting access of the second spherical image associated with the second field of view from the source from which the first spherical image is accessible in response to the detecting of the use of second field of view occurring before completion of the initiated provision of the first non-spherical image.   
     
     
         7 . The method of  claim 1 , further comprising:
 in response to the use of the second field of view being detected after the initiating of the provision of the first non-spherical image but before completion of the initiated provision of the first non-spherical image, providing the second non-spherical image associated with the second field of view to a device, the second non-spherical image being outputted from the initiated conversion of the first spherical image associated with the first field of view in lieu of being outputted from a further conversion of a second spherical image associated with the second field of view.   
     
     
         8 . A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
 initiating provision of a first non-spherical image converted from a first spherical image associated with a first field of view;   detecting use of a second field of view before complete provision of the first non-spherical image; and   initiating conversion of the first spherical image into a second non-spherical image associated with the second field of view.   
     
     
         9 . The non-transitory machine-readable storage medium of  claim 8 , wherein:
 the first non-spherical image is a first equirectangular image corresponding to the first field of view;   the detecting of the use of the second field of view includes detecting a change from the first field of view to the second field of view; and   the second non-spherical image is a second equirectangular image corresponding to the second field of view.   
     
     
         10 . The non-transitory machine-readable storage medium of  claim 8 , wherein the operations further comprise:
 initiating a first conversion of the first spherical image associated with the first field of view into the first non-spherical image; and   initiating storage of the first non-spherical image; and wherein:   the detecting of the use of the second field of view occurs after the initiating of the first conversion of the first spherical image into the first non-spherical image but before completion of the initiated storage of the first non-spherical image.   
     
     
         11 . The non-transitory machine-readable storage medium of  claim 8 , wherein the operations further comprise:
 omitting conversion of a second spherical image associated with the second field of view into the second non-spherical image associated with the second field of view in response to the detecting of the use of second field of view occurring before completion of the initiated provision of the first non-spherical image.   
     
     
         12 . The non-transitory machine-readable storage medium of  claim 8 , wherein:
 the initiated conversion of the first spherical image associated with the first field of view into the second non-spherical image associated with the second field of view is performed in lieu of converting a second spherical image associated with the second field of view into the second non-spherical image associated with the second field of view.   
     
     
         13 . The non-transitory machine-readable storage medium of  claim 8 , wherein the operations further comprise:
 accessing the first spherical image associated with the first field of view from a source from which a second spherical image associated with the second field of view is accessible; and   omitting access of the second spherical image associated with the second field of view from the source from which the first spherical image is accessible in response to the detecting of the use of second field of view occurring before completion of the initiated provision of the first non-spherical image.   
     
     
         14 . The non-transitory machine-readable storage medium of  claim 8 , wherein the operations further comprise:
 in response to the use of the second field of view being detected after the initiating of the provision of the first non-spherical image but before completion of the initiated provision of the first non-spherical image, providing the second non-spherical image associated with the second field of view to a device, the second non-spherical image being outputted from the initiated conversion of the first spherical image associated with the first field of view in lieu of being outputted from a further conversion of a second spherical image associated with the second field of view.   
     
     
         15 . A system comprising:
 one or more processors; and   a memory storing instructions that, when executed by at least one processor among the one or more processors, cause the system to perform operations comprising:   initiating provision of a first non-spherical image converted from a first spherical image associated with a first field of view;   detecting use of a second field of view before complete provision of the first non-spherical image; and   initiating conversion of the first spherical image into a second non-spherical image associated with the second field of view.   
     
     
         16 . The system of  claim 15 , wherein the operations further comprise:
 initiating a first conversion of the first spherical image associated with the first field of view into the first non-spherical image; and   initiating storage of the first non-spherical image; and wherein:   the detecting of the use of the second field of view occurs after the initiating of the first conversion of the first spherical image into the first non-spherical image but before completion of the initiated storage of the first non-spherical image.   
     
     
         17 . The system of  claim 15 , wherein the operations further comprise:
 omitting conversion of a second spherical image associated with the second field of view into the second non-spherical image associated with the second field of view in response to the detecting of the use of second field of view occurring before completion of the initiated provision of the first non-spherical image.   
     
     
         18 . The system of  claim 15 , wherein:
 the initiated conversion of the first spherical image associated with the first field of view into the second non-spherical image associated with the second field of view is performed in lieu of converting a second spherical image associated with the second field of view into the second non-spherical image associated with the second field of view.   
     
     
         19 . The system of  claim 15 , wherein the operations further comprise:
 accessing the first spherical image associated with the first field of view from a source from which a second spherical image associated with the second field of view is accessible; and   omitting access of the second spherical image associated with the second field of view from the source from which the first spherical image is accessible in response to the detecting of the use of second field of view occurring before completion of the initiated provision of the first non-spherical image.   
     
     
         20 . The system of  claim 15 , wherein the operations further comprise:
 in response to the use of the second field of view being detected after the initiating of the provision of the first non-spherical image but before completion of the initiated provision of the first non-spherical image, providing the second non-spherical image associated with the second field of view to a device, the second non-spherical image being outputted from the initiated conversion of the first spherical image associated with the first field of view in lieu of being outputted from a further conversion of a second spherical image associated with the second field of view.