Adaptive streaming of virtual reality data

Abstract

Apparatus and methods are provided for distributing linear and interactive virtual reality (“VR”) video content. The VR content may be distributed by streaming over varying consumer network connections such as Wi-Fi, LTE Wireless Radio, or wired network connections. Bandwidth and latency may vary from network-user to network-user and may also vary widely during the playback of the VR content as other network users vary their bandwidth usage. Apparatus and methods are provided for ingesting, managing, composing, tracking, and experiencing VR content. Apparatus and methods are provided for reducing bandwidth and computational overhead associated with streaming VR content to a viewing device.

1 . A system for regulating temperature of a viewing device during playback of virtual reality (“VR”) content, the system comprising a software application stored in a non-transitory memory of the viewing device and executable by a CPU and/or GPU of the viewing device, wherein the software application controls the playback of the VR content by:
 for a first frame of the VR content, buffering a first data segment, wherein the first data segment comprises less than all of the first frame and has a first quality level; 
 in response to a first shift in a viewer focal area, rendering pixels on a screen of the viewing device based on the first data segment; 
 for a second frame of the VR content, buffering a second data segment, wherein the second data segment comprises less than all of the second frame and has a second quality level; and, 
 in response to a second shift in the viewer focal area, rendering pixels on the screen of the device based on the second data segment. 
 
     
     
         2 . The system of  claim 1  wherein, the software application controls the playback of the VR content by:
 monitoring the temperature of the viewing device; and 
 selecting the second quality level for the second data segment; 
 wherein a plurality of second data segments are available for selection, each member of the plurality having a different quality level. 
 
     
     
         3 . The system of  claim 1  wherein, the software application controls the playback of the VR content by:
 monitoring the temperature of the viewing device; and 
 in response to detecting that the temperature is within a threshold temperature, selecting the second quality level such that decoding of the second data segment does not raise the temperature above the threshold temperature; 
 wherein a plurality of second data segments are available for selection, each member of the plurality having a different quality level. 
 
     
     
         4 . The system of  claim 1  wherein, the software application controls the playback of the VR content by:
 calculating an expected position of a focal area of the viewer with respect to the second frame of the VR data; and 
 based on the expected position of the focal area, selecting the second data segment from among a plurality of data segments, wherein, the plurality of data segments collectively comprise all of the second frame of the VR data. 
 
     
     
         5 - 14 . (canceled) 
     
     
         15 . A method for streaming of virtual reality (“VR”) content over a network to a viewing device, the streaming occurring during playback of the VR content on the viewing device, the method comprising:
 calculating an expected position of the focal area of the viewer with respect to a video frame of the VR content; 
 based on the expected position of the focal area, requesting a data segment stored in a rasterized rendition, wherein the data segment comprises less than all of the video frame; and, 
 when an actual position of the focal area overlaps the expected position of the focal area, displaying, on a screen of the viewing device, at least a subset of the VR content included in the data segment. 
 
     
     
         16 . The method of  claim 15  further comprising:
 based on the expected position of the viewer focal point with respect to the video frame of the VR content, requesting:
 a first data segment that comprises less than all of the video frame; and 
 a second data segment that comprises less than all of the video frame of the VR content; and 
 
 during the playback of the VR content, when the actual position of the focal area overlaps the expected position of the focal area, synchronizing display of the video frame of the VR data such that:
 at least a subset of the first data segment is displayed on a first area of the screen; and 
 at least a subset of the second data segment is displayed on a second area of the screen; 
 
 wherein, the actual position of the focal area comprises the first and the second areas. 
 
     
     
         17 . The method of  claim 16  further comprising:
 receiving the first data segment over the network at a first time; and 
 receiving the second data segment over the network at a second time. 
 
     
     
         18 . The method of  claim 15 , wherein the data segment is a first data segment and the rasterized rendition is a first rasterized rendition, the method further comprises, based on the expected position of the focal area, requesting a second data segment stored in a second rasterized rendition;
 wherein:
 the second data segment comprises less than all of the video frame and is outside the expected position of the viewer focal area; and 
 the first rasterized rendition has a resolution that is different from the second rasterized rendition. 
   
     
     
         19 . The method of  claim 15  wherein, the video frame comprises a plurality of data segments. 
     
     
         20 . The method of  claim 19  further comprising, receiving each of the plurality of data segments in a discrete bit-stream over the network. 
     
     
         21 . The method of  claim 19  further comprising, based on the expected viewer focal area, receiving fewer than all of the plurality of data segments. 
     
     
         22 . The method of  claim 19  further comprising decoding fewer than all of the plurality of data segments. 
     
     
         23 . The method of  claim 19  further comprising:
 requesting a first number of the plurality of data segments from rasterized renditions of a first resolution; and 
 requesting a second number of the plurality of data segments from rasterized renditions associated with a second resolution. 
 
     
     
         24 . The method of  claim 19  further comprising:
 dynamically recalculating the expected position of the viewer focal area; and 
 in response to the recalculating, dynamically selecting one or more of the plurality of data segments for decoding. 
 
     
     
         25 . The method of  claim 19 , wherein the rasterized rendition is one of a plurality of renditions that includes the data segment, the method further comprising selecting one the plurality of renditions based on one or more of:
 a narrative path associated with the VR content;   bandwidth available on the network;   a performance characteristic of the viewing device;   viewing behavior of the viewer; and   a temperature of the viewing device.