Adaptive field of view prediction

Abstract

A method for streaming a 360 degree video over a communications network, wherein the video is streamed in a plurality of chunks, includes selecting a prediction window during which to predict a field of view within the video, the field of view is expected to be visible by a viewer at a time of playback of a next chunk of the video, wherein a duration of the prediction window is based on at least one condition within the communications network, selecting a machine learning algorithm to predict the field of view based on a head movement of the viewer, wherein the machine learning algorithm is selected based on the duration of the prediction window, predicting the field of view based on the head movement of the viewer and the machine learning algorithm, identifying a tile of the next chunk that corresponds to the field of view, and downloading the tile.

What is claimed is: 
     
         1 . A method for streaming a 360 degree video over a communications network, wherein the 360 degree video is streamed in a plurality of chunks, the method comprising:
 monitoring, by a processor, at least one condition within the communications network, wherein the monitoring indicates that the at least one condition is unstable;   selecting, by the processor, a first prediction window during which to predict a field of view within the 360 degree video, wherein the field of view is expected to be visible by a viewer at a time of playback of a next chunk of the plurality of chunks, and wherein the first prediction window is selected based on the at least one condition being detected as unstable;   selecting, by the processor, a first machine learning algorithm to predict the field of view;   predicting, by the processor, the field of view based on a head movement of the viewer and the first machine learning algorithm;   identifying, by the processor, a first tile of the next chunk that corresponds to the field of view; and   downloading, by the processor, the first tile.   
     
     
         2 . The method of  claim 1 , wherein the first prediction window defines a first period of time prior to the time of playback of the next chunk. 
     
     
         3 . The method of  claim 1 , wherein the at least one condition comprises a latency of the communications network. 
     
     
         4 . The method of  claim 3 , wherein a duration of the first prediction window is proportional to the latency. 
     
     
         5 . The method of  claim 3 , wherein a duration of the first prediction window is preassigned to the at least one condition in a lookup table. 
     
     
         6 . The method of  claim 1 , wherein the first machine learning algorithm is one of a plurality of machine learning algorithms available to use for predicting the field of view. 
     
     
         7 . The method of  claim 6 , wherein the first machine learning algorithm is preassigned to a duration of the first prediction window in a lookup table. 
     
     
         8 . The method of  claim 6 , wherein the first machine learning algorithm is observed to perform best among the plurality of machine learning algorithms for a duration of the first prediction window. 
     
     
         9 . The method of  claim 1 , wherein the method is performed every time a new chunk of the plurality of chunks is to be downloaded. 
     
     
         10 . The method of  claim 1 , wherein the method is performed by a wearable display device worn by the viewer, and wherein the wearable display device displays at least a portion of the 360 degree video to the viewer. 
     
     
         11 . The method of  claim 2 , further comprising:
 selecting, by the processor, a second prediction window during which to predict the field of view within the 360 degree video, wherein the second prediction window defines a second period of time prior to the time of playback of the next chunk that is shorter in duration when compared to the first prediction window;   predicting, by the processor, the field of view based on the head movement of the viewer and a second machine learning algorithm;   identifying, by the processor, a second tile of the next chunk that corresponds to the field of view; and   wherein the downloading comprises downloading the second tile instead of the first tile when the first tile and the second tile are different.   
     
     
         12 . A wearable display device comprising:
 a processor configured to stream a 360 degree video over a communications network, wherein the 360 degree video is streamed in a plurality of chunks; and   a computer-readable medium storing instructions which, when executed by the processor, cause the processor to perform operations, the operations comprising:
 monitoring at least one condition within the communications network, wherein the monitoring indicates that the at least one condition is unstable; 
 selecting a first prediction window during which to predict a field of view within the 360 degree video, wherein the field of view is expected to be visible by a viewer at a time of playback of a next chunk of the plurality of chunks, and wherein the first prediction window is selected based on the at least one condition being detected as unstable; 
 selecting a first machine learning algorithm to predict the field of view; 
 predicting the field of view based on a head movement of the viewer and the first machine learning algorithm; 
 identifying a first tile of the next chunk that corresponds to the field of view; and 
 downloading the first tile. 
   
     
     
         13 . The wearable display device of  claim 12 , wherein the first prediction window defines a first period of time prior to the time of playback of the next chunk. 
     
     
         14 . The wearable display device of  claim 12 , wherein the at least one condition comprises a latency of the communications network. 
     
     
         15 . The wearable display device of  claim 14 , wherein a duration of the first prediction window is proportional to the latency. 
     
     
         16 . The wearable display device of  claim 14 , wherein a duration of the first prediction window is preassigned to the at least one condition in a lookup table. 
     
     
         17 . The wearable display device of  claim 12 , wherein the first machine learning algorithm is one of a plurality of machine learning algorithms available to use for predicting the field of view. 
     
     
         18 . The wearable display device of  claim 17 , wherein the first machine learning algorithm is preassigned to a duration of the first prediction window in a lookup table. 
     
     
         19 . The wearable display device of  claim 13 , the operations further comprise:
 selecting a second prediction window during which to predict the field of view within the 360 degree video, wherein the second prediction window defines a second period of time prior to the time of playback of the next chunk that is shorter in duration when compared to the first prediction window;   predicting the field of view based on the head movement of the viewer and a second machine learning algorithm;   identifying a second tile of the next chunk that corresponds to the field of view; and   wherein the downloading comprises downloading the second tile instead of the first tile when the first tile and the second tile are different.   
     
     
         20 . A non-transitory computer-readable storage medium storing instructions which, when executed by a processor, cause the processor to perform operations for streaming a 360 degree video over a communications network, wherein the 360 degree video is streamed in a plurality of chunks, the operations comprising:
 monitoring at least one condition within the communications network, wherein the monitoring indicates that the at least one condition is unstable;   selecting a first prediction window during which to predict a field of view within the 360 degree video, wherein the field of view is expected to be visible by a viewer at a time of playback of a next chunk of the plurality of chunks, and wherein the first prediction window is selected based on the at least one condition being detected as unstable;   selecting a first machine learning algorithm to predict the field of view;   predicting the field of view based on a head movement of the viewer and the first machine learning algorithm;   identifying a first tile of the next chunk that corresponds to the field of view; and   downloading the first tile.