Video encoding and decoding acceleration utilizing IMU sensor data for cloud virtual reality

Abstract

An encoding acceleration method of cloud VR (virtual reality) video, the method comprising: constructing a reference frame candidate for encoding of a current frame; selecting a specific reference frame among the reference frame candidate based on a sensor data of IMU (Inertial Measurement Unit); selecting a prediction mode for encoding the current frame based on information included in the specific reference frame; and encoding the current frame based on the selected prediction mode.

The invention claimed is: 
     
       1. An encoding acceleration method of cloud VR (virtual reality) video, the method comprising:
 obtain a center point of a current frame as the identifier of a current frame, wherein the center point is determined according to a sensor data of IMU (Inertial Measurement Unit); 
 constructing a reference frame candidate for encoding of the current frame based on the center point of the current frame; 
 selecting a specific reference frame among the reference frame candidate by comparing a rotation orientation of the reference frame candidate and a rotation orientation of the current frame, wherein the rotation orientation of the current frame is determined according to the sensor data of IMU; 
 selecting a prediction mode for encoding the current frame based on information included in the specific reference frame; and 
 encoding the current frame based on the selected prediction mode. 
 
     
     
       2. The method of  claim 1 ,
 wherein the IMU includes accelerometers, gyroscopes, and magnetometers. 
 
     
     
       3. The method of  claim 1 ,
 wherein the specific reference frame is a frame having a minimum Euclidean distance from the current frame among the reference frame candidate. 
 
     
     
       4. The method of  claim 3 ,
 wherein a difference between a roll of the specific reference frame and a roll of the current frame is less than a preset threshold. 
 
     
     
       5. The method of  claim 1 ,
 wherein the reference frame candidate is composed of one or more frames having a center point within the scanning zone, 
 wherein the scanning zone corresponds to twice a height of the current frame and twice a width of the current frame. 
 
     
     
       6. The method of  claim 1 ,
 wherein the selecting a prediction mode for encoding the current frame based on information included in the specific reference frame comprising: 
 selecting a specific reference block among blocks comprising the specific reference frame corresponding to a specific current block among blocks comprising the current frame; 
 encoding the specific current block based on a motion vector of the specific reference block. 
 
     
     
       7. The method of  claim 6 ,
 wherein the specific reference block is one of blocks of an overlapping region between the current frame and the specific reference frame. 
 
     
     
       8. The method of  claim 7 ,
 where the specific reference block is a block having the smallest SAD (Sum of Absolute Difference) among SADs of blocks of the overlapping region, 
 wherein the SADs are calculated from an upper-left block among blocks of the overlapping region. 
 
     
     
       9. An encoding acceleration device performing an encoding acceleration method of a cloud VR (Virtual Reality) video, the encoding acceleration device comprising:
 a processor, 
 wherein the processor is configured to: 
 obtain a center point as an identifier of a current frame, wherein the center point is determined according to a sensor data of IMU (Inertial Measurement Unit), 
 construct a reference frame candidate for encoding of the current frame based on the center point of the current frame, 
 select a specific reference frame among the reference frame candidate by comparing a rotation orientation of the reference frame candidate and a rotation orientation of the current frame, wherein the rotation orientation of the current frame is determined according to the sensor data of IMU, 
 select a prediction mode for encoding the current frame based on information included in the specific reference frame, 
 encode the current frame based on the selected prediction mode. 
 
     
     
       10. The device of  claim 9 ,
 wherein the IMU includes accelerometers, gyroscopes, and magnetometers. 
 
     
     
       11. The device of  claim 9 ,
 wherein the specific reference frame is a frame having a minimum Euclidean distance from the current frame among the reference frame candidate. 
 
     
     
       12. The device of  claim 11 ,
 wherein a difference between a roll of the specific reference frame and a roll of the current frame is less than a preset threshold. 
 
     
     
       13. The device of  claim 9 ,
 wherein the reference frame candidate is composed of one or more frames having a center point within the scanning zone, 
 wherein the scanning zone corresponds to twice a height of the current frame and twice a width of the current frame. 
 
     
     
       14. The device of  claim 9 ,
 wherein the processor selecting a prediction mode for encoding the current frame based on information included in the specific reference frame comprising: 
 selecting a specific reference block among blocks comprising the specific reference frame corresponding to a specific current block among blocks comprising the current frame; 
 encoding the specific current block based on a motion vector of the specific reference block. 
 
     
     
       15. The device of  claim 14 ,
 wherein the specific reference block is one of blocks of an overlapping region between the current frame and the specific reference frame. 
 
     
     
       16. The device of  claim 15 ,
 where the specific reference block is a block having the smallest SAD (Sum of Absolute Difference) among SADs of blocks of the overlapping region, 
 wherein the SADs are calculated from an upper-left block among blocks of the overlapping region.