Method and apparatus for video encoding and decoding based on neural network implementation of cabac

Abstract

Methods and apparatuses for video coding and decoding are provided. The method of video encoding includes accessing a bin of a syntax element associated with a block in a picture of a video, determining a context for the bin of the syntax element associated with the syntax element and entropy encoding the bin of the syntax element based on the determined context wherein either the bin of the syntax element is based on the relevance of a prediction by a neural network of the syntax element or the probability associated to the context is determined by a neural network. A bitstream formatted to include encoded data, a computer-readable storage medium and a computer-readable program product are also described.

1 - 17 . (canceled) 
     
     
         18 . A method for video encoding, comprising:
 accessing a bin of a syntax element associated with a current block in a picture of a video;   determining a context for the bin of the syntax element, the context being associated with the syntax element; and   entropy encoding the bin of the syntax element based on the determined context;   wherein determining a context for the bin of the syntax element comprises determining a probability associated with the context by applying a neural network on data encoded prior to the syntax element, the data encoded prior to the syntax element comprising spatial and temporal information relative to a block previously encoded or to the current block.   
     
     
         19 . The method of  claim 18 , wherein the data encoded prior to the syntax element comprises at least one of:
 values of S previously encoded syntax elements of the same type;   values of T previously encoded bin values of the same type;   values of K previous probabilities of syntax elements of the same type;   values of previously coded syntax elements of the same type in a neighborhood of the current block;   reconstructed samples in a L-shape of the current block;   prediction samples of the current block;   reconstructed residuals of the current block;   samples in a L-shape of a reference block for the current block when the current block is coded in inter;   a motion field for the current block when the current block is coded in inter; or   prediction samples of the current block generated with motion compensation using the motion field.   
     
     
         20 . The method of  claim 18 , wherein the neural network is a recursive neural network and comprises long short-term memory (LSTM) units. 
     
     
         21 . The method of  claim 18 , wherein the neural network estimates an increment or decrement and the probability is updated based on the increment or decrement. 
     
     
         22 . The method of  claim 18 , wherein the neural network further determines the context for the bin of the syntax element. 
     
     
         23 . An apparatus for video encoding, comprising:
 a memory; and   one or more processors configured to:
 access a bin of a syntax element associated with a current block in a picture of a video; 
 determine a context for the bin of the syntax element associated with the syntax element; and 
 entropy encode the bin of the syntax element based on the determined context; 
 wherein the one or more processors are further configured to:
 determine by a neural network, a probability associated with the context responsive to data encoded prior to the syntax element, the data encoded prior to the syntax element comprising spatial and temporal information relative to a block previously encoded or to the current block. 
 
   
     
     
         24 . The apparatus of  claim 23 , wherein the data encoded prior to the syntax element comprises at least one of:
 values of S previously encoded syntax elements of the same type;   values of T previously encoded bin values of the same type;   values of K previous probabilities of syntax elements of the same type;   values of previously coded syntax elements of the same type in a neighborhood of the current block;   reconstructed samples in a L-shape of the current block;   prediction samples of the current block;   reconstructed residuals of the current block;   samples in a L-shape of a reference block for the current block when the current block is coded in inter;   a motion field for the current block when the current block is coded in inter; or   prediction samples of the current block generated with motion compensation using the motion field.   
     
     
         25 . The apparatus of  claim 23 , wherein the neural network is a recursive neural network and comprises long short-term memory (LSTM) units. 
     
     
         26 . The apparatus of  claim 23 , wherein the neural network estimates an increment or decrement and the probability is updated based on the increment or decrement. 
     
     
         27 . The apparatus of  claim 23 , wherein the neural network further determines the context for the bin of the syntax element. 
     
     
         28 . A method for video decoding, comprising:
 accessing an encoded bin of a syntax element associated with a current block in a picture of an encoded video;   determining a context for the bin of the syntax element associated with the syntax element; and   entropy decoding the encoded bin of syntax element based on the determined context;   wherein determining a context for the bin of the syntax element comprises determining a probability associated with the context by applying a neural network on data decoded prior to the syntax element, the data decoded prior to the syntax element comprising spatial and temporal information relative to a block previously decoded or to the current block.   
     
     
         29 . The method of  claim 28 , wherein the data decoded prior to the syntax element comprises at least one of:
 values of S previously decoded syntax elements of the same type;   values of T previously decoded bin values of the same type;   values of K previous probabilities of syntax elements of the same type;   values of previously decoded syntax elements of the same type in a neighborhood of the current block;   reconstructed samples in a L-shape of the current block;   prediction samples of the current block;   reconstructed residuals of the current block;   samples in a L-shape of a reference block for the current block when the current block is coded in inter;   a motion field for the current block when the current block is coded in inter; or   prediction samples of the current block generated with motion compensation using the motion field.   
     
     
         30 . The method of  claim 28 , wherein the neural network is a recursive neural network and comprises long short-term memory (LSTM) units. 
     
     
         31 . The method of  claim 28 , wherein the neural network estimates an increment or decrement and the probability is updated based on the increment or decrement. 
     
     
         32 . The method of  claim 28 , wherein the neural network further determines the context for the bin of the syntax element. 
     
     
         33 . An apparatus for video decoding, comprising:
 a memory; and   one or more processors configured to:
 access an encoded bin of a syntax element associated with a current block in a picture of an encoded video; 
 determine a context for the bin of the syntax element associated with the syntax element; and 
 entropy decode the encoded bin of the syntax element based on the determined context; 
 wherein said one or more processors are further configured to:
 determine by a neural network, a probability associated with the context responsive to data decoded prior to the syntax element, the data decoded prior to the syntax element comprising spatial and temporal information relative to a block previously decoded or to the current block. 
 
   
     
     
         34 . The apparatus of  claim 33 , wherein the data decoded prior to the syntax element comprises at least one of:
 values of S previously decoded syntax elements of the same type;   values of T previously decoded bin values of the same type;   values of K previous probabilities of syntax elements of the same type;   values of previously decoded syntax elements of the same type in a neighborhood of the current block;   reconstructed samples in a L-shape of the current block;   prediction samples of the current block;   reconstructed residuals of the current block;   samples in a L-shape of a reference block for the current block when the current block is coded in inter;   a motion field for the current block when the current block is coded in inter; or   prediction samples of the current block generated with motion compensation using the motion field.   
     
     
         35 . The apparatus of  claim 33 , wherein the neural network is a recursive neural network and comprises long short-term memory (LSTM) units. 
     
     
         36 . The apparatus of  claim 33 , wherein the neural network estimate an increment or decrement and the probability is updated based on the increment or decrement. 
     
     
         37 . The apparatus of  claim 33 , wherein the neural network further determines the context for the bin of the syntax element.