Reference picture management for layered video

Abstract

An encoded representation ( 60 ) of a picture ( 10 ) of a video stream ( 1 ) is decoded by retrieving buffer description information defining at least one reference picture ( 40, 42 ). The buffer description information is used to determine a picture identifier of a reference picture ( 40, 42 ), which is compared to picture identifiers of reference pictures ( 40, 42 ) stored in a decoded picture buffer ( 230, 350 ). If the reference picture ( 40, 42 ) is determined to be missing based on the comparison of picture identifiers the layer identifier of the reference picture ( 40, 42 ) obtained based on the buffer description information is compared to a layer identifier of the current picture ( 10 ) in order to determine whether the missing reference picture ( 40, 42 ) has been intentionally removed or is unintentionally lost.

1 . A method of decoding an encoded representation of a picture of a video stream of multiple pictures comprising:
 retrieving buffer description information defining at least one reference picture from said encoded representation of said picture;   determining, based on said buffer description information, a picture identifier identifying a reference picture of said at least one reference picture as decoding reference for said picture or for a subsequent picture of said video stream;   determining said reference picture identified by said picture identifier as missing if said picture identifier is not equal to any picture identifier of reference pictures stored in a decoded picture buffer;   comparing, if said reference picture identified by said picture identifier is determined as missing, a layer identifier of said picture with a layer identifier, retrieved based on said buffer description information, of said reference picture identified by said picture identifier; and   performing at least one of:
 determining said reference picture identified by said picture identifier as unintentionally lost if said layer identifier of said reference picture identified by said picture identifier indicates a lower layer or an equal layer than said layer identifier of said picture; and 
   determining said reference picture identified by said picture identifier as intentionally removed if said layer identifier of said reference picture identified by said picture identifier indicates a higher layer than said layer identifier of said picture.   
     
     
         2 . The method according to  claim 1 , further comprising preventing generation of a non-existing reference picture in said decoded picture buffer if said layer identifier of said reference picture identified by said picture identifier indicates a higher layer than said layer identifier of said picture. 
     
     
         3 . The method according to  claim 1 , further comprising generating a non-existing reference picture in said decoded picture buffer if said layer identifier of said reference picture identified by said picture identifier indicates a lower layer or an equal layer than said layer identifier of said picture. 
     
     
         4 . The method according to  claim 3 , further comprising storing said non-existing reference picture in said decoded picture buffer. 
     
     
         5 . The method according to  claim 3 , wherein generating said non-existing reference picture comprises assigning said picture identifier and said layer identifier retrieved based on said buffer description information to said non-existing reference picture. 
     
     
         6 . The method according to  claim 3 , wherein generating said non-existing reference picture is performed prior to decoding said picture. 
     
     
         7 . The method according to  claim 1 , further comprising performing reference picture list initialization by ordering reference pictures in a reference picture list according to an order that said buffer description information defines, wherein said reference picture determined as intentionally removed is excluded from said reference picture list initialization. 
     
     
         8 . The method according to  claim 1 , wherein
 comparing said layer identifier comprises comparing, if said reference picture identified by said picture identifier is determined as missing, a temporal layer identifier of said picture with a temporal layer identifier, retrieved based on said buffer description information, of said reference picture identified by said picture identifier; and further comprising:   performing at least one of:
 determining said reference picture identified by said picture identifier as unintentionally lost if said temporal layer identifier of said reference picture identified by said picture identifier is lower than or equal to said temporal layer identifier of said picture; and 
 determining said reference picture comprises determining said reference picture identified by said picture identifier as intentionally removed if said temporal layer identifier of said reference picture identified by said picture identifier is higher than said temporal layer identifier of said picture. 
   
     
     
         9 . A decoder configured to decode an encoded representation of a picture of a video stream of multiple pictures, comprising:
 a data retriever configured to retrieve buffer description information defining at least one reference picture from said encoded representation of said picture;   a picture identifier determiner configured to determine, based on said buffer description information, a picture identifier identifying a reference picture of said at least one reference picture as decoding reference for said picture or for a subsequent picture of said video stream;   a picture determiner configured to determine said reference picture identified by said picture identifier as missing if said picture identifier is not equal to any picture identifier of reference pictures stored in a decoded picture buffer;   a layer identifier comparator configured to compare, if said reference picture identified by said picture identifier is determined as missing by said picture determiner, a layer identifier of said picture with a layer identifier, retrieved based on said buffer description information, of said reference picture identified by said picture identifier; and   a generator controller configured to perform at least one of:
 determine said reference picture identified by said picture identifier as unintentionally lost if said layer identifier of said reference picture identified by said picture identifier indicates a lower layer or an equal layer than said layer identifier of said picture; and 
 determine said reference picture identified by said picture identifier as intentionally removed if said layer identifier of said reference picture identified by said picture identifier indicates a higher layer than said layer identifier of said picture. 
   
     
     
         10 . The decoder according to  claim 9 , wherein said generator controller is configured to prevent a picture generator from generating a non-existing reference picture in said decoded picture buffer if said layer identifier of said reference picture identified by said picture identifier indicates a higher layer than said layer identifier of said picture. 
     
     
         11 . The decoder according to  claim 9 , wherein said generator controller is configured to control a picture generator to generate a non-existing reference picture in said decoded picture buffer if said layer identifier of said reference picture identified by said picture identifier indicates a lower layer or an equal layer than said layer identifier of said picture. 
     
     
         12 . The decoder according to  claim 11 , further comprising a buffer manager configured to store said non-existing reference picture in said decoded picture buffer. 
     
     
         13 . The decoder according to  claim 11 , wherein said picture generator is configured to assign said picture identifier and said layer identifier retrieved based on said buffer description information to said non-existing reference picture. 
     
     
         14 . The decoder according to  claim 11 , wherein said picture generator is configured to generate said non-existing reference picture prior to said decoder decoding said picture. 
     
     
         15 . The decoder according to  claim 9 , further comprising a list manager configured to perform reference picture list initialization by ordering reference pictures in a reference picture list according to an order that said buffer description information defines, wherein said reference picture determined as intentionally removed is excluded from said reference picture list initialization. 
     
     
         16 . The decoder according to  claim 9 , wherein
 said layer identifier comparator is configured to compare, if said reference picture identified by said picture identifier is determined as missing by said picture determiner, a temporal layer identifier of said picture with a temporal layer identifier, retrieved based on said buffer description information, of said reference picture identified by said picture identifier; and   said generator controller is configured to perform at least one of:
 determine said reference picture identified by said picture identifier as unintentionally lost if said temporal layer identifier of said reference picture identified by said picture identifier is lower than or equal to said temporal layer identifier of said picture; and 
 determine said reference picture identified by said picture identifier as intentionally removed if said temporal layer identifier of said reference picture identified by said picture identifier is higher than said temporal layer identifier of said picture. 
   
     
     
         17 . A receiver comprising:
 an input section configured to receive encoded representations of multiple pictures of a video stream;   a decoder configured to decode an encoded representation of a picture of a video stream of multiple pictures, comprising:   a data retriever configured to retrieve buffer description information defining at least one reference picture from said encoded representation of said picture;   a picture identifier determiner configured to determine, based on said buffer description information, a picture identifier identifying a reference picture of said at least one reference picture as decoding reference for said picture or for a subsequent picture of said video stream;   a picture determiner configured to determine said reference picture identified by said picture identifier as missing if said picture identifier is not equal to any picture identifier of reference pictures stored in a decoded picture buffer;   a layer identifier comparator configured to compare, if said reference picture identified by said picture identifier is determined as missing by said picture determiner, a layer identifier of said picture with a layer identifier, retrieved based on said buffer description information, of said reference picture identified by said picture identifier; and   a generator controller configured to perform at least one of:
 determine said reference picture identified by said picture identifier as unintentionally lost if said layer identifier of said reference picture identified by said picture identifier indicates a lower layer or an equal layer than said layer identifier of said picture; 
 determine said reference picture identified by said picture identifier as intentionally removed if said layer identifier of said reference picture identified by said picture identifier indicates a higher layer than said layer identifier of said picture; and 
   an output section configured to output decoded pictures of said video stream.   
     
     
         18 . A decoder comprising:
 an input section configured to receive encoded representations of multiple pictures of a video stream;   a processor configured to process code of a computer program stored in a memory, said code causing, when run on said processor, said processor to:
 retrieve buffer description information defining at least one reference picture from said encoded representation of said picture; 
 determine, from said buffer description information, a picture identifier identifying a reference picture of said at least one reference picture as decoding reference for said picture or for a subsequent picture of said video stream; 
 determine said reference picture identified by said picture identifier as missing if said picture identifier is not equal to any picture identifier of reference pictures stored in a decoded picture buffer; 
 compare, if said reference picture identified by said picture identifier is determined as missing, a layer identifier of said picture with a layer identifier, retrieved based on said buffer description information, of said reference picture identified by said picture identifier; and 
 perform at least one of:
 determine said reference picture identified by said picture identifier as unintentionally lost if said layer identifier of said reference picture identified by said picture identifier indicates a lower layer or an equal layer than said layer identifier of said picture; and 
 determine said reference picture identified by said picture identifier as intentionally removed if said layer identifier of said reference picture identified by said picture identifier indicates a higher layer than said layer identifier of said picture; and 
 
   an output section configured to output decoded pictures of said video stream.   
     
     
         19 . A method of processing a video stream comprising encoded representations of pictures comprising:
 retrieving, from an encoded representation of a picture in said video stream, a picture identifier and a layer identifier of said picture;   retrieving buffer description information defining at least one reference picture from said encoded representation of said picture;   determining, based on said buffer description information, a picture identifier identifying a reference picture of said at least one reference picture as decoding reference for said picture or for a subsequent picture of said video stream;   comparing said picture identifier identifying said reference picture with picture identifiers retrieved from previously forwarded encoded representations of pictures in said video stream;   comparing, if said picture identifier identifying said reference picture is not equal to any of said picture identifiers retrieved from previously forwarded encoded representations of pictures in said video stream, a layer identifier of said reference picture with said layer identifier of said picture; and   removing said encoded representation of said picture from said video stream if said layer identifier of said reference picture indicates a lower layer or an equal layer than said layer identifier of said picture.   
     
     
         20 . The method according to  claim 19 , further comprising:
 receiving said video stream from an encoder; and   forwarding said video stream to a decoder.   
     
     
         21 . A network node configured to process a video stream comprising encoded representations of pictures, said network node comprising:
 a data retriever configured to retrieve, from an encoded representation of a picture in said video stream, a picture identifier and a layer identifier of said picture and buffer description information defining at least one reference picture from said encoded representation of said picture;   a picture identifier determiner configured to determine, based on said buffer description information, a picture identifier identifying a reference picture of said at least one reference picture as decoding reference for said picture or for a subsequent picture of said video stream;   a picture identifier comparator configured to compare said picture identifier identifying said reference picture with picture identifiers retrieved from previously forwarded encoded representations of pictures in said video stream;   a layer identifier comparator configured to compare, if said picture identifier identifying said reference picture is not equal to any of said picture identifiers retrieved from previously forwarded encoded representations of pictures in said video stream, a layer identifier of said reference picture with said layer identifier of said picture; and   a data remover configured to remove said encoded representation of said picture from said video stream if said layer identifier of said reference picture indicates a lower layer or an equal layer than said layer identifier of said picture.   
     
     
         22 . The network node according to  claim 21 , further comprising:
 a receiver configured to receive said video stream from an encoder; and   a transmitter configured to forward said video stream to a decoder.