Apparatus, a method and a computer program for video coding and decoding

Abstract

A method comprising: encoding a first picture on a first scalability layer and on a lowest temporal sub-layer; encoding a second picture on a second scalability layer and on the lowest temporal sub-layer, wherein the first picture and the second picture represent the same time instant, encoding one or more first syntax elements, associated with the first picture, with a value indicating that a picture type of the first picture is other than a step-wise temporal sub-layer access (STSA) picture; encoding one or more second syntax elements, associated with the second picture, with a value indicating that a picture type of the second picture is a step-wise temporal sub-layer access picture; and encoding at least a third picture on a second scalability layer and on a temporal sub-layer higher than the lowest temporal sub-layer.

The invention claimed is: 
     
       1. A method comprising:
 receiving coded pictures of a first scalability layer; 
 decoding the coded pictures of the first scalability layer; 
 receiving coded pictures of a second scalability layer, the second scalability layer depending on the first scalability layer; 
 selecting a layer access picture on the second scalability layer from the coded pictures of the second scalability layer, wherein the selected layer access picture is a step-wise temporal sub-layer access (STSA) picture on a lowest temporal sub-layer, wherein receiving coded pictures of the second scalability layer comprises receiving the STSA step-wise temporal sub-layer access picture on the lowest sub-layer in a Supplemental Enhancement Information (SEI) supplemental enhancement information message that also defines a number of sub-layers that are decodable when the decoding starts from the selected layer access picture; 
 ignoring coded pictures on a the second scalability layer prior to, in decoding order, the selected layer access picture; 
 decoding the selected layer access picture; and 
 starting step-wise decoding of at least one enhancement layer in response to said at least one enhancement layer containing an intra random access point (IRAP) picture or an STSA a step-wise temporal sub-layer access picture on the lowest sub-layer, wherein starting step-wise decoding comprises: 
 when the a current picture is a temporal sub-layer access (TSA) or STSA step-wise temporal sub-layer access picture at a higher sub-layer than the lowest sub-layer and decoding of the next lower sub-layer in the same layer has been started, and decoding of the same sub-layer of all the reference layers of the TSA temporal sub-layer access or STSA step-wise temporal sub-layer access picture has been started, the TSA temporal sub-layer access or STSA step-wise temporal sub-layer access picture and all pictures at the same sub-layer following the TSA temporal sub-layer access or STSA step-wise temporal sub-layer access picture, in decoding order, in the same layer are decoded. 
 
     
     
       2. The method according  claim 1 , wherein said selecting comprises concluding that a picture is a step-wise temporal sub-layer access picture on the basis of its Network Access Layer (NAL) network abstraction layer unit type indicating the step-wise temporal sub-layer access picture and its temporal sub-layer identifier indicating the lowest temporal sub-layer. 
     
     
       3. The method according to  claim 1 , the method further comprising
 starting decoding of the a bitstream in response to a base layer containing an intra random access point (IRAP) picture or a step-wise temporal sub-layer access (STSA) picture on the lowest sub-layer; and 
 increasing progressively the number of decoded layers and/or the number of decoded temporal sub-layers. 
 
     
     
       4. The method according to  claim 3 , wherein starting the stepwise decoding further comprises the following conditional operation:
 when a the current picture is an IRAP intra random access point picture and decoding of all reference layers of the IRAP intra random access point picture has been started, the IRAP intra random access point picture and all pictures following it, in decoding order, in the same layer are decoded. 
 
     
     
       5. A method comprising
 receiving coded pictures of a first scalability layer; 
 receiving coded pictures of a second scalability layer, the second scalability layer depending on the first scalability layer; 
 selecting a layer access picture on the second scalability layer from the coded pictures of a the second scalability layer, wherein the selected layer access picture is a step-wise temporal sub-layer access picture on the lowest temporal sub-layer, wherein the step-wise temporal sub-layer access picture provides an access point for layer-wise initialization of decoding of a bitstream with one or more temporal sub-layers; 
 ignoring coded pictures on a the second scalability layer prior to, in decoding order, the selected layer access picture; 
 sending the coded pictures of the first scalability layer and the selected layer access picture in a the bitstream; and 
 signaling the step-wise temporal sub-layer access picture in a Supplemental Enhancement Information (SEI) supplemental enhancement information message that also defines the number of decodable sub-layers when the decoding starts from the selected layer access picture. 
 
     
     
       6. An apparatus comprising:
 at least one processor and at least one memory, said at least one memory stored with code thereon, which when executed by said at least one processor, causes an apparatus to perform at least: 
 receiving coded pictures of a first scalability layer; 
 decoding the coded pictures of the first scalability layer; 
 receiving coded pictures of a second scalability layer, the second scalability layer depending on the first scalability layer; 
 selecting a layer access picture on the second scalability layer from the coded pictures of a the second scalability layer, wherein the selected layer access picture is a step-wise temporal sub-layer access (STSA) picture on the lowest temporal sub-layer, wherein receiving coded pictures of the second scalability layer comprises receiving the STSA step-wise temporal sub-layer access picture on the lowest sub-layer in a Supplemental Enhancement Information (SEI) supplemental enhancement information message that also defines a number of sub-layers that are decodable when the decoding starts from the selected layer access picture; 
 ignoring coded pictures on a the second scalability layer prior to, in decoding order, the selected layer access picture; 
 decoding the selected layer access picture; and 
 starting step-wise decoding of at least one enhancement layer in response to said at least one enhancement layer containing an intra random access point (IRAP) picture or an STSA a step-wise temporal sub-layer access picture on the lowest sub-layer, wherein starting step-wise decoding comprises: 
 when the a current picture is a temporal sub-layer access (TSA) or STSA step-wise temporal sub-layer access picture at a higher sub-layer than the lowest sub-layer and decoding of the next lower sub-layer in the same layer has been started, and decoding of the same sub-layer of all the reference layers of the TSA temporal sub-layer access or STSA step-wise temporal sub-layer access picture has been started, the TSA temporal sub-layer access or STSA step-wise temporal sub-layer access picture and all pictures at the same sub-layer following the TSA temporal sub-layer access or STSA step-wise temporal sub-layer access picture, in decoding order, in the same layer are decoded. 
 
     
     
       7. An apparatus comprising:
 at least one processor and at least one memory, said at least one memory stored with code thereon, which when executed by said at least one processor, causes an the apparatus to perform at least:  
 receiving coded pictures of a first scalability layer; 
 receiving coded pictures of a second scalability layer, the second scalability layer depending on the first scalability layer; 
 selecting a layer access picture on the second scalability layer from the coded pictures of a the second scalability layer, wherein the selected layer access picture is a step-wise temporal sub-layer access picture on the lowest temporal sub-layer, wherein the step-wise temporal sub-layer access picture provides an access point for layer-wise initialization of decoding of a bitstream with one or more temporal sub-layers; 
 ignoring coded pictures on a the second scalability layer prior to, in decoding order, the selected layer access picture; 
 sending the coded pictures of the first scalability layer and the selected layer access picture in a the bitstream; and 
 signaling the step-wise temporal sub-layer access picture in a Supplemental Enhancement Information (SEI) supplemental enhancement information message that also defines the number of decodable sub-layers when the decoding starts from the selected layer access picture. 
 
     
     
       8. A method comprising:
 encoding a first picture on a first scalability layer and on a lowest temporal sub-layer; 
 encoding a second picture on a second scalability layer and on the lowest temporal sub-layer, wherein the first picture and the second picture represent the same time instant, 
 encoding one or more first syntax elements, associated with the first picture, with a value indicating that a picture type of the first picture is other than a step-wise temporal sub-layer access (STSA) picture; 
 encoding one or more second syntax elements, associated with the second picture, with a value indicating that a picture type of the second picture is a step-wise temporal sub-layer access picture, wherein the step-wise temporal sub-layer access picture is the step-wise temporal sub-layer access picture on a the lowest temporal sub-layer and provides an access point for layer-wise initialization of decoding of a bitstream with one or more temporal sub-layers; 
 encoding at least a third picture on a the second scalability layer and on a temporal sub-layer higher than the lowest temporal sub-layer; and 
 signaling the step-wise temporal sub-layer access picture in a Supplemental Enhancement Information (SEI) supplemental enhancement information message that also defines the number of decodable sub-layers when the decoding starts from the a selected layer access picture. 
 
     
     
       9. The method according to  claim 8 , the method further comprising
 signaling the step-wise temporal sub-layer access picture in the bitstream by a specific NAL network abstraction layer unit type. 
 
     
     
       10. The method according to  claim 8 , the method further comprising
 encoding said second or any further scalability layer to comprise more frequent TSA temporal sub-layer access or STSA step-wise temporal sub-layer access pictures than the first scalability layer. 
 
     
     
       11. An apparatus comprising:
 at least one processor and at least one memory, said at least one memory stored with code thereon, which when executed by said at least one processor, causes an the apparatus to perform at least: 
 encoding a first picture on a first scalability layer and on a lowest temporal sub-layer; 
 encoding a second picture on a second scalability layer and on the lowest temporal sub-layer, 
 wherein the first picture and the second picture represent the same time instant, 
 encoding one or more first syntax elements, associated with the first picture, with a value indicating that a picture type of the first picture is other than a step-wise temporal sub-layer access picture; 
 encoding one or more second syntax elements, associated with the second picture, with a value indicating that a picture type of the second picture is a step-wise temporal sub-layer access picture, wherein the step-wise temporal sub-layer access picture is the step-wise temporal sub-layer access picture on a the lowest temporal sub-layer and provides an access point for layer-wise initialization of decoding of a bitstream with one or more temporal sub-layers; 
 encoding at least a third picture on a the second scalability layer and on a temporal sub-layer higher than the lowest temporal sub-layer; and 
 signaling the step-wise temporal sub-layer access picture in a Supplemental Enhancement Information (SEI) supplemental enhancement information message that also defines the number of decodable sub-layers when the decoding starts from the a selected layer access picture. 
 
     
     
       12. The apparatus according to  claim 11 , further comprising code causing the apparatus to perform signaling the step-wise temporal sub-layer access picture in the bitstream by a specific NAL network abstraction layer unit type. 
     
     
       13. The apparatus according to  claim 11 , further comprising code causing the apparatus to perform encoding said second or any further scalability layer to comprise more frequent TSA temporal sub-layer access or STSA step-wise temporal sub-layer access pictures than the first scalability layer. 
     
     
       14. A non-transitory computer readable storage medium stored with code thereon for use by an apparatus, which when executed by a processor, causes the apparatus to perform:
 encoding a first picture on a first scalability layer and on a lowest temporal sub-layer; 
 encoding a second picture on a second scalability layer and on the lowest temporal sub-layer, wherein the first picture and the second picture represent the same time instant, 
 encoding one or more first syntax elements, associated with the first picture, with a value indicating that a picture type of the first picture is other than a step-wise temporal sub-layer access picture; 
 encoding one or more second syntax elements, associated with the second picture, with a value indicating that a picture type of the second picture is a step-wise temporal sub-layer access picture, wherein the step-wise temporal sub-layer access picture is the step-wise temporal sub-layer access picture on a the lowest temporal sub-layer and provides an access point for layer-wise initialization of decoding of a bitstream with one or more temporal sub-layers; 
 encoding at least a third picture on a the second scalability layer and on a temporal sub-layer higher than the lowest temporal sub-layer; and 
 signaling the step-wise temporal sub-layer access picture in a Supplemental Enhancement Information (SEI) supplemental enhancement information message that also defines the number of decodable sub-layers when the decoding starts from the a selected layer access picture. 
 
     
     
       15. A method according to claim 1, wherein the coded pictures on the second scalability layer that are ignored are not enabled for decoding and are not specified for output.  
     
     
       16. A method according to claim 5, wherein the coded pictures on the second scalability layer that are ignored are not enabled for decoding and are not specified for output.  
     
     
       17. An apparatus according to claim 6, wherein the coded pictures on the second scalability layer that are ignored are not enabled for decoding and are not specified for output.  
     
     
       18. An apparatus according to claim 7, wherein the coded pictures on the second scalability layer that are ignored are not enabled for decoding and are not specified for output.  
     
     
       19. A method comprising:
 decoding coded pictures of a first scalability layer;   receiving coded pictures of a second scalability layer, the second scalability layer depending on the first scalability layer;   selecting a layer access picture on the second scalability layer from the coded pictures of the second scalability layer, wherein the selected layer access picture is a step-wise temporal sub-layer access picture on a lowest temporal sub-layer, wherein receiving coded pictures of the second scalability layer comprises receiving the step-wise temporal sub-layer access picture on the lowest sub-layer signaled in a supplemental enhancement information message that also defines a number of sub-layers that are decodable when the decoding starts from the selected layer access picture as the coded pictures on the second scalability layer that are prior to, in decoding order, the selected layer access layer are not enabled for decoding and are not specified for output;   decoding the selected layer access picture; and   starting step-wise decoding of at least one enhancement layer in response to said at least one enhancement layer containing an intra random access point picture or a step-wise temporal sub-layer access picture on the lowest sub-layer, wherein starting step-wise decoding comprises:   when a current picture is a temporal sub-layer access or step-wise temporal sub-layer access picture at a higher sub-layer than the lowest sub-layer and decoding of the next lower sub-layer in the same layer has been started, and decoding of the same sub-layer of all the reference layers of the temporal sub-layer access or step-wise temporal sub-layer access picture has been started, the temporal sub-layer access or step-wise temporal sub-layer access picture and all pictures at the same sub-layer following the temporal sub-layer access or step-wise temporal sub-layer access picture, in decoding order, in the same layer are decoded.