Adaptive screen and video coding scheme

Abstract

An adaptive screen and video coding system adaptively selects different coding schemes for coding an image block of a screen or video image based on a type of content included in the image block. The adaptive screen and video coding system further includes a screen coding scheme that classifies pixels of the image block into major colors and escape pixels and convert values of the pixels into index values assigned to the major colors and the escape pixels. Since the number of index values is usually less than a normal range of pixel values, using the index values instead of actual pixel values further facilitate compression and coding of the image block, thus improving coding accuracy and efficiency of the screen or video image. In one embodiment, the adaptive screen and video coding system may be used for screen sharing application.

1 .- 20 . (canceled) 
     
     
         21 . A computer system, comprising one or more processing units and memory, that implements a video encoder configured to perform operations comprising:
 encoding an image to produce encoded data, wherein the image is partitioned into coding units (“CUs”), and wherein the encoding the image includes, for a given CU of the CUs:
 determining N major colors for pixel values of the given CU; 
 assigning a color index value to each of the N major colors and an escape value; 
 converting the pixel values of the given CU to color index values; 
 encoding the color index values for the pixel values of the given CU using one or more of multiple modes, wherein, according to a prediction mode of the multiple modes, for a given pixel value among the pixel values of the given CU, the color index value for a neighboring pixel value is used as the color index value for the given pixel value; and 
 for any of the pixel values of the given CU that is represented with the escape value, encoding the pixel value; and 
   outputting the encoded data.   
     
     
         22 . The computer system of  claim 21 , wherein the operations further comprise:
 encoding information that indicates the N major colors, wherein the encoded data that is output includes the encoded information that indicates the N major colors.   
     
     
         23 . The computer system of  claim 21 , wherein, for a given one of the color index values, a quantization zone includes a range of pixel values around one of the N major colors, and wherein the converting includes representing any of the pixel values of the given CU in the range of pixel values with the given color index value. 
     
     
         24 . The computer system of  claim 21 , wherein each of the N major colors has three color components according to an RGB color model. 
     
     
         25 . The computer system of  claim 21 , wherein the encoding the color index values for the pixel values of the given CU uses, for at least some of the pixel values of the given CU, one bit per pixel value to indicate a selection between the prediction mode and a normal mode. 
     
     
         26 . The computer system of  claim 25 , wherein the encoding the color index values for the pixel values of the given CU further uses a run length to indicate a count of pixel values encoded using the prediction mode or a count of pixel values encoded using the normal mode. 
     
     
         27 . The computer system of  claim 21 , wherein the operations further comprise:
 determining that a screen coding scheme is enabled;   outputting, as part of a sequence parameter set, a flag that indicates the screen coding scheme is enabled;   determining to encode the given CU using the screen coding scheme; and   outputting a flag that indicates the screen coding scheme is used to encode the given CU.   
     
     
         28 . The computer system of  claim 21 , wherein, for any of the pixel values of the given CU that is represented with the escape value, the encoding the pixel value includes quantizing the pixel value and entropy coding the quantized pixel value. 
     
     
         29 . The computer system of  claim 21 , wherein, according to the prediction mode, the neighboring pixel value is immediately above the given pixel value. 
     
     
         30 . The computer system of  claim 21 , wherein the determining N major colors for the pixel values of the given CU includes:
 determining a distribution of the pixel values of the given CU; and   based on the distribution, determining N most common colors among the pixel values of the given CU, for use as the N major colors for the pixel values of the given CU.   
     
     
         31 . The computer system of  claim 21 , wherein the determining N major colors for the pixel values of the given CU includes clustering the pixel values of the given CU to find the N major colors, and wherein the clustering uses k-means clustering. 
     
     
         32 . In a computer system that includes a video encoder, a method comprising:
 encoding an image to produce encoded data, wherein the image is partitioned into coding units (“CUs”), and wherein the encoding the image includes, for a given CU of the CUs:
 determining N major colors for pixel values of the given CU; 
 assigning a color index value to each of the N major colors and an escape value; 
 converting the pixel values of the given CU to color index values; 
 encoding the color index values for the pixel values of the given CU using one or more of multiple modes, wherein, according to a prediction mode of the multiple modes, for a given pixel value among the pixel values of the given CU, the color index value for a neighboring pixel value is used as the color index value for the given pixel value; and 
 for any of the pixel values of the given CU that is represented with the escape value, encoding the pixel value; and 
   outputting the encoded data.   
     
     
         33 . The method of  claim 32 , wherein the encoding the color index values for the pixel values of the given CU uses, for at least some of the pixel values of the given CU, one bit per pixel value to indicate a selection between the prediction mode and a normal mode. 
     
     
         34 . The method of  claim 33 , wherein the encoding the color index values for the pixel values of the given CU further uses a run length to indicate a count of pixel values encoded using the prediction mode or a count of pixel values encoded using the normal mode. 
     
     
         35 . The method of  claim 32 , further comprising:
 determining that a screen coding scheme is enabled;   outputting, as part of a sequence parameter set, a flag that indicates the screen coding scheme is enabled;   determining to encode the given CU using the screen coding scheme; and   outputting a flag that indicates the screen coding scheme is used to encode the given CU.   
     
     
         36 . The method of  claim 32 , wherein, according to the prediction mode, the neighboring pixel value is immediately above the given pixel value. 
     
     
         37 . One or more computer-readable media having stored therein computer-executable instructions for causing one or more processing units, when programmed thereby, to perform operations comprising:
 encoding an image to produce encoded data, wherein the image is partitioned into coding units (“CUs”), and wherein the encoding the image includes, for a given CU of the CUs:
 determining N major colors for pixel values of the given CU; 
 assigning a color index value to each of the N major colors and an escape value; 
 converting the pixel values of the given CU to color index values; 
 encoding the color index values for the pixel values of the given CU using one or more of multiple modes, wherein, according to a prediction mode of the multiple modes, for a given pixel value among the pixel values of the given CU, the color index value for a neighboring pixel value is used as the color index value for the given pixel value; and 
 for any of the pixel values of the given CU that is represented with the escape value, encoding the pixel value; and 
   outputting the encoded data   
     
     
         38 . The one or more computer-readable media of  claim 37 , wherein the encoding the color index values for the pixel values of the given CU uses, for at least some of the pixel values of the given CU, one bit per pixel value to indicate a selection between the prediction mode and a normal mode. 
     
     
         39 . The one or more computer-readable media of  claim 38 , wherein the encoding the color index values for the pixel values of the given CU further uses a run length to indicate a count of pixel values encoded using the prediction mode or a count of pixel values encoded using the normal mode. 
     
     
         40 . The one or more computer-readable media of  claim 37 , wherein, according to the prediction mode, the neighboring pixel value is immediately above the given pixel value.