System and method for scalable portrait video

Abstract

Generation, coding and transmission of an effective video form, scalable portrait video. As an expansion to bi-level video, portrait video is composed of more gray levels, and therefore possesses higher visual quality while it maintains a low bit rate and low computational costs. Portrait video is a scalable video in that each video with a higher level always contains all the information of the video with a lower level. The bandwidths of 2-4 level portrait videos fit into the bandwidth range of 20-40 Kbps that GPRS and CDMA 1X can stably provide. Therefore, portrait video is very promising for video broadcast and communication on 2.5 G wireless networks. With portrait video technology, this system and method is the first to enable two-way video conferencing on Pocket PCs and Handheld PCs.

1 . A computer-implemented process for generating and transmitting a scalable portrait video, comprising the following process actions:
 inputting a video signal;   if an available bandwidth is between about 10-20 Kbps, then converting said input video signal to a bi-level video format and transmitting it; and   if the available bandwidth is between about 20-40 Kbps, then converting said input video to a portrait video to display more grayscale levels than said bi-level video based on said available bandwidth and transmitting it.   
   
   
       2 . A computer-implemented process for creating a multi-level portrait video, comprising the following process actions:
 obtaining a frame in grayscale format;   converting the frame in grayscale format to a frame in 2 i  level video format, to obtain a bit stream of the frame in 2 i  level video and 2 i−1  partial images of the frame in grayscale format;   dividing each partial image into two smaller partial images using 2 i−1  thresholds;   converting each of said smaller partial images into a bi-level image using a threshold;   combining the smaller partial bi-level images to obtain a combined bi-level image; and   combining said combined bi-level image into the lowest bit plane of a frame of a 2 i+1  level video with the bit stream of the frame of the 2 i  level video.   
   
   
       3 . The computer-implemented process of  claim 2  wherein combining said combined bi-level image into the lowest bit plane of a frame of a 2 i+1  level video with the bit stream of the frame of the of the 2 i  level video, comprises the process action of:
 combining the highest i bit planes of each frame of the 2 i  level video and the lowest bit plane of each frame of the combined bi-level image to obtain each frame of the 2 i+1  level video.   
   
   
       4 . The computer-implemented process of  claim 2  further comprising the process action of:
 compressing the combined bi-level image combined into the lowest bit plane of a frame of the 2 i+1  level video and the bit stream of the frame of the 2 i  level video prior to combining the highest i bit planes of each frame of the 2 i  level video and the lowest bit plane of each frame of the combined bi-level image.   
   
   
       5 . The computer-implemented process of  claim 2  further comprising the process action of at least one of:
 transmitting the compressed bit streams; or   storing the compressed bit streams.   
   
   
       6 . The computer-implemented process of  claim 2  further comprising the process actions of:
 obtaining the at least one of transmitted or stored compressed bit streams;   decompressing the combined bi-level image compressed into the lowest bit plane of a frame of a 2 i−1  level video and the bit stream of the frame of the 2 i  level video prior to combining the highest i bit plane of each frame of the 2 i  level video and the lowest bit plane of each frame of the combined bi-level image to obtain each frame of the 2 i+1  level video.   
   
   
       7 . The computer-implemented process of  claim 2  further comprising the process action of:
 downsampling bit rates of the third or higher bit planes.   
   
   
       8 . The computer-implemented process of  claim 7  wherein said down sampling process action comprises:
 dividing the 2 i+1  level video into 2 by 2 pixel blocks;   if the values of the four pixels in all bit planes that are higher than the current bit plane are the same, regarding the average binary value of the current bit plane as the binary value of all four pixels in the current bit plane;   coding the pixels in raster order such that the binary values of all the other three pixels in the current bit plane are predicted but only the bottom right one is coded; and   not setting the binary values of all the other three pixels in the current plane until the bottom right pixel is decoded.   
   
   
       9 . A computer-implemented process for generating and transmitting a scalable portrait video, comprising the following process actions:
 inputting a video signal;   if an available bandwidth is between about 10-20 Kbps, then converting said input video signal to a bi-level video format and transmitting it;   if the available bandwidth is between about 20-40 Kbps, then converting said input video signal to a scalable portrait video to display more grayscale levels than said bi-level video based on said available bandwidth and transmitting it.   
   
   
       10 . The computer-implemented process of  claim 9  wherein the process action of converting the input video signal to a scalable portrait video, based on available bandwidth, further comprises the process actions of:
 obtaining a frame of video in grayscale format;   applying a first threshold T 1  to said grayscale frame to generate two partial grayscale images, a first of which, S 1 , comprises pixels the non-zero valued ones of which have values greater than said first threshold T 1 , and a second of which, S 2 , comprises pixels the non-zero valued ones of which have values less than said first threshold T 1 ;   generating a first bi-level image comprising pixels assigned a first binary value whenever the value of the correspondingly located pixel in the grayscale frame exceeds T 1 , and comprising pixels assigned a second binary value whenever the value of the correspondingly located pixel in the grayscale frame is less than T 1 ;   applying a second threshold T 21  to the first partial grayscale image, S 1 , to generate a second bi-level image comprising pixels assigned a first binary value whenever the value of the correspondingly located pixel in the first partial grayscale image exceeds T 21  and pixels assigned a second binary value whenever the value of the correspondingly located pixel in the first partial grayscale image is less than T 21 ;   applying a threshold T 22  to the second partial grayscale image, S 2 , to generate a third bi-level image comprising pixels assigned a first binary value whenever the value of the correspondingly located pixel in the second partial grayscale image exceeds T 22  and pixels assigned a second binary value whenever the value of the correspondingly located pixel in the second partial grayscale image is less than T 22 ;   encoding the first, second and third bi-level images; and then   combining the first, second and third bi-level images to create a four level grayscale video frame representing a frame of the scalable portrait video.   
   
   
       11 . The computer-implemented process of  claim 10  wherein the process action of encoding the first, second and third bi-level images comprises:
 combining the second and third bi-level images by adding the pixel values of corresponding pixel locations of these images to create a combined bi-level image;   separately encoding the first bi-level image and the combined bi-level image using a bi-level encoding process.