Mixing of encoded video streams

Abstract

Describe is a video stream mixer ( 60 ) for generating a mixed encoded video stream from an encoded primary video stream associated with a primary window, and at least one encoded secondary video stream, each associated with a corresponding secondary window that is smaller than the primary window. The mixer includes the following elements: A video stream splitter ( 20 ) configured to split the encoded primary video stream into an encoded unmixed sub-stream associated with a sub-window in the primary window and not involved in mixing, and an encoded complementary sub-stream associated with a sub-window complementary to the other sub-window in the primary window and involved in mixing; A video stream decoder ( 22 ) configured to decode the complementary sub-stream and each secondary video stream; A video signal mixer ( 24 ) configured to mix the decoded complementary sub-stream with each decoded secondary video stream to form a mixed video signal; A video encoder ( 26 ) configured to encode the mixed video signal into an encoded mixed sub-stream; A video stream combiner ( 28 ) configured to combine the encoded unmixed sub-stream with the encoded mixed sub-stream.

The invention claimed is: 
     
       1. A method of generating a mixed encoded video stream from an encoded primary video stream associated with a primary window and at least one encoded secondary video stream, each encoded secondary video stream associated with a corresponding secondary window that is smaller than the primary window, said method comprising:
 splitting the encoded primary video stream into an encoded unmixed sub-stream associated with a sub-window in the primary window and not involved in mixing, and an encoded complementary sub-stream associated with a sub-window complementary to the sub-window in the primary window and involved in mixing; 
 decoding the complementary sub-stream and each encoded secondary video stream; 
 mixing the decoded complementary sub-stream with each decoded secondary video stream to form a mixed video signal; 
 encoding the mixed video signal into an encoded mixed sub-stream; and 
 combining the encoded unmixed sub-stream with the encoded mixed sub-stream. 
 
     
     
       2. The method of  claim 1 , further comprising:
 determining macro blocks involved in mixing and macro blocks not involved in mixing; and 
 separating the encoded primary video stream into at least one unmixed slice of macro blocks not involved in mixing and at least one complementary slice of macro blocks involved in mixing. 
 
     
     
       3. The method of  claim 2 , further comprising:
 renumbering macro blocks of each complementary slice before decoding; and 
 resetting a numbering of the renumbered macro blocks after mixing and encoding. 
 
     
     
       4. A video stream mixer for generating a mixed encoded video stream from an encoded primary video stream associated with a primary window, and at least one encoded secondary video stream, each encoded secondary video stream associated with a corresponding secondary window that is smaller than the primary window, said mixer comprising:
 a video stream splitter configured to split the encoded primary video stream into
 an encoded unmixed sub-stream associated with a sub-window in the primary window and not involved in mixing, and 
 an encoded complementary sub-stream associated with a sub-window complementary to the sub-window in the primary window and involved in mixing; 
 
 a video stream decoder configured to decode the complementary sub-stream and each encoded secondary video stream; 
 a video signal mixer configured to mix the decoded complementary sub-stream with each decoded secondary video stream to form a mixed video signal; 
 a video encoder configured to encode the mixed video signal into an encoded mixed sub-stream; and 
 a video stream combiner configured to combine the encoded unmixed sub-stream with the encoded mixed sub-stream. 
 
     
     
       5. The video stream mixer of  claim 4 , comprising:
 a mix zone detector configured to determine macro blocks involved in mixing and macro blocks not involved in mixing; and 
 a slice separator configured to separate the encoded primary video stream into at least one unmixed slice of macro blocks not involved in mixing and at least one complementary slice of macro blocks involved in mixing. 
 
     
     
       6. The video stream mixer of  claim 5 , wherein the slice separator includes a macro block renumbering unit configured to renumber macro blocks of each complementary slice before decoding. 
     
     
       7. The video stream mixer of  claim 6 , wherein the video stream combiner includes:
 a macro block numbering reset unit configured to reset a numbering of the renumbered macro blocks after mixing and encoding; and 
 a slice combiner configured to combine unmixed and mixed slices. 
 
     
     
       8. A media resource device, comprising:
 a video stream mixer configured to generate a mixed encoded video stream from an encoded primary video stream associated with a primary window, and at least one encoded secondary video stream, each encoded secondary video stream associated with a corresponding secondary window that is smaller than the primary window, the video stream mixer comprising:
 a video stream splitter configured to split the encoded primary video stream into
 an encoded unmixed sub-stream associated with a sub-window in the primary window and not involved in mixing, and 
 an encoded complementary sub-stream associated with a sub-window complementary to the sub-window in the primary window and involved in mixing; 
 
 a video stream decoder configured to decode the complementary sub-stream and each encoded secondary video stream; 
 a video signal mixer configured to mix the decoded complementary sub-stream with each decoded secondary video stream to form a mixed video signal; 
 a video encoder configured to encode the mixed video signal into an encoded mixed sub-stream; and 
 a video stream combiner configured to combine the encoded unmixed sub-stream with the encoded mixed sub-stream. 
 
 
     
     
       9. A non-transitory computer readable storage medium storing a computer program comprising program instructions which, when executed on a media resource function processor (MRFP), causes the MRFP to:
 generate a mixed encoded video stream from an encoded primary video stream associated with a primary window, and at least one encoded secondary video stream, each encoded secondary video stream associated with a corresponding secondary window that is smaller than the primary window; 
 split the encoded primary video stream into
 an encoded unmixed sub-stream associated with a sub-window in the primary window and not involved in mixing, and 
 an encoded complementary sub-stream associated with a sub-window complementary to the sub-window in the primary window and involved in mixing; 
 
 decode the complementary sub-stream and each encoded secondary video stream; 
 mix the decoded complementary sub-stream with each decoded secondary video stream to form a mixed video signal; 
 encode the mixed video signal into an encoded mixed sub-stream; and 
 combine the encoded unmixed sub-stream with the encoded mixed sub-stream. 
 
     
     
       10. A video conferencing system comprising a media resource device, the media resource device comprising:
 a video stream mixer configured to generate a mixed encoded video stream from an encoded primary video stream associated with a primary window, and at least one encoded secondary video stream, each encoded secondary video stream associated with a corresponding secondary window that is smaller than the primary window, the video stream mixer comprising:
 a video stream splitter configured to split the encoded primary video stream into an encoded unmixed sub-stream associated with a sub-window in the primary window and not involved in mixing, and
 an encoded complementary sub-stream associated with a sub-window complementary to the sub-window in the primary window and involved in mixing; 
 
 a video stream decoder configured to decode the complementary sub-stream and each encoded secondary video stream; 
 a video signal mixer configured to mix the decoded complementary sub-stream with each decoded secondary video stream to form a mixed video signal; 
 a video encoder configured to encode the mixed video signal into an encoded mixed sub-stream; and 
 a video stream combiner configured to combine the encoded unmixed sub-stream with the encoded mixed sub-stream. 
 
 
     
     
       11. A video conferencing system comprising a video stream mixer configured to generate a mixed encoded video stream from an encoded primary video stream associated with a primary window, and at least one encoded secondary video stream, each encoded secondary video stream associated with a corresponding secondary window that is smaller than the primary window, the video stream mixer comprising:
 a video stream splitter configured to split the encoded primary video stream into
 an encoded unmixed sub-stream associated with a sub-window in the primary window and not involved in mixing, and 
 an encoded complementary sub-stream associated with a sub-window complementary to the sub-window in the primary window and involved in mixing; 
 
 a video stream decoder configured to decode the complementary sub-stream and each encoded secondary video stream; 
 a video signal mixer configured to mix the decoded complementary sub-stream with each decoded secondary video stream to form a mixed video signal; 
 a video encoder configured to encode the mixed video signal into an encoded mixed sub-stream; and 
 a video stream combiner configured to combine the encoded unmixed sub-stream with the encoded mixed sub-stream.