Systems and Methods for Scalable Composition of Media Streams for Real-Time Multimedia Communication

Abstract

A new approach is proposed that contemplates systems and methods to support the operation of a Virtual Media Room or Virtual Meeting Room (VMR), wherein each VMR can accept from a plurality of participants at different geographic locations a variety of video conferencing feeds from video conference endpoints that can be either proprietary or standards-based and enable a multi-party video conferencing session among the plurality of participants by composing one composite audio and video stream for each of the participants. Each single VMR can be implemented across an infrastructure of globally distributed set of servers/media processing nodes co-located in Points of Presence (POPs) for Internet access. Each VMR also gives its users a rich set of conferencing and collaboration interaction hitherto not experienced by video conferencing users.

1 .- 31 . (canceled) 
     
     
         32 . A video-conferencing system comprising:
 one or more media processing nodes, each node configured to:
 accept a plurality of audio and video streams from a plurality of endpoints in a video conference, at least some of the endpoints being associated with videoconferencing services that are incompatible with each other; and 
 for each of the endpoints in the video conference:
 convert in real-time the plurality of audio and video streams into one or more composite audio and video streams compatible with the endpoint; and 
 provide the one or more composite audio and video streams to the endpoint for rendering to one or more participants to the video conference that are associated with the endpoint. 
 
   
     
     
         33 . The system of  claim 32 , wherein the one or more media processing nodes are multi-party conferencing units built from off-the-shelf components. 
     
     
         34 . The system of  claim 32 , wherein each of the plurality of endpoints is associated with a videoconferencing service that is one of standards-based or proprietary. 
     
     
         35 . The system of  claim 32 , wherein the video streams comprise at least one of original compressed video, uncompressed raw video, and low resolution compressed thumbnail video. 
     
     
         36 . The system of  claim 32 , wherein converting the plurality of audio and video streams comprises converting, into a form compatible with the endpoint, at least one of video encoding formats, audio encoding formats, communication protocols, video resolutions, screen ratios, encryption standards, and acoustic considerations of the audio and video streams. 
     
     
         37 . The system of  claim 32 , wherein converting the plurality of audio and video streams into the composite audio and video streams comprises at least one of transcoding, upscaling, and downscaling the audio and video streams. 
     
     
         38 . The system of  claim 32 , wherein at least one of the media processing nodes is further configured to subscribe to one or more audio and video streams required to construct composite audio and video streams that are compatible with an endpoint. 
     
     
         39 . The system of  claim 32 , wherein at least one of the media processing nodes is further configured to transfer composite audio and video streams to at least one other media processing node. 
     
     
         40 . The system of  claim 32 , wherein each of the media processing nodes is further configured to track video metadata associated with decoded video streams in order to apply operations based on the video metadata to individual video streams in the one or more composite video streams. 
     
     
         41 . The system of  claim 32 , wherein each of the media processing nodes is further configured to create a unique audio stream for each endpoint in the video conference based on audio streams associated with other endpoints, thereby reducing noise and echo in the composite audio streams. 
     
     
         42 . A method of operating a video conference, the method comprising:
 accepting, by a media processing node, a plurality of audio and video streams from a plurality of endpoints in a video conference, at least some of the endpoints being associated with videoconferencing services that are incompatible with each other; and   for each of the endpoints in the video conference:
 converting in real-time the plurality of audio and video streams into one or more composite audio and video streams compatible with the endpoint; and 
 providing the one or more composite audio and video streams to the endpoint for rendering to one or more participants to the video conference that are associated with the endpoint. 
   
     
     
         43 . The method of  claim 42 , wherein each of the plurality of endpoints is associated with a videoconferencing service that is one of standards-based or proprietary. 
     
     
         44 . The method of  claim 42 , wherein the video streams comprise at least one of original compressed video, uncompressed raw video, and low resolution compressed thumbnail video. 
     
     
         45 . The method of  claim 42 , wherein converting the plurality of audio and video streams comprises converting, into a form compatible with the endpoint, at least one of video encoding formats, audio encoding formats, communication protocols, video resolutions, screen ratios, encryption standards, and acoustic considerations of the audio and video streams. 
     
     
         46 . The method of  claim 42 , wherein converting the plurality of audio and video streams into the composite audio and video streams comprises at least one of transcoding, upscaling, and downscaling the audio and video streams. 
     
     
         47 . The method of  claim 42 , further comprising subscribing, by the media processing node, to one or more audio and video streams required to construct composite audio and video streams that are compatible with an endpoint. 
     
     
         48 . The method of  claim 42 , further comprising transferring, by the media processing node, composite audio and video streams to at least one other media processing node. 
     
     
         49 . The method of  claim 42 , further comprising tracking, by the media processing node, video metadata associated with decoded video streams in order to apply operations based on the video metadata to individual video streams in the one or more composite video streams. 
     
     
         50 . The method of  claim 42 , further comprising creating, by the media processing node, a unique audio stream for each endpoint in the video conference based on audio streams associated with other endpoints, thereby reducing noise and echo in the composite audio streams.