Interactive Streaming Media Production Tool Using Communication Optimization

Abstract

The invention provides interactive streaming mixed media production. A mixed media content creation tool includes a resource-computation element that computes the likely resource load to be imposed on a server, communication link, and client device, when sent by the server to the client and presented to a user at the client. The resource-computation element is responsive to amounts of communication bandwidth available using the communication link, presentation buffer space and computing resources available at the client, and other measures of resource load. The resource-computation element attempts to optimize resource usage by the mixed media content. The system also includes a presentation element that responds to the resource-computation element, and to user requests, and interactively presents measures of projected resource usage by the mixed media content, comparisons of that projected resource usage with available resources, and comparisons of that projected resource usage before and after optimization.

1 - 25 . (canceled) 
   
   
       26 . A method for interactive production of mixed media, including steps of
 calculating a total necessary communication bandwidth and transmission time corresponding to a stream of mixed media including multiple media streams, said step of calculating including steps of   receiving inputs at a fitter;   dividing data comprising said stream of mixed media into data packets;   determining a minimal send time for each of said data packets; and   assigning a send time to each of said data packets, the assigned time being no earlier than the determined minimal send time; and   modifying the stream of mixed media based upon said total necessary communication bandwidth and transmission time.   
   
   
       27 . A method as in  claim 26 , wherein said minimal send time is the earliest time each of said packets may be sent with respect to a maximum permitted send ahead time imposed by the network. 
   
   
       28 . A method as in  claim 26 , wherein said inputs include a maximum packet size and an available channel bandwidth as a function of time, a time offset of for each said media stream, a maximum decoding buffer size, a maximum permitted send ahead time, and an overhead of transport protocol per packet. 
   
   
       29 . A method as in  claim 26 , including a step of notifying a content creator of the feasibility of sending said mixed media. 
   
   
       30 . A method as in  claim 26 , wherein said step of dividing the data is performed by a fitter. 
   
   
       31 . A method as in  claim 26 , wherein said step of determining the minimal send time includes simulating a decoding buffer. 
   
   
       32 . A method as in  claim 30 , wherein said minimal send time is set to a maximum of (1) a date stamp of a current said packet minus the maximum permitted send ahead time or (2) the date stamp of the packet most recently removed from said decoding buffer. 
   
   
       33 . A method as in  claim 32 , wherein said step of assigning said send time is performed starting with said data packets associated with an end of a performance of said mixed media and moving toward said data packets associated with a beginning of the performance of said mixed media. 
   
   
       34 . A method as in  claim 26 , wherein said send time is set to a nominal send time minus a size of said packet divided by an available bandwidth at said nominal send time. 
   
   
       35 . A method as in  claim 26 , wherein said step of modifying includes adding, deleting, or changing attributes associated with one or more of said multiple media streams. 
   
   
       36 . A method comprising steps of importing multiple media streams; encoding each of said media streams; for each of said media streams, computing a bit rate associated with one or more elements included in said media stream; generating a graphical image responsive to said step of computing, wherein said graphical image includes a set of information on bit rate usage for each of said media streams and for a stream of mixed media comprised of said multiple media streams; and exporting said multiple media streams. 
   
   
       37 . A method as in  claim 36 , wherein said step of encoding includes steps of converting a format associated with each of said media streams to an MPEG 4 format. 
   
   
       38 . A method as in  claim 36 , wherein said step of computing includes computing a bit rate that is responsive to a buffer space at an end user, a communication link or a server. 
   
   
       39 . A method as in  claim 36 , wherein said display includes a time display including a timeline, comprising a life cycle associated with said one or more elements. 
   
   
       40 . A method as in  claim 36 , wherein said step of updating is responsive to one or more changes made in at least one of said media streams. 
   
   
       41 . A method including steps of composing a mixed media including one or more objects; calculating a total necessary communication bandwidth and transmission time corresponding to said mixed media; modifying said mixed media in response to said step of calculating, wherein said step of modifying changes an order of transmission or a specific transmission time for said one or more object; and displaying a graphical representation of the total communication bandwidth and transmission time before and after the step of modifying said mixed media. 
   
   
       42 . A method as in  claim 41 , wherein said step of modifying includes adding, deleting, or changing attributes associated with said one or more object. 
   
   
       43 . A method as in  claim 41 , wherein said step of displaying includes displaying a set of different media taking part in a global resource usage. 
   
   
       44 . A method as in  claim 41 , wherein step of displaying includes providing a visual feedback of a download and decoding time required to transfer and decode said mixed media from a server to a player.