Progressive adaptive time stamp resolution in multimedia authoring

Abstract

Environments with unreliable delivery may result in faltering presentation of multimedia objects, due to missing time stamp deadlines. This may be alleviated by introducing more flexible time stamping. To avoid this, additional MPEG-4 object time information is sent to the client. This requires a new dedicated descriptor, carried in the Elementary Stream Descriptor. The new more flexible timing information will have two features. First, instead of fixed start and end times, the duration of an object can be given a range. And second, the start and end times are made relative to other multimedia object start and end times. This information can then be used by the client to adapt the timing of the ongoing presentation to the environment, while having more room to stay within the presentation author's intent and expectations.

1. A computer-implemented method of progressive time stamp resolution in a multimedia presentation, comprising the steps of:
 supplying a player of a multimedia presentation with information comprising two labels, one for a multimedia object's start time and one for the multimedia object's end time relative to other multimedia object start and stop times, and three durations, a maximum duration and a preferred duration for each multimedia object prior to playback of the multimedia object; and 
 resolving the durations of the multimedia objects using said information based on actual multimedia object durations and arrival of information of multimedia objects to be played, wherein the step of resolving comprises the steps of: 
 collecting all the dependency relations for a label Px, by taking all objects n that have Px as the label for their end time:
     t   n +minimum( n )≦ t   x   ≦t   n +maximum( n ) n=1, . . . , N 
 
 where t n  is the start time of object n, and N is the number of objects; 
 using the N relations to calculate the tightest bounds on t x  
   min {t x }≦{t x }≦max{t x } 
 
 with
   min{ t   x }=max{ t   x +minimum( n )} n=1, . . . , N 
   max{ t   x }=min{ t   x +maximum( n )} n=1, . . . , N; 
 
 recalculating bounds on the duration of each object n, by using:
   duration( n )= t   x   −t   n    
 
 to get
   min{ t   x   }−t   n ≦duration( n )≦max{ t   x   }−t   n  n=1, . . . N; and 
 
 recalculating the preferred duration of each object n according to the process:
   if (preferred( n )<min{ t   x   }−t   n ) then 
   preferred( n )=min{ t   x   }−t   n    
   else if (preferred( n )>max{ t   x   }−t   n ) 
   then preferred( n )=max{ t   x   }−t   n    
 
 end if. 
 
   
   
     2. The method of progressive time stamp resolution in a multimedia presentation recited in  claim 1  wherein the step of resolving further comprises the steps of:
 using as the general error criterion for resolving the duration of each multimedia object: 
 E=
       ∑     n   =   1     N       
 
 
     {duration(n)−preferred(n)} 2  
 or, substituting duration(n)=t x −t n : 
 E=
       ∑     n   =   1     N       
 
 
     {t x −t n −preferred(n)} 2  
 and taking the derivative of E with respect to t x , and setting this to 0 to obtain the optimal solution for the absolute time t x  of label Px as: 
 t x =
         1   N     ⁢     ∑     n   =   1     N         
 
 
     {t n +preferred(n)}; and
 calculating the corresponding duration of multimedia object n as:
   duration( n )= t   x   −t   n .