Motion estimation with variable spatial resolution

Abstract

A motion estimator has a spatial sub-sampler to receive input images; at least one motion estimator determining motion vectors between input images and sub-sampled motion vectors between sub-sampled images; an up-sampler for up-sampling the sub-sampled motion vectors; and a selector for providing a motion vector output by selecting between the motion vectors and the (up-sampled) sub-sampled motion vectors, according to motion vector confidence.

The invention claimed is: 
     
       1. A method of motion estimation to provide output motion vectors, comprising the steps in a processor of:
 comparing input images in a sequence of images at a first spatial resolution to determine first spatial resolution motion vectors that describe pixel positional differences between said input images, 
 comparing said input images at a second spatial resolution which is different from said first spatial resolution to determine second spatial resolution motion vectors that describe pixel positional differences between said input images, 
 deriving a measure of motion vector confidence for at least one selected from the group of the first spatial resolution motion vectors and the second spatial resolution motion vectors, and 
 selecting at least once per image between the first spatial resolution motion vectors and the second spatial resolution motion vectors to provide the output motion vectors, according to said measure of motion vector confidence. 
 
     
     
       2. A method according to  claim 1 , in which the first and second spatial resolutions vary in the number of pixels used to represent the input images that are compared. 
     
     
       3. A method according to  claim 1 , where motion vectors are derived from phase correlation and said measure of motion vector confidence is derived from a phase correlation peak height. 
     
     
       4. A method according to  claim 1 , where motion vectors are derived from block matching and said measure of motion vector confidence is derived from a block match error. 
     
     
       5. Apparatus for motion estimation comprising:
 an input for receiving input images in a sequence of images; 
 a spatial sub-sampler to receive input images and provide sub-sampled images; 
 at least one motion estimator for determining
 first spatial resolution motion vectors that describe pixel positional differences between said input images and 
 second spatial resolution motion vectors that describe pixel positional differences between said sub-sampled images, 
 the first spatial resolution motion vectors being determined independently of the second spatial resolution motion vectors and the second spatial resolution motion vectors being determined independently of the first spatial resolution motion vectors; 
 
 an up-sampler for up-sampling said second motion vectors; and 
 a motion vector selector for providing a motion vector output by selecting between the first spatial resolution motion vectors and the up-sampled second spatial resolution motion vectors, wherein said selection is according to a measure of motion vector confidence. 
 
     
     
       6. Apparatus according to  claim 5 , where motion vectors are derived from phase correlation and said measure of motion vector confidence is taken from a phase correlation peak height. 
     
     
       7. Apparatus according to  claim 5 , where motion vectors are derived from block matching and said measure of motion vector confidence is taken from a block match error. 
     
     
       8. A non-transitory computer program product adapted to cause programmable apparatus to implement a method of motion estimation to provide output motion vectors comprising the steps of:
 comparing input images in a sequence of images at a first variable spatial resolution to determine first spatial resolution motion vectors that describe pixel positional differences between said input images; 
 comparing said input images at a second spatial resolution which is different from said first spatial resolution to determine second spatial resolution motion vectors that describe pixel positional differences between said input images, 
 deriving a measure of motion vector confidence for at least one selected from the group consisting of the first spatial resolution motion vectors and the second spatial resolution motion vectors, and 
 selecting at least once per image between the first spatial resolution motion vectors and the second spatial resolution motion vectors to provide the output motion vectors, according to a measure of motion vector confidence. 
 
     
     
       9. A computer program product according to  claim 8 , in which said selecting between the first spatial resolution motion vectors and the second spatial resolution motion vectors is made for every pixel block in the input image. 
     
     
       10. A method of motion estimation for a sequence of images, each image comprising a plurality of blocks of pixels, each block comprising at least one pixel, the method comprising the steps of:
 determining first motion vectors that describe pixel positional differences between input images in a sequence of images at a first spatial image resolution; 
 determining second motion vectors that describe pixel positional differences between input images in a sequence of images at a second spatial image resolution which is different from said first spatial image resolution, the first motion vectors being determined independently of the second motion vectors and the second motion vectors being determined independently of the first motion vectors; 
 comparing a first measure of confidence in a said first motion vector with a second measure of confidence in a said second motion vector: and 
 for each block of pixels switching between a first motion vector and a second motion vectors in accordance with the results of said comparison.