Graphic Reference Matrix for Virtual Insertions

Abstract

A system and method for automatically acquiring a matrix of digital reference images containing landmarks relevant to recognizing and tracking objects in a digital video image is disclosed. In order to virtually insert a digital graphic having a correct pose in a stream of images, an indexed matrix of images of the landmarks in a venue is accumulated. The system and method allow for automated acquisition of new landmarks within a venue, and for the automated expansion of the reference matrix to accommodate reference images indicative of changed conditions within a venue and for reference images of new landmarks. When a landmark is recognized in an image, the graphic reference matrix is queried to obtain a preferred reference image to be used in making a virtual insertion.

1 : A method of using a graphic reference matrix for a virtual insertion of a digital graphic, comprising:
 providing said digital graphic;   providing a required pose for said virtual insertion of said digital graphic, said required pose being articulated with respect to a first reference landmark;   providing a digital data processor programmed to perform the functions of:
 receiving digital image containing a representation of said first reference landmark; 
 retrieving a preferred reference image containing a representation of said first reference landmark from said graphic reference matrix; 
 contrasting said digital image with said preferred reference image to determine a required transformation, said required transformation comprising a set of warp parameters for conforming said third digital image to said preferred reference image; and 
 using said required transformation to perform said virtual insertion of said digital graphic into said third digital image. 
   
     
     
         2 : The method of  claim 1 , wherein said digital data processor programmed to perform the further functions of:
 storing said digital image in said graphic reference matrix as a first reference image, said graphic reference matrix comprising a database operational on a computer memory functionally connected to said digital data processor;   receiving a second digital image containing a representation of said first reference landmark;   comparing said second digital image to said first reference image to determine a difference value; and   storing, if said difference value exceeds a predetermined value, said second digital image in said graphic reference matrix as a second reference image.   
     
     
         3 : The method of  claim 2 , wherein comparing said second digital image to said first reference image further comprises transforming said second digital image into a transformed second digital image such that a magnification, an orientation and a location of said representation of said first reference landmark within said transformed second digital image corresponds to the size, orientation and location of said representation of said first reference landmark within said first reference image. 
     
     
         4 : The method of  claim 3 , wherein said comparing further comprises evaluating a difference between a value of a pixel of said transformed second digital image and a value of a corresponding pixel of said first reference image, and wherein said difference value comprises a mathematical accumulation of a plurality of said differences of evaluated pairs of pixels. 
     
     
         5 : The method of  claim 1 , wherein said digital processor further performs the functions of:
 recognizing within said digital image, a representation of an object suitable for use as a new, second reference landmark;   calculating a pose of said new, second reference landmark with respect to said first reference landmark; and   storing said digital image and said calculated pose in a graphic reference matrix as a reference image.   
     
     
         6 : The method of  claim 1 , wherein said preferred reference image is the reference image having a difference value closest to a predetermined required difference value. 
     
     
         7 : The method of  claim 3 , wherein said evaluating a difference between values of corresponding pixels uses only one of a red, a green or a blue component of the value of the pixel. 
     
     
         8 : The method of  claim 3 , wherein said mathematical accumulation comprises a square root of the sum of the differences in pixel values. 
     
     
         9 : The method of  claim 3 , wherein said mathematical accumulation comprises a square root of the sum of the weighted differences in pixel values. 
     
     
         10 : The method of  claim 1 , wherein said set of warp parameters comprises an affine transformation. 
     
     
         11 : The method of  claim 1 , wherein said digital graphic comprises an animated video sequence.