Method for Editing Computer-Generated Images to Maintain Alignment Between Objects Specified in Frame Space and Objects Specified in Scene Space

Abstract

In an image processing system, an image insertion is to be included onto, or relative to, a first and second frame, each depicting images of a set of objects of a geometric model. A point association is determined for a depicted object that is depicted in both the first frame and the second frame, representing reference coordinates in a virtual scene space of a first location on the depicted object independent of at least one position change and a mapping of a first image location in the first image to where the first location appears in the first image. A corresponding location in the second image is determined based on where the first location on the depicted object appears according to the reference coordinate in the virtual scene space and a second image location on the second image where the first location appears in the second image.

1 .- 20 . (canceled) 
     
     
         21 . A computer-implemented method for processing image data, the method comprising:
 obtaining a first frame representing a first image of a set of objects of a geometric model having object positions in a virtual scene space;   obtaining an image insertion for inclusion onto the first frame, wherein an image insertion location is associated with the image insertion;   determining an attachment object set, wherein the attachment object set comprises one or more objects of the set of objects that the image insertion is to track, the attachment object set including one or more depicted object that is depicted in the first frame;   determining a set of point associations, wherein a first point association of the set of point associations is a first data structure representing an association of a first image location and object location coordinates, wherein the first image location indicates a virtual scene space position of an origin point on the one or more depicted object where the origin point appears in the first image and the object location coordinates indicate an object location of the origin point, wherein object reference coordinates are independent of at least one position change in the virtual scene space of the one or more depicted object; and   using the set of point associations to position the image insertion in a second frame other than the first frame.   
     
     
         22 . The computer-implemented method of  claim 21 , wherein using the set of point associations to position the image insertion comprises:
 determining an image insertion reference point on the image insertion;   determining a corresponding location, in the first frame, for the image insertion reference point; and   modifying the first frame to account for the image insertion relative to the corresponding location.   
     
     
         23 . The computer-implemented method of  claim 22 , further comprising modifying the second frame with the image insertion placed in the second frame relative to a second location of the origin point in the second frame. 
     
     
         24 . The computer-implemented method of  claim 21 , wherein at least some of the object positions for the set of objects change between the first frame and the second frame. 
     
     
         25 . The computer-implemented method of  claim 21 , wherein the first image location is a first pixel location of the first frame and a corresponding location in the second frame comprises a nearby location in the second frame indicating pixels in the second frame interpolated to correspond to a neighborhood of the first pixel location. 
     
     
         26 . The computer-implemented method of  claim 21 , wherein the image insertion is a texture to be applied to the one or more depicted object, and wherein the attachment object set is determined based on artist input of object selection and/or determined based on a computer-determined object selection. 
     
     
         27 . A non-transitory computer-readable storage medium storing instructions, which when executed by at least one processor of a computer system, causes the computer system to:
 obtain a first frame representing a first image of a set of objects of a geometric model having object positions in a virtual scene space, the set of objects including a first object;   store a data structure defining contents of the first frame, the data structure including an array of pixel data structures wherein a pixel data structure of the array represents values associated with a pixel of the first frame;   determine an image insertion to be inserted into the first frame post-rendering;   determine an image insertion position in the first frame where the image insertion is to be positioned;   determine an attachment object that is depicted in the first frame at an attachment object pixel position relative to the image insertion position;   set a corresponding location value of the pixel data structure to a location value representing a location on the first object that corresponds to the attachment object pixel position;   set an object identifier value of the pixel data structure to an identifier value identifying the first object;   store an association of a displacement of the image insertion position relative to the attachment object pixel position; and   process a second frame post-rendering to include the image insertion, positioned based on the association and on a changed attachment object pixel position that is changed from the first frame to the second frame.   
     
     
         28 . The non-transitory computer-readable storage medium of  claim 27 , wherein instructions provide that the attachment object pixel position is at the same pixel position as the image insertion position. 
     
     
         29 . The non-transitory computer-readable storage medium of  claim 27 , wherein instructions provide that the attachment object pixel position is at a pre-defined offset relative to the image insertion position. 
     
     
         30 . The non-transitory computer-readable storage medium of  claim 27 , further comprising instructions for modifying the second frame with the image insertion placed in the second frame relative to a second location of the attachment object pixel position in the second frame. 
     
     
         31 . The non-transitory computer-readable storage medium of  claim 30 , further comprising instructions for interpolating the second location is a neighborhood of pixels from the attachment object pixel position from the first frame and a second frame position of the attachment object. 
     
     
         32 . The non-transitory computer-readable storage medium of  claim 27 , further comprising instructions for applying the image insertion as a texture to the first object, and wherein the attachment object set is based on artist input of object selection and/or based on a computer-determined object selection. 
     
     
         33 . The non-transitory computer-readable storage medium of  claim 27 , further comprising instructions to process a stored correspondence vector defining a region in the second frame that maps position changes in the first frame relative to the second frame and/or maps position changes in the second frame relative to the first frame. 
     
     
         34 . A computer system for processing image data, the computer system comprising:
 a) at least one processor;   b) a first storage for an attachment object set, wherein the attachment object set comprises one or more objects that an image insertion is to track, the attachment object set including one or more depicted object that is depicted in a first frame, the first frame representing a first image of objects of a geometric model having object positions in a virtual scene space;   c) a second storage for an image insertion location is associated with the image insertion; and   d) a computer-readable medium storing instructions, which when executed by the at least one processor, causes the computer system to:
 1) obtain the first frame; 
 2) obtain the image insertion for inclusion onto, or relative to, the first frame; 
 3) determine a set of point associations, wherein a first point association of the set of point associations is a first data structure representing an association of a first image location and object location coordinates, wherein the first image location indicates a virtual scene space position of an origin point on the one or more depicted object where the origin point appears in the first image and the object location coordinates indicate an object location of the origin point, wherein object reference coordinates are independent of at least one position change in the virtual scene space of the one or more depicted object; and 
 4) use the set of point associations to position the image insertion in a second frame other than the first frame. 
   
     
     
         35 . The computer system of  claim 34 , wherein at least some of the object positions for the one or more objects are different between the first frame and the second frame. 
     
     
         36 . The computer system of  claim 34 , wherein a stored correspondence vector defines a region in the second frame that maps position changes in the first frame relative to the second frame or in the second frame relative to the first frame. 
     
     
         37 . The computer system of  claim 34 , wherein the image insertion is placed in the second frame relative to a second location of the origin point in the second frame, wherein the second location is a neighborhood of pixels interpolated from the origin point from the first frame and a second frame position of the attachment object. 
     
     
         38 . The computer system of  claim 34 , wherein the image insertion is a texture to be applied to the one or more depicted object. 
     
     
         39 . The computer system of  claim 34 , wherein the second frame is in a different time sequence but uses a shared set of reference coordinates with the first frame. 
     
     
         40 . The computer system of  claim 34 , wherein the set of point associations consists of a single point association.