Method of estimating the speed of displacement of a camera

Abstract

This method comprises the estimation of the speed X vR of displacement of a camera by searching for the speed X vR which minimizes a discrepancy directly between: -a first value of a physical quantity at the level of a first point (p*) of a reference image, and -a second value of the same physical quantity at the level of a second point (p v 2 ) of a current image, the first value of the physical quantity at the level of the first point (p*) of the reference image being constructed: -by selecting neighbour points of the first point (p*) as a function of the speed X vR and of a time to equal to the exposure time of the first camera, then -by averaging the values of the physical quantity at the level of the neighbour points selected and of the first point in such a way as to generate a new value of the physical quantity at the level of the first point.

1 - 15 . (canceled) 
     
     
         16 . A method for estimating the speed of movement of a first video camera at the moment at which that first video camera captures a current image of a three-dimensional scene, this method including:
 a) storing in an electronic memory a reference image corresponding to an image of the same scene captured by a second video camera in a different pose, the reference image including pixels organized in parallel rows, the memory containing for each pixel of the reference image the measurement of a physical quantity measured by that pixel, that physical quantity being chosen in the group made up of the intensity of radiation emitted by the point photographed by that pixel and of a depth separating that pixel from the point of the scene photographed by that pixel,   b) storing in the electronic memory the current image, the current image including pixels organized in parallel rows, the memory containing for each pixel of the current image the measurement of a physical quantity measured by that pixel, that physical quantity being the same as the physical quantity measured by the pixels of the reference image,   c) storing in the electronic memory for each pixel of the reference image or of the current image the measurement of a depth that separates that pixel from the point of the scene photographed by that pixel,   d) estimating the pose x pR  of the first video camera,   e) estimating the speed x vR  of movement of the first video camera during the capture of the current image,wherein the step e) is executed by seeking the speed x vR  that minimizes, for N points of the reference image, where N is an integer greater than 10% of the number of pixels of the reference image, a difference directly between:   a first value of the physical quantity at the level of a first point of the reference image, that first value being constructed from at least one measurement of that physical quantity stored in that reference image, and   a second value of the same physical quantity at the level of a second point of the current image, that second value being constructed from measurements of that physical quantity stored in the current image and the coordinates of the second point, the coordinates of the second point being obtained from a projection of the point of the scene photographed by the first point onto the plane of the current image, this projection being a function of the estimated pose x pR  and of the measurements of the depths stored in the current or reference image,   the first value of the physical quantity at the level of the first point of the reference image being constructed:   by selecting points adjacent the first point, each adjacent point corresponding to the projection onto the plane of the reference image of a third point the coordinates of which are obtained by shifting the first point a distance T 2 (−tx vR ), where t is a time elapsed since the beginning of an exposure time t e , that time being less than or equal to the exposure time t e , and T 2 ( . . . ) is a function that integrates the speed x vR  during the time t, each adjacent point corresponding to a respective value of the time t and the time t e  being equal to the exposure time of the first video camera, then   by averaging the values of the physical quantity at the level of the selected adjacent points and the first point so as to generate a new value of the physical quantity at the level of the first point, that new value constituting an estimate of that which would be measured if the exposure time of the pixels of the second video camera were equal to t e  and if the second video camera were to move at the speed x vR  during the exposure time t e , the values of the physical quantity at the level of the adjacent points being obtained from the measurements stored in the reference image and the coordinates of the adjacent points.   
     
     
         17 . The method as claimed in  claim 16 , wherein the method includes:
 providing a current image in which the rows of pixels have been captured one after the other so that a non-zero time t Δ  elapses between the moments of capture of two successive rows of the current image, and   obtaining the coordinates of the second point in the plane of the current image:
 by determining the coordinates of a third point in the plane of the current image that corresponds to the projection onto that plane of the point of the scene photographed by the first point, those coordinates being determined from the estimated pose x pR  and the measurements of the depths stored in the current image or the reference image, then 
 by shifting the third point a distance equal and opposite to the distance travelled by the first video camera between a time t 1  at which a first row of the current image is captured and a time t i  at which the row of pixels to which the third point belongs was captured, that distance being a function of the time t Δ  and the speed x vR , and finally 
 by projecting the third point shifted in this way onto the plane of the current image to obtain the coordinates of the second point. 
   
     
     
         18 . The method as claimed in  claim 17 , wherein, during the step e), the coordinates of the second point are obtained with the aid of the following relation:
     p   w2   =w   2 ( T   2 (−τ x   vR ),  p   w1 ),
   where:   p w2  and p w1  are respectively the coordinates of the second and third points in the plane of the current image,   τ is the time that has elapsed between the time t 1  and the time t i ,   T 2 (−τx vR ) is a function that returns the opposite of the distance travelled by the first video camera between the times t 1  and t i  by integrating the speed -x vR  during the time τ, and   w 2 ( . . . ) is a central projection that returns the coordinates in the plane of the current image of the third point after it has been shifted by the distance T 2 (−τx vR ), this central projection being a function of intrinsic parameters of the first video camera notably including its focal length.   
     
     
         19 . The method as claimed in  claim 16 , wherein the speed x vR  is a vector with six coordinates coding the speed of movement in translation and in rotation of the first video camera along three mutually orthogonal axes so that during the step e) the speed in translation and in rotation of the first video camera is estimated. 
     
     
         20 . The method as claimed in  claim 16 , wherein during the step e) the coordinates of the pose x pR  are considered as being unknowns to be estimated so that the steps d) and e) are then executed simultaneously by simultaneously seeking the pose x pR  and the speed x vR  that minimize the difference between the first and second values of the physical quantity. 
     
     
         21 . The method as claimed in  claim 20 , wherein while the simultaneously seeking the pose x pR  and the speed x vR , the coordinates of the pose x pR  are defined by the relation X pR =t p x vR +x pR-1 , where x pR-1  is the estimate of the pose of the first video camera at the moment at which that first video camera captured the preceding current image and t p  is the time that separates the moment of capture of the current image from the moment of capture of the preceding current image by the first video camera so that only six coordinates are to be estimated during the steps d) and e) to obtain simultaneously estimates of the speed x vR  and the pose x pR . 
     
     
         22 . The method as claimed in  claim 16 , wherein the reference image is an image captured by a second immobile camera. 
     
     
         23 . A method for estimating the speed of movement of a first video camera at the moment at which that first video camera captures a current image of a three-dimensional scene, this method including:
 a) storing in an electronic memory a reference image corresponding to an image of the same scene captured by a second video camera in a different pose, the reference image including pixels organized in parallel rows, the memory containing for each pixel of the reference image the measurement of a physical quantity measured by that pixel, that physical quantity being chosen in the group made up of the intensity of radiation emitted by the point photographed by that pixel and a depth separating that pixel from the point of the scene photographed by that pixel,   b) storing in the electronic memory the current image, the current image including pixels organized in parallel rows, the memory containing for each pixel of the current image the measurement of a physical quantity measured by that pixel, that physical quantity being the same as the physical quantity measured by the pixels of the reference image,   c) storing in the electronic memory for each pixel of the reference image or of the current image the measurement of a depth that separates that pixel from the point of the scene photographed by that pixel,   d) estimating a pose x pR  of the first video camera,   e) estimating the speed x vR  of movement of the first video camera during the capture of the current image,wherein the step e) is executed by seeking the speed x vR  that minimizes, for N points of the current image, where N is an integer greater than 10% of the number of pixels of the current image, a difference directly between:   a first value of the physical quantity at the level of a first point of the current image, that first value being constructed from at least one measurement of that physical quantity stored in that current image, and   a second value of the same physical quantity at the level of a second point of the reference image, that second value being constructed from measurements of that physical quantity stored in the reference image and the coordinates of the second point in the plane of the reference image, the coordinates of the second point being obtained from a projection of the point of the scene photographed by the first point onto the plane of the reference image, this projection being a function of the estimated pose x pR  and of the measurements of the depths stored in the current or reference image,   the second value of the physical quantity at the level of the second point of the reference image being constructed:   by selecting points adjacent the second point, each adjacent point corresponding to the projection onto the plane of the reference image of a third point the coordinates of which are obtained by shifting the second point a distance T 2 (−tx vR ), where t is a time elapsed since the beginning of an exposure time t e , that time being less than or equal to the exposure time t e , and T 2 ( . . . ) is a function that integrates the speed x vR  during the time t, each adjacent point corresponding to a respective value of the time t and the time t e  being equal to the exposure time of the first video camera, then   by averaging the values of the physical quantity at the level of the selected adjacent points and the second point so as to generate a new value of the physical quantity at the level of the second point, that new value constituting an estimate of that which would be measured if the exposure time of the pixels of the second video camera were equal to t e  and if the second video camera were to move at the speed x vR  during the exposure time t e , the values of the physical quantity at the level of the adjacent points being obtained from the measurements stored in the reference image and the coordinates of the adjacent points.   
     
     
         24 . The method as claimed in  claim 23 , wherein the method includes:
 providing a current image in which the rows of pixels have been captured one after the other so that a non-zero time t Δ  elapses between the moments of capture of two successive rows of the current image, and   obtaining the coordinates of the second point in the plane of the reference image:
 by determining the coordinates of a third point in the plane of the reference image that corresponds to the projection onto that plane of the point of the scene photographed by the first point, those coordinates being determined from the estimated pose x pR  and the measurements of the depths stored in the current image or the reference image, then 
 by shifting the third point a distance equal to and in the same direction as the distance traveled by the first video camera between a time t 1  at which a first row of the current image is captured and a time t i  at which the row of pixels to which the first point belongs was captured, that distance being a function of the time t Δ  and the speed x vR , and finally 
 by projecting the third point shifted in this way onto the plane of the reference image to obtain the coordinates of the second point. 
   
     
     
         25 . The method as claimed in  claim 24 , wherein, during the step e), the coordinates of the second point are obtained with the aid of the following relation:
     p   w5   =w   5 ( T   2 (τ x   vR ),  p   w4 ),
   where:   p w5  and p w4  are respectively the coordinates of the second and third points in the plane of the reference image,   τ is the time that has elapsed between the time t 1  and the time t i ,   T 2 (τx vR ) is a function that returns the distance travelled by the first video camera between the times t 1  and t i  by integrating the speed x vR  during the time τ, and   w 5 ( . . . ) is a central projection that returns the coordinates in the plane of the reference image of the third point after it has been shifted by the distance T 2 (τx vR ), this central projection being a function of intrinsic parameters of the second video camera notably including its focal length.   
     
     
         26 . The method as claimed in  claim 23 , wherein the speed x vR  is a vector with six coordinates coding the speed of movement in translation and in rotation of the first video camera along three mutually orthogonal axes so that during the step e) the speed in translation and in rotation of the first video camera is estimated. 
     
     
         27 . The method as claimed in  claim 23 , wherein during the step e) the coordinates of the pose x pR  are considered as being unknowns to be estimated so that the steps d) and e) are then executed simultaneously by simultaneously seeking the pose x pR  and the speed x vR  that minimize the difference between the first and second values of the physical quantity. 
     
     
         28 . The method as claimed in  claim 27 , wherein while the simultaneously seeking the pose x pR  and the speed x vR , the coordinates of the pose x pR  are defined by the relation X pR =t p x vR +x pR-1 , where x pR-1  is the estimate of the pose of the first video camera at the moment at which that first video camera captured the preceding current image and t p  is the time that separates the moment of capture of the current image from the moment of capture of the preceding current image by the first video camera so that only six coordinates are to be estimated during the steps d) and e) to obtain simultaneously estimates of the speed x vR  and the pose x pR . 
     
     
         29 . The method as claimed in  claim 23 , wherein the reference image is an image captured by a second immobile camera. 
     
     
         30 . The method according to  claim 20 , wherein the method further comprises ar construction of a trajectory of the first video camera, said construction including:
 a) acquiring a three-dimensional model of the scene,   b) storing in an electronic memory a succession of temporally ordered images captured by the first video camera during its movement within the scene, each image including pixels organized in parallel rows, the memory containing for each pixel of the current image a measurement of a physical quantity chosen in the group made up of the intensity of radiation emitted by the point photographed by that pixel and a depth separating that pixel from the photographed point of the scene,   c) for each current image:   constructing or selecting from the three-dimensional model of the scene a reference image including pixels that have photographed the same points of the scene as the pixels of the current image,   estimating said pose x pR  of the first video camera at the moment at which the latter captures that current image,   constructing the trajectory of the first video camera from the various estimated poses of the first video camera.   
     
     
         31 . The method according to  claim 27 , wherein the method further comprises a construction of a trajectory of the first video camera, that method including:
 a) acquiring a three-dimensional model of the scene,   b) storing in an electronic memory a succession of temporally ordered images captured by the first video camera during its movement within the scene, each image including pixels organized in parallel rows, the memory containing for each pixel of the current image a measurement of a physical quantity chosen in the group made up of the intensity of radiation emitted by the point photographed by that pixel and a depth separating that pixel from the photographed point of the scene,   c) for each current image:   constructing or selecting from the three-dimensional model of the scene a reference image including pixels that have photographed the same points of the scene as the pixels of the current image,   estimating said pose x pR  of the first video camera at the moment at which the latter captures that current image,   constructing the trajectory of the first video camera from the various estimated poses of the first video camera.   
     
     
         32 . The method according to  claim 16 , wherein the method comprises a processing of a current image of a three-dimensional scene, the current image including pixels organized in parallel rows, that method including:
 a) estimating the speed x vR  of movement of a first video camera at the moment at which that video camera captured the current image,   b) automatically modifying the current image to correct the current image as a function of the estimated speed x vR  so as to limit the distortions of the current image caused by the motion blur.   
     
     
         33 . The method according to  claim 23 , wherein the method comprises a processing of a current image of a three-dimensional scene, the current image including pixels organized in parallel rows, that method including:
 a) estimating the speed x vR  of movement of a first video camera at the moment at which that video camera captured the current image,   b) automatically modifying the current image to correct the current image as a function of the estimated speed x vR  so as to limit the distortions of the current image caused by the motion blur.   
     
     
         34 . An information storage medium, wherein it contains instructions for the execution of a method as claimed in  claim 16  when those instructions are executed by an electronic computer. 
     
     
         35 . A system for estimating the speed of movement of a first video camera at the moment at which that first video camera captures a current image of a three-dimensional scene, that system including:
 an electronic memory containing:
 a reference image corresponding to an image of the same scene captured by a second video camera in a different pose, the reference image including pixels organized in parallel rows, the memory containing for each pixel of the reference image the measurement of a physical quantity measured by that pixel, that physical quantity being chosen in the group made up of the intensity of radiation emitted by the point photographed by that pixel and a depth separating that pixel from the point of the scene photographed by that pixel, 
 the current image, the current image including pixels organized in parallel rows, the memory containing for each pixel of the current image the measurement of a physical quantity measured by that pixel, that physical quantity being the same as the physical quantity measured by the pixels of the reference image, 
 for each pixel of the reference image or the current image, a measurement of a depth that separates that pixel from the point of the scene photographed by that pixel, 
   an information processing unit adapted to:
 estimate a pose x pR  of the first video camera, 
 estimate the speed x vR  of movement of the first video camera during the capture of the current image,wherein the information processing unit is able to estimate the speed x vR  by seeking the speed x vR  that minimizes, for N points of the reference image, where N is an integer greater than 10% of the number of pixels of the reference image, a difference directly between: 
   a first value of the physical quantity at the level of a first point of the reference image, that first value being constructed from at least one measurement of that physical quantity stored in that reference image, and   a second value of the same physical quantity at the level of a second point of the current image, that second value being constructed from measurements of that physical quantity stored in the current image and the coordinates of the second point, the coordinates of the second point being obtained from a projection of the point of the scene photographed by the first point onto the plane of the current image, this projection being a function of the estimated pose x pR  and of the measurements of the depths stored in the current or reference image, the first value of the physical quantity at the level of the first point of the reference image being constructed:   by selecting points adjacent the first point, each adjacent point corresponding to the projection onto the plane of the reference image of a third point the coordinates of which are obtained by shifting the first point a distance T 2 (−tx vR ), where t is a time elapsed since the beginning of an exposure time t e , that time being less than the exposure time t e , and T 2 ( . . . ) is a function that integrates the speed −x vR  during the time t, each adjacent point corresponding to a respective value of the time t and the time t e  being equal to the exposure time of the first video camera, then   by averaging the values of the physical quantity at the level of the selected adjacent points and the first point so as to generate a new value of the physical quantity at the level of the first point, that new value constituting an estimate of that which would be measured if the exposure time of the pixels of the second video camera were equal to t e  and if the second video camera were to move at the speed x vR  during the exposure time t e , the values of the physical quantity at the level of the adjacent points being obtained from the measurements stored in the reference image and the coordinates of the adjacent points.   
     
     
         36 . A system for estimating the speed of movement of a first video camera at the moment at which that first video camera captures a current image of a three-dimensional scene, that system including:
 an electronic memory containing:
 a reference image corresponding to an image of the same scene captured by a second video camera in a different pose, the reference image including pixels organized in parallel rows, the memory containing for each pixel of the reference image the measurement of a physical quantity measured by that pixel, that physical quantity being chosen in the group made up of the intensity of radiation emitted by the point photographed by that pixel and a depth separating that pixel from the point of the scene photographed by that pixel, 
 the current image, the current image including pixels organized in parallel rows, the memory containing for each pixel of the current image the measurement of a physical quantity measured by that pixel, that physical quantity being the same as the physical quantity measured by the pixels of the reference image, 
 for each pixel of the reference image or the current image, a measurement of a depth that separates that pixel from the point of the scene photographed by that pixel, 
   an information processing unit adapted to:
 estimate the pose x pR  of the first video camera, 
 estimate the speed x vR  of movement of the first video camera during the capture of the current image,wherein the processing unit is able to estimate the speed x vR  by seeking the speed x vR  that minimizes, for N points of the current image, where N is an integer greater than 10% of the number of pixels of the current image, a difference directly between: 
   a first value of the physical quantity at the level of a first point of the current image constructed from at least one measurement of that physical quantity stored in that current image, and   a second value of the same physical quantity at the level of a second point of the reference image, that second value being constructed from measurements of that physical quantity stored in the reference image and the coordinates of the second point, the coordinates of the second point being obtained from a projection of the point of the scene photographed by the first point onto the plane of the reference image, this projection being a function of the estimated pose x pR  and of the measurements of the depths stored in the current or reference image,   the second value of the physical quantity at the level of the second point of the reference image being constructed:   by selecting points adjacent the second point, each adjacent point corresponding to the projection onto the plane of the reference image of a third point the coordinates of which are obtained by shifting the second point a distance T 2 (−tx vR ), where t is a time elapsed since the beginning of an exposure time t e , that time being less than the exposure time t e , and T 2 ( . . . ) is a function that integrates the speed x vR  during the time t, each adjacent point corresponding to a respective value of the time t and the time t e  being equal to the exposure time of the first video camera, then   by averaging the values of the physical quantity at the level of the selected adjacent points and the second point so as to generate a new value of the physical quantity at the level of the second point, that new value constituting an estimate of that which would be measured if the exposure time of the pixels of the second video camera were equal to t e  and if the second video camera were to move at the speed x vR  during the exposure time t e , the values of the physical quantity at the level of the adjacent points being obtained from the measurements stored in the reference image and the coordinates of the adjacent points.