Methods for automatic registration of 3d image data

Abstract

A method for automatic registration of 3D image data, captured by a 3D image capture system having an RGB camera and a depth camera, includes capturing 2D image data with the RGB camera at a first pose; capturing depth data with the depth camera at the first pose; performing an initial registration of the RGB camera to the depth camera; capturing 2D image data with the RGB camera at a second pose; capturing depth data at the second pose; and calculating an updated registration of the RGB camera to the depth camera.

We claim: 
     
         1 . A method comprising:
 performing registration of the monochrome camera to the depth camera based at least in part on a first pose associated with a system at a first time;   capturing 2D data with the monochrome camera at a second time;   capturing depth data at the second time;   determining a second pose associated with the system at the second time; and   correcting the registration of the monochrome camera to the depth camera based at least in part on the second pose.   
     
     
         2 . The method of  claim 1 , wherein the 2D data is second 2D data and the depth data is second depth data and the method further comprises:
 capturing first 2D data with a monochrome camera of a system at a first time;   capturing first depth data with a depth camera of the system at the first time; and   determining a first pose associated with the system at the first time.   
     
     
         3 . The method of  claim 2 , wherein correcting the registration comprises:
 identifying a set of points common to the first depth and the second depth data;   calculating a texture error function for the set of points; and   updating the registration to reduce the texture error function.   
     
     
         4 . The method of  claim 3 , further comprising:
 tracking motion of the system between the first pose and the second pose; and   wherein identifying the set of points common to the first depth data and the second depth data comprises identifying the set of points by applying a rigid body transform based at least in part on the motion.   
     
     
         5 . The method of  claim 3 , wherein the texture error function is based at least in part on a perceived surface quality. 
     
     
         6 . The method of  claim 1 , further comprising:
 tracking motion of the system between the first pose and the second pose;   wherein capturing the depth data comprises using a derived data technique; and   wherein using the derived data technique comprises calculating the depth data based at least in part on prior captured depth data.   
     
     
         7 . The method of  claim 1 , wherein updating the registration comprises selecting a registration technique based on a lighting condition. 
     
     
         8 . The method of  claim 1 , further comprising:
 temporally synchronizing the monochrome camera and the depth camera based at least in part on timing signals provided to the monochrome camera and the depth camera; and   causing the monochrome camera to capture two frames for every frame captured by the depth camera.   
     
     
         9 . The method of  claim 1 , wherein performing the registration of the monochrome camera to the depth camera comprises performing the registration using an image feature detection technique. 
     
     
         10 . The method of  claim 1 , wherein receiving user input comprises:
 displaying a depth map image overlaid on a monochrome image on a display;   enabling user alignment of the depth map image and the monochrome image on the display; and   measuring transformations resulting from the user alignment of the depth map image and the monochrome image; and   correcting the registration of the monochrome camera to the depth camera based on at least in part on the user input.   
     
     
         11 . A method comprising:
 capturing first 2D infrared data captured by a first camera at a first time;   capturing first depth data captured by a second camera at the first time;   performing registration of the first camera to the second camera based at least in part on motion data at the first time;   receiving second 2D infrared data captured by the first camera at a second time;   receiving second depth data captured by the second camera at the second time;   identifying a set of points common to the first depth data and the second depth data; and   updating the registration of the first camera to the second camera to reduce misalignment between the first camera and the second camera based at least in part on set of points common to the first depth data and the second depth data.   
     
     
         12 . The method of  claim 11 , further comprising:
 determining a texture error function from the set of points common to the first depth data and the second depth data; and   wherein updating the registration is based at least in part on the texture error function.   
     
     
         13 . The method of  claim 11 , wherein the second depth data and the second 2D infrared data is captured in response to the motion data indicating a change in position of a system including the first camera and the second camera. 
     
     
         14 . The method of  claim 11 , wherein identifying the set of points common to the first depth data and the second depth data comprises identifying the set of points using an iterative closest point algorithm. 
     
     
         15 . The method of  claim 11 , wherein the first camera is an infrared camera and the second camera is a depth camera. 
     
     
         16 . A method comprising:
 receiving first 2D monochrome data;   receiving first depth data, the first depth data and the first 2D monochrome data captured at substantially a first time by a system;   determining a first pose associated with the system at the first time;   performing registration of a first camera to a second camera based at least in part on a comparison of the first 2D monochrome data to the first depth data;   receiving second 2D monochrome data;   receiving second depth data, the second depth data captured and the second 2D monochrome data captured at substantially a second time;   determining a second pose associated with the system at the second time; and   correcting the registration of the first camera to the second camera based at least in part on the second pose.   
     
     
         17 . The method of  claim 16 , wherein correcting the registration comprises:
 identifying a set of points common to the first depth and the second depth data;   calculating a texture error function for the set of points; and   updating the registration to reduce the texture error function.   
     
     
         18 . The method of  claim 17 , wherein the texture error function is based at least in part on a perceived surface quality. 
     
     
         19 . The method of  claim 16 , further comprising correcting the registration of the first camera to the second camera based on at least in part on a user input. 
     
     
         20 . The method of  claim 16 , wherein performing the registration of the first camera to the second camera includes overlaying the depth data on the 2D monochrome data.