Camera calibration using depth data

Abstract

An apparatus is described herein. The apparatus includes an image capture module to capture depth data and sensor data for a plurality of views and an extraction module to extract a first plurality of features from the depth data and a second plurality of features from the sensor data for each view. The apparatus also includes a correspondence module to locate corresponding features in the first plurality of features and the second plurality of features for each view and a depth module to generate three-dimensional data for each feature of the first plurality of features for each view. Additionally, the apparatus includes a calibration module to calibrate the multiple cameras by matching the generated three dimensional data with the corresponding features in the first plurality of features and the second plurality of features.

What is claimed is: 
     
         1 . An apparatus for calibrating multiple cameras, comprising:
 an image capture module to capture depth data and sensor data for a plurality of views;   an extraction module to extract a first plurality of features from the depth data and a second plurality of features from the sensor data for each view;   a correspondence module to locate corresponding features in the first plurality of features and the second plurality of features for each view;   a depth module to generate three-dimensional data for each feature of the first plurality of features for each view; and   a calibration module to calibrate the multiple cameras by matching the generated three dimensional data with the corresponding features in the first plurality of features and the second plurality of features.   
     
     
         2 . The apparatus of  claim 1 , wherein a controller is to average the three-dimensional data across multiple views, and the calibration module is to calibrate the multiple cameras by matching the averaged three dimensional data with the corresponding features in the first plurality of features and the second plurality of features. 
     
     
         3 . The apparatus of  claim 1 , comprising averaging the three dimensional data across multiple views by calculating a best fit rigid transform from each view of a plurality of views to a reference view. 
     
     
         4 . The apparatus of  claim 1 , comprising averaging the three dimensional data by tracking a camera pose as each view of the plurality of views is captured and using the camera pose to average the views into a single view. 
     
     
         5 . The apparatus of  claim 1 , wherein the depth data is obtained from a stereoscopic infrared image pair. 
     
     
         6 . The apparatus of  claim 1 , comprising applying subpixel refinement to each feature of the plurality of features. 
     
     
         7 . The apparatus of  claim 1 , wherein features are detected by observing points of interest in each view of the plurality of views. 
     
     
         8 . The apparatus of  claim 1 , wherein calibrating the multiple views results in a calibration target model used to transformation between world and image coordinates. 
     
     
         9 . The apparatus of  claim 1 , wherein the plurality of views include multiple views of a calibration target. 
     
     
         10 . The apparatus of  claim 9 , wherein calibration is performed without prior knowledge of the calibration target. 
     
     
         11 . A method for calibrating multiple cameras, comprising:
 capturing depth data and sensor data for a plurality of views;   extracting a plurality of corresponding features from the depth data and the sensor data;   generating three-dimensional data for each feature of the plurality of corresponding features; and   calibrating the multiple cameras by calculating a projection based on the three-dimensional data and the plurality of corresponding features.   
     
     
         12 . The method of  claim 11 , wherein the three-dimensional data is averaged across the multiple views, and the multiple cameras are calibrated by calculating a projection based on the averaged three-dimensional data and the plurality of corresponding features across multiple views. 
     
     
         13 . The method of  claim 11 , comprising averaging the three dimensional data across multiple views by calculating a best fit rigid transform from each view of a plurality of views to a reference view. 
     
     
         14 . The method of  claim 11 , comprising averaging the three dimensional data by tracking a camera pose as each view of the plurality of views is captured and using the camera pose to average the views into a single view. 
     
     
         15 . The method of  claim 11 , wherein the depth data is obtained from a stereoscopic infrared image pair. 
     
     
         16 . A system for calibrating multiple cameras, comprising:
 a depth camera and a sensor to capture images;   a memory configured to receive image data; and   a processor coupled to the memory, depth camera, and sensor, the processor to:
 capture depth data and sensor data for a plurality of views; 
 extract a plurality of corresponding features from the depth data and the sensor data; 
 generate three-dimensional data for each feature of the plurality of corresponding features; and 
 calibrate the multiple cameras by calculating a projection based on the three-dimensional data and the plurality of corresponding features. 
   
     
     
         17 . The system of  claim 16 , wherein the processor is to average the three-dimensional data for each view, and the calibration module is to calibrate the multiple cameras by matching the averaged three dimensional data with the corresponding features in the first plurality of features and the second plurality of features. 
     
     
         18 . The system of  claim 16 , comprising averaging the three dimensional data across multiple views by calculating a best fit rigid transform from each view of a plurality of views to a reference view. 
     
     
         19 . The system of  claim 16 , comprising averaging the three dimensional data by tracking a camera pose as each view of the plurality of views is captured and using the camera pose to average the views into a single view. 
     
     
         20 . The system of  claim 16 , wherein the depth data is obtained from a stereoscopic infrared image pair. 
     
     
         21 . The system of  claim 16 , comprising applying subpixel refinement to each feature of the plurality of features. 
     
     
         22 . At least one machine readable medium comprising a plurality of instructions that, in response to being executed on a computing device, cause the computing device to:
 capture depth data and sensor data for a plurality of views;   extract a plurality of corresponding features from the depth data and the sensor data;   generate three-dimensional data for each feature of the plurality of corresponding features; and   calibrate the multiple cameras by calculating a projection based on the three-dimensional data and the plurality of corresponding features.   
     
     
         23 . The computer readable medium of  claim 22 , wherein the three-dimensional data is averaged across the multiple views, and the multiple cameras are calibrated by calculating a projection based on the averaged three-dimensional data and the plurality of corresponding features across multiple views. 
     
     
         24 . The computer readable medium of  claim 22 , comprising averaging the three dimensional data across multiple views by calculating a best fit rigid transform from each view of a plurality of views to a reference view. 
     
     
         25 . The computer readable medium of  claim 22 , comprising averaging the three dimensional data by tracking a camera pose as each view of the plurality of views is captured and using the camera pose to average the views into a single view.