Systems and methods for 2D image and spatial data capture for 3D stereo imaging

Abstract

Systems and methods for 2D image and spatial data capture for 3D stereo imaging are disclosed. The system utilizes a cinematography camera and at least one reference or “witness” camera spaced apart from the cinematography camera at a distance much greater that the interocular separation to capture 2D images over an overlapping volume associated with a scene having one or more objects. The captured image date is post-processed to create a depth map, and a point cloud is created form the depth map. The robustness of the depth map and the point cloud allows for dual virtual cameras to be placed substantially arbitrarily in the resulting virtual 3D space, which greatly simplifies the addition of computer-generated graphics, animation and other special effects in cinemagraphic post-processing.

1 . A method of converting two-dimensional (2D) images of a scene having therein at least one object to one or more three-dimensional (3D) images of the scene, comprising:
 simultaneously capturing at least first and second 2D images of the scene from corresponding at least first and second cameras having respective camera positions and orientations measured relative to a reference coordinate system;   forming a disparity map from the at least first and second 2D images, wherein the disparity map has a gray scale that corresponds to distance information of the at least one object relative to the reference coordinate system; and   forming from the disparity map a 3D point cloud P(x,y,z) representative of the at least one object, wherein the point cloud is configured to support first and second virtual cameras to create a stereo camera pair arrangeable in substantially arbitrary virtual locations.   
     
     
         2 . The method of  claim 1 , further comprising forming the disparity map using photogrammetric triangulation operation on the at least first and second 2D images. 
     
     
         3 . The method of  claim 1 , further comprising:
 defining the first and second virtual cameras for the 3D point cloud to define a stereo camera pair.   
     
     
         4 . The method of  claim 1 , further comprising:
 defining one or more regions of interest in one of the at least first and second 2D images;   forming the disparity map for the one or more regions of interest;   transforming the one or more regions of interest to conform to the other of the least first and second 2D images; and   forming the point cloud to include the one or more regions of interest.   
     
     
         5 . The method of  claim 1 , further comprising:
 calibrating the at least first and second cameras for one or more camera parameters.   
     
     
         6 . The method of  claim 5 , further comprising:
 performing a registration operation for the at least first and second cameras relative to a reference coordinate system.   
     
     
         7 . The method of  claim 3 , further comprising:
 combining the point cloud with at least one visual effect.   
     
     
         8 . The method of  claim 7 , further comprising forming the at least one visual effect as:
 a) one or more computer-generated objects;   c) one or more animated objects; and   b) one or more computer-generated environments.   
     
     
         9 . The method of  claim 7 , further comprising:
 forming a 3D stereo movie from the combined point cloud and the at least-one visual effect.   
     
     
         10 . The method of  claim 7 , wherein the 3D stereo movie includes a plurality of 3D stereo frames, and further comprising providing different positions for the first and second virtual cameras for at least two of the frames. 
     
     
         11 . The method of  claim 7 , further comprising defining for the first and second virtual camera an interocular distance and convergence values for the scene, thereby creating a true left and a true right eye for the stereo pair for theatrical viewing of the 3D movie. 
     
     
         12 . The method of  claim 1 , further comprising:
 supporting the at least first and second cameras on a camera support device.   
     
     
         13 . The method of  claim 1 , further comprising:
 collecting an amount of image data in respective removable memories respectively operably arranged in the at least first and second cameras; and   downloading the image data from the removable memories to a memory module.   
     
     
         14 . The method of  claim 1 , further comprising employing three cameras, and using one of the three cameras as a cinemagraphic camera and the other two cameras for capturing spatial date of the scene. 
     
     
         15 . A method of forming a distance representation of a scene from two-dimensional (2D) images of the scene, comprising:
 simultaneously capturing at least first and second 2D images of the scene from corresponding at least first and second cameras having respective camera positions and orientations measured relative to a reference coordinate system;   defining one or more regions of interest in the at least first and second 2D images;   associating differences between pixels in the at least first and second cameras with distances from a reference point; and   assigning different gray-scale intensities to different ones of the distances.   
     
     
         16 . The method of  claim 15 , further comprising transforming the at least one or more regions of interest from the at least first and second 2D images so that like regions of interest in the different images match despite being taken with different cameras at different orientations. 
     
     
         17 . The method of  claim 16 , further comprising:
 forming from the disparity map a 3D point cloud P (x,y,z) representing spatial positions of points in three-space that make up the 3D scene.   
     
     
         18 . The method of  claim 17 , further comprising defining a stereo pair of first and second virtual cameras. 
     
     
         19 . The method of  claim 18 , further comprising forming a 3D stereo movie from the 3D point cloud and the stereo pair of first and second virtual cameras. 
     
     
         20 . The method of  claim 15 , further comprising forming from the disparity map a polygonal mesh PM(x,y,z;RGB) of spatial positions and corresponding RGB color. 
     
     
         21 . The method of  claim 20 , further comprising:
 mapping 2D photographed images on to the polygonal mesh to form a textured 3D geometry;   defining first and second virtual cameras for the polygonal mesh; and   re-imaging the textured 3D geometry with the first and second virtual cameras to form a 3D movie, including setting an interocular distance and convergence values for the scene, thereby creating a true left and a true right eye stereo pair for theatrical viewing of the 3D movie.