Method and apparatus for scanning and printing a 3D object

Abstract

A smartphone may be freely moved in three dimensions as it captures a stream of images of an object. Multiple image frames may be captured in different orientations and distances from the object and combined into a composite image representing an three-dimensional image of the object. The image frames may be formed into the composite image based on representing features of each image frame as a set of points in a three dimensional depth map. Coordinates of the points in the depth map may be estimated with a level of certainty. The level of certainty may be used to determine which points are included in the composite image. The selected points may be smoothed and a mesh model may be formed by creating a convex hull of the selected points. The mesh model and associated texture information may be used to render a three-dimensional representation of the object on a two-dimensional display. Additional techniques include processing and formatting of the three-dimensional representation data to be printed by a three-dimensional printer so a three-dimensional model of the object may be formed.

What is claimed is: 
     
       1. A portable electronic device, comprising:
 a camera; 
 at least one processor, configured to:
 process a plurality of images acquired with the camera to form a representation of an object depicted in the plurality of images, wherein the representation indicates a surface of the object in three-dimensional space, and wherein the processing comprises:
 forming multiple localized point clouds, each localized point cloud comprising probabilities associated with respective points in the localized point cloud indicative of a probability that the representation accurately indicates a location of a first feature of a plurality of features; 
 fusing the localized point clouds into a combined point cloud; 
 adjusting one or more points in the combined point cloud to reduce inconsistency; and 
 smoothing the combined point cloud based at least in part on the probabilities associated with the localized point clouds. 
 
 
 
     
     
       2. The portable electronic device of  claim 1 , wherein the at least one processor is configured to compute probabilities associated with respective points in a localized point cloud at least in part by determining a probability that a feature depicted in a first image of the plurality of images and a feature depicted in a second image of the plurality of images are both the first feature. 
     
     
       3. The portable electronic device of  claim 2 , wherein determining the probability that the feature depicted in the first image and the feature depicted in the second image are both the first feature comprises performing cross-correlation to identify similar regions of the object in the first image and the second image. 
     
     
       4. The portable electronic device of  claim 1 , wherein the at least one processor is configured to compute probabilities associated with respective points in a localized point cloud at least in part using geometric analysis to identify locations of specific features of the plurality of features depicted in more than one of the plurality of images. 
     
     
       5. The portable electronic device of  claim 4 , comprising a sensor for detecting motion of the portable electronic device, wherein the at least one processor is configured to form the localized point clouds at least in part by analyzing motion of the portable electronic device, detected by the sensor, between acquisition of one of the plurality of images and acquisition of another of the plurality of images. 
     
     
       6. The portable electronic device of  claim 1 , wherein the at least one processor is configured to form each one of the localized point clouds at least in part by:
 forming a first version of the one localized point cloud; and 
 using probability to determine whether to update the first version. 
 
     
     
       7. The portable electronic device of  claim 6 , wherein using probability to determine whether to update the first version comprises:
 identifying a feature of the object that is not indicated in the first version and that is depicted in a new image of the plurality of images, acquired after the first version was formed; and 
 determining a probability that the feature is accurately located in the new image. 
 
     
     
       8. The portable electronic device of  claim 7 , wherein the at least one processor is configured to update the first version if the determined probability that the feature is accurately located in the new image satisfies a predetermined threshold. 
     
     
       9. The portable electronic device of  claim 1 , wherein the at least one processor is configured to:
 compute a surface description from the smoothed combined point cloud by calculating a convex hull for one or more points in the smoothed combined point cloud; and 
 store the surface description as a three-dimensional file. 
 
     
     
       10. The portable electronic device of  claim 9 , wherein the at least one processor is configured to add texture information to the surface description. 
     
     
       11. A method for use in a portable electronic device comprising at least one processor, the method comprising acts of:
 processing, by the at least one processor, a plurality of images acquired by the portable electronic device to form a representation of an object depicted in the plurality of images, the representation indicating respective locations of a plurality of features of the object in three-dimensional space, the processing comprising:
 forming multiple localized point clouds, each localized point cloud comprising probabilities associated with respective points in the localized point cloud indicative of a probability that the representation accurately indicates a location of a first feature of the plurality of features; 
 fusing the localized point clouds into a combined point cloud; 
 adjusting one or more points in the combined point cloud to reduce inconsistency; and 
 smoothing the combined point cloud based at least in part on the probabilities associated with the localized point clouds. 
 
 
     
     
       12. The method of  claim 11 , wherein computing probabilities associated with respective points in a localized point cloud comprises determining a probability that a feature depicted in a first image of the plurality of images and a feature depicted in a second image of the plurality of images are both the first feature. 
     
     
       13. The method of  claim 12 , wherein determining the probability that the feature depicted in the first image and the feature depicted in the second image are both the first feature comprises performing cross-correlation to identify similar regions of the object in the first image and the second image. 
     
     
       14. The method of  claim 11 , wherein forming localized point clouds comprises using geometric analysis to identify locations of specific features of the plurality of features depicted in more than one of the plurality of images. 
     
     
       15. The method of  claim 11 , wherein forming localized point clouds comprises analyzing motion of the portable electronic device between acquisition of one of the plurality of images and another of the plurality of images. 
     
     
       16. The method of  claim 11 , wherein forming each one of the localized point clouds comprises forming a first version of one localized point cloud and using probability to determine whether to update the first version. 
     
     
       17. The method of  claim 16 , wherein using probability to determine whether to update the first version comprises:
 identifying a feature of the object that is not indicated in the first version and that is depicted in a new image of the plurality of images, acquired after the first version was formed; and 
 determining a probability that the feature is accurately located in the new image. 
 
     
     
       18. The method of  claim 17 , further comprising updating the first version if the determined probability that the feature is accurately located in the new image satisfies a predetermined threshold. 
     
     
       19. The method of  claim 11 , further comprising:
 computing a surface description from the smoothed combined point cloud by calculating a convex hull for one or more points in the smoothed combined point cloud; and 
 storing the surface description as a three-dimensional file. 
 
     
     
       20. The method of  claim 19 , further comprising adding texture information to the surface description.