Method for object recognition

Abstract

The present disclosure proposes a computer implemented of object recognition of an object to be identified using a method for reconstruction of a 3D point cloud. The method comprises the steps of acquiring, by a mobile device, a plurality of pictures of said object, sending the acquired pictures to a cloud server, reconstructing, by the cloud server, a 3D points cloud reconstruction of the object, performing a 3D match search in a 3D database using the 3D points cloud reconstruction, to identify the object, the 3D match search comprising a comparison of the reconstructed 3D points cloud of the object with 3D points clouds of known objects stored in the 3D database.

1 - 16 . (canceled) 
     
     
         17 . A computer-implemented method of object recognition, the method comprising:
 receiving a plurality of pictures of an object;   reconstructing, from the plurality of pictures, a 3D point cloud of the object, wherein the 3D point cloud comprises a plurality of vertices in a 3D space;   extracting, from the 3D point cloud, a plurality of 3D descriptors, wherein each descriptor comprises information about:
 at least one vertex of the plurality of vertices, and 
 a 3D primitive of the 3D point cloud; 
   encoding the plurality of 3D descriptors into one or more 3D feature vectors;   performing a recognition process, based on the 3D feature vectors, to identify the object; and   outputting an identifier of the object.   
     
     
         18 . The method of  claim 17 , wherein performing the recognition process comprises inputting the 3D feature vectors to a machine learning algorithm. 
     
     
         19 . The method of  claim 17 , wherein performing the recognition process comprises executing a 3D geometric comparison, execution of the 3D geometric comparison comprising comparing the 3D point cloud with 3D models of known objects based on the one or more 3D feature vectors. 
     
     
         20 . The method of  claim 19 , wherein executing the 3D geometric comparison comprises executing a segmentation and a clustering, wherein the segmentation is performed during the step of reconstructing the 3D point cloud, and wherein the clustering is executed on the 3D point cloud. 
     
     
         21 . The method of  claim 20 , wherein the clustering separates multiple objects visible in the plurality of pictures. 
     
     
         22 . The method of  claim 17 , wherein each 3D primitive is representative of a geometrical shape, and wherein the geometrical shape is a plane, sphere, cylinder, cube, or torus. 
     
     
         23 . The method of  claim 17 , further comprising performing a 2D match search in a 2D pictures database, the 2D pictures database comprising 2D pictures randomly generated from 3D models of known objects. 
     
     
         24 . The method of  claim 23 , wherein the 2D match search is performed before the recognition process, and wherein the recognition process is performed after the 2D match search fails to find a match. 
     
     
         25 . The method of  claim 23 , wherein the 2D pictures are randomly generated by performing at least one of generating different random lightings, generating different random points of view, or generating different random exposures. 
     
     
         26 . The method of  claim 17 , wherein reconstructing the 3D point cloud comprises:
 extracting a plurality of key points from the plurality of pictures;   defining the plurality of vertices, wherein a vertex of the plurality of vertices corresponds in 3D to a key point of the plurality of key points; and   deriving the 3D point cloud based on the plurality of vertices.   
     
     
         27 . The method of  claim 17 , wherein reconstructing the 3D point cloud comprises:
 extracting a plurality of key points from the plurality of pictures;   defining a plurality of 3D slices of the object, wherein each 3D slice of the plurality of 3D slices comprises at least one key point of the plurality of key points; and   constructing, based on the 3D slices, the 3D point cloud.   
     
     
         28 . A system comprising at least one processor and memory storing a plurality of executable instructions which, when executed by the at least one processor of the system, cause the system to:
 receive a plurality of pictures of an object;   reconstruct, from the plurality of pictures, a 3D point cloud of the object, wherein the 3D point cloud comprises a plurality of vertices in a 3D space;   extract, from the 3D point cloud, a plurality of 3D descriptors, wherein each descriptor comprises information about:
 at least one vertex of the plurality of vertices, and 
 a 3D primitive of the 3D point cloud; 
   encode the plurality of 3D descriptors into one or more 3D feature vectors;   perform a recognition process, based on the 3D feature vectors, to identify the object; and   output an identifier of the object.   
     
     
         29 . The system of  claim 28 , wherein the instructions that cause the system to perform the recognition process comprise instructions that cause the system to input the 3D feature vectors to a machine learning algorithm. 
     
     
         30 . The system of  claim 28 , wherein the instructions that cause the system to perform the recognition process comprise instructions that cause the system to execute a 3D geometric comparison by comparing the 3D point cloud with 3D models of known objects based on the one or more 3D feature vectors. 
     
     
         31 . The system of  claim 28 , wherein each 3D primitive is representative of a geometrical shape, and wherein the geometrical shape is a plane, sphere, cylinder, cube, or torus. 
     
     
         32 . A non-transitory computer-readable medium containing instructions which, when executed by a processor, cause the processor to:
 receive a plurality of pictures of an object;   reconstruct, from the plurality of pictures, a 3D point cloud of the object, wherein the 3D point cloud comprises a plurality of vertices in a 3D space;   extract, from the 3D point cloud, a plurality of 3D descriptors, wherein each descriptor comprises information about:
 at least one vertex of the plurality of vertices, and 
 a 3D primitive of the 3D point cloud; 
   encode the plurality of 3D descriptors into one or more 3D feature vectors;   perform a recognition process, based on the 3D feature vectors, to identify the object; and   output an identifier of the object.   
     
     
         33 . The non-transitory computer-readable medium of  claim 32 , wherein the instructions, when executed by the processor, cause the processor to perform a 2D match search in a 2D pictures database, the 2D pictures database comprising 2D pictures randomly generated from 3D models of known objects. 
     
     
         34 . The non-transitory computer-readable medium of  claim 32 , wherein the instructions that cause the processor to reconstruct the 3D point cloud comprise instructions that cause the processor to:
 extract a plurality of key points from the plurality of pictures;   define the plurality of vertices, wherein a vertex of the plurality of vertices corresponds in 3D to a key point of the plurality of key points; and   derive the 3D point cloud based on the 3D vertices.   
     
     
         35 . The non-transitory computer-readable medium of  claim 32 , wherein the instructions that cause the processor to reconstruct the 3D point cloud comprise instructions that cause the processor to:
 extract a plurality of key points from the plurality of pictures;   define a plurality of 3D slices of the object, wherein each 3D slice of the plurality of 3D slices comprises at least one key point of the plurality of key points; and   construct, based on the 3D slices, the 3D point cloud.   
     
     
         36 . The non-transitory computer-readable medium of  claim 32 , wherein the instructions that cause the processor to perform the recognition process comprise instructions that cause the processor to input the 3D feature vectors to a machine learning algorithm.