System and method for concurrent odometry and mapping

Abstract

An electronic device tracks its motion in an environment while building a three-dimensional visual representation of the environment that is used to correct drift in the tracked motion. A motion tracking module estimates poses of the electronic device based on feature descriptors corresponding to the visual appearance of spatial features of objects in the environment. A mapping module builds a three-dimensional visual representation of the environment based on a stored plurality of maps, and feature descriptors and estimated device poses received from the motion tracking module. The mapping module provides the three-dimensional visual representation of the environment to a localization module, which identifies correspondences between stored and observed feature descriptors. The localization module performs a loop closure by minimizing the discrepancies between matching feature descriptors to compute a localized pose. The localized pose corrects drift in the estimated pose generated by the motion tracking module.

1 - 9 . (canceled) 
     
     
         10 . A method, comprising:
 tracking, at an electronic device, motion of the electronic device relative to an environment of the electronic device based on feature descriptors of one or more spatial features representing one or more objects in the environment of the electronic device, wherein the feature descriptors are generated based on images captured from one or more visual sensors and non-visual data from one or more non-image sensors at a first rate;   generating, at the electronic device, a three-dimensional visual representation of the environment based on a plurality of stored maps comprising stored feature descriptors at a second rate; and   matching the one or more spatial features to spatial features of the plurality of stored maps to generate a first localized pose of the electronic device.   
     
     
         11 . The method of  claim 10 , further comprising updating the three-dimensional visual representation by adding the tracked motion to the three-dimensional visual representation of the environment. 
     
     
         12 . The method of  claim 11 , wherein adding the tracked motion comprises generating linearization points based on the generated feature descriptors and solving a non-linear estimation of the three-dimensional visual representation based on the linearization points and the stored feature descriptors. 
     
     
         13 . The method of  claim 12 , wherein the generated feature descriptors are periodically added to the three-dimensional visual representation of the environment. 
     
     
         14 . The method of  claim 10 , further comprising identifying discrepancies between the matched generated feature descriptors and stored feature descriptors and minimizing the discrepancies. 
     
     
         15 . The method of  claim 10 , wherein the first rate is higher than the second rate. 
     
     
         16 . An electronic device, comprising:
 a motion tracking module configured to track motion of the electronic device relative to an environment of the electronic device based on feature descriptors of one or more spatial features representing one or more objects in the environment of the electronic device, wherein the feature descriptors are generated based on images captured from one or more visual sensors and non-visual data from one or more non-image sensors at a first rate;   a mapping module configured to generate a three-dimensional visual representation of the environment of the electronic device based on a plurality of stored maps comprising stored feature descriptors at a second rate; and   a localization module configured to match the one or more spatial features to spatial features of the plurality of stored maps to generate a localized pose of the electronic device.   
     
     
         17 . The electronic device of  claim 16 , wherein the mapping module is further to update the three-dimensional visual representation by adding the tracked motion to the three-dimensional visual representation of the environment. 
     
     
         18 . The electronic device of  claim 17 , wherein the mapping module is further to generate linearization points based on generated feature descriptors and solve a non-linear estimation of the three-dimensional visual representation based on the linearization points and the stored feature descriptors. 
     
     
         19 . The electronic device of  claim 17 , wherein the mapping module is to periodically add the generated feature descriptors to the three-dimensional visual representation of the environment. 
     
     
         20 . The electronic device of  claim 16 , wherein the first rate is higher than the second rate. 
     
     
         21 . The electronic device of  claim 16 , wherein the localization module is further to identify discrepancies between the matched generated feature descriptors and stored feature descriptors and minimize the discrepancies.