Visual-Inertial Positional Awareness for Autonomous and Non-Autonomous Tracking

Abstract

The described positional awareness techniques employing visual-inertial sensory data gathering and analysis hardware with reference to specific example implementations implement improvements in the use of sensors, techniques and hardware design that can enable specific embodiments to provide positional awareness to machines with improved speed and accuracy.

What is claimed is: 
     
         1 . A mobile platform, including:
 at least one camera; and   one or more processors coupled to memory loaded with executable instructions to guide the mobile platform using updates to a position of the mobile platform, which instructions, when executed on the processors, implement actions comprising:
 obtaining a perspective from which a new frame was viewed by the camera from a visually corrected pose calculated using positions of matching features in a set of matching features determined using feature points extracted from the new frame that match feature points retrieved from a field of view within a propagated pose from a 3D map; and 
 responsive to requests for location of the mobile platform, and/or the perspective of the camera, providing data based on one or both of the propagated pose and the visually corrected pose. 
   
     
     
         2 . The mobile platform of  claim 1 , further including a multi-axis inertial measuring unit (IMU) including at least one of a gyroscope and an accelerometer and wherein the propagated pose is generated by updating an initial pose using inertial data from the multi-axis inertial measuring unit (IMU). 
     
     
         3 . The mobile platform of  claim 1 , further implementing actions comprising retrieving at least some feature points from a point cloud hybrid map. 
     
     
         4 . The mobile platform of  claim 3 , wherein matching features in a set of matching features are determined based on (1) reuse of matched features from a previous frame and (2) matching of features in the new frame with reprojected feature positions from the 3D map onto a 2D view from a perspective of the propagated pose. 
     
     
         5 . The mobile platform of  claim 4 , further implementing actions comprising using the 2D view from the perspective of the propagated pose to plan a path of travel. 
     
     
         6 . The mobile platform of  claim 5 , further implementing actions comprising using the point cloud hybrid map while travelling the path of travel as planned to avoid colliding with obstacles. 
     
     
         7 . The mobile platform of  claim 1 , wherein features are matched by smartly searching the 3D map, using previous time period velocity/acceleration to predict pose, thereby narrowing search region, including:
 performing a linear search when a count of unmatched features is below a threshold; and   otherwise performing a log(n) search using a kd-tree or an octree.   
     
     
         8 . The mobile platform of  claim 1 , further implementing actions comprising correcting drift between the propagated pose, and an actual perspective of a new pose, using the new frame captured by the camera, by using the propagated pose, estimating an overlap between successive camera frames to reduce computation requirements, correlating the new frame with a previous frame by 2D comparison of the successive camera frames, beginning with the overlap estimated. 
     
     
         9 . A method of updating a position of a mobile unit that includes a camera, the method including:
 obtaining a perspective from which a new frame was viewed by the camera from a visually corrected pose calculated using positions of matching features in a set of matching features determined using feature points extracted from the new frame that match feature points retrieved from a field of view within a propagated pose from a 3D map; and   responsive to requests for location of the mobile unit, and/or the perspective of the camera, providing data based on one or both of the propagated pose and the visually corrected pose.   
     
     
         10 . The method of  claim 9 , wherein the propagated pose is generated by updating an initial pose using inertial data from the inertial measuring unit (IMU) comprising one or more of accelerometers configured to measure linear acceleration and gyroscopes configured to measure rotational rate. 
     
     
         11 . The method of  claim 9 , further including retrieving at least some feature points from a point cloud hybrid map. 
     
     
         12 . The method of  claim 11 , wherein matching features in a set of matching features are determined based on (1) reuse of matched features from a previous frame and (2) matching of features in the new frame with reprojected feature positions from the 3D map onto a 2D view from a perspective of the propagated pose. 
     
     
         13 . The method of  claim 12 , further including using the 2D view from the perspective of the propagated pose to plan a path of travel. 
     
     
         14 . The method of  claim 13 , further including using the point cloud hybrid map while travelling the path of travel as planned to avoid colliding with obstacles. 
     
     
         15 . The method of  claim 9 , wherein features are matched by smartly searching the 3D map using previous time period velocity/acceleration to predict pose, thereby narrowing search region, further including:
 performing a linear search when a count of unmatched features is below a threshold; and   otherwise performing a log(n) search using a kd-tree or an octree.   
     
     
         16 . A non-transitory computer readable storage medium impressed with computer program instructions to update a position of a mobile unit that includes a camera, which instructions, when executed on a processor, implement:
 obtaining a perspective from which a new frame was viewed by the camera from a visually corrected pose calculated using positions of matching features in a set of matching features determined using feature points extracted from the new frame that match feature points retrieved from a field of view within a propagated pose from a 3D map; and   responsive to requests for location of the mobile unit, and/or the perspective of the camera, providing data based on one or both of the propagated pose and the visually corrected pose.   
     
     
         17 . The non-transitory computer readable storage medium of  claim 16 , wherein the propagated pose is generated by updating an initial pose using inertial data from an inertial measuring unit (IMU) comprising one or more of accelerometers configured to measure linear acceleration and gyroscopes configured to measure rotational rate. 
     
     
         18 . The non-transitory computer readable storage medium of  claim 16 , further implementing retrieving at least some feature points from a point cloud hybrid map. 
     
     
         19 . The non-transitory computer readable storage medium of  claim 18 , wherein matching features in a set of matching features is determined based on (1) reuse of matched features from a previous frame and (2) matching of features in the new frame with reprojected feature positions from the 3D map onto a 2D view from a perspective of the propagated pose. 
     
     
         20 . The non-transitory computer readable storage medium of  claim 19 , further implementing using the 2D view from the perspective of the propagated pose to plan a path of travel. 
     
     
         21 . The non-transitory computer readable storage medium of  claim 20 , further implementing using the point cloud hybrid map while travelling the path of travel as planned to avoid colliding with obstacles.