Large-scale environmental mapping in real-time by a robotic system

Abstract

According to one implementation, a robotic system for performing large-scale environmental mapping in real-time includes a mobile reconnaissance unit having a color sensor, a depth sensor, and a graphics processing unit (GPU) with a GPU memory, and a navigation unit communicatively coupled to the mobile reconnaissance unit and having a central processing unit (CPU) with a CPU memory. The robotic system begins a three-dimensional (3D) scan of an environment of the mobile reconnaissance unit using the color sensor and the depth sensor, and generates mapping data for populating a volumetric representation of the environment. The robotic system continues the 3D scan of the environment using the color sensor and the depth sensor, updates the mapping data, and partitions the volumetric representation between the GPU memory and the CPU memory based on a memory allocation criteria.

What is claimed is: 
     
       1. A robotic system comprising:
 a plurality of mobile reconnaissance units including a mobile reconnaissance unit having a color sensor, a depth sensor, and a graphics processing unit (GPU) with a GPU memory; 
 a navigation unit communicatively coupled to the plurality of mobile reconnaissance units, the navigation unit having a central processing unit (CPU) with a CPU memory, wherein the navigation unit is a remote base station for all of the plurality of mobile reconnaissance units; 
 the robotic system configured to:
 begin a three-dimensional (3D) scan of an environment of the mobile reconnaissance unit using the color sensor and the depth sensor; 
 generate a mapping data for populating a volumetric representation of the environment produced using the GPU; 
 continue the 3D scan of the environment using the color sensor and the depth sensor; 
 update the mapping data based on the continued 3D scan; and 
 partition the volumetric representation between the GPU memory and the CPU memory based on a memory allocation criteria; 
 
 wherein the volumetric representation of the environment is stored partially in the GPU memory of the mobile reconnaissance unit and partially in the CPU memory of the navigation unit. 
 
     
     
       2. The robotic system of  claim 1 , wherein the navigation unit further comprises a display screen, and wherein the robotic system is further configured to display a visual representation of the environment of the mobile reconnaissance unit on the display screen. 
     
     
       3. The robotic system of  claim 1 , wherein the navigation unit is integrated with the mobile reconnaissance unit. 
     
     
       4. The robotic system of  claim 1 , wherein the mobile reconnaissance unit comprises an aerial vehicle. 
     
     
       5. The robotic system of  claim 1 , wherein the mobile reconnaissance unit comprises one of a wheeled vehicle, a legged vehicle, and a continuous track propulsion vehicle. 
     
     
       6. The robotic system of  claim 1 , wherein the mobile reconnaissance unit comprises a submersible vehicle. 
     
     
       7. A robotic system comprising:
 a mobile reconnaissance unit having a color sensor, a depth sensor, and a graphics processing unit (GPU) with a GPU memory; 
 a navigation unit communicatively coupled to the mobile reconnaissance unit, the navigation unit having a central processing unit (CPU) with a CPU memory; 
 the robotic system configured to:
 begin a three-dimensional (3D) scan of an environment of the mobile reconnaissance unit using the color sensor and the depth sensor; 
 generate a mapping data for populating a volumetric representation of the environment produced using the GPU; 
 continue the 3D scan of the environment using the color sensor and the depth sensor; 
 update the mapping data based on the continued 3D scan; and 
 partition the volumetric representation between the GPU memory and the CPU memory based on a memory allocation criteria; 
 
 wherein the volumetric representation of the environment is stored partially in the GPU memory of the mobile reconnaissance unit and partially in the CPU memory of the navigation unit, wherein the mobile reconnaissance unit further comprises an inertial sensor, and wherein the robotic system is further configured to: 
 detect an abrupt movement of the mobile reconnaissance unit during the second 3D scan, using the inertial sensor; 
 generate a perturbation data corresponding to the abrupt movement; and 
 correct the mapping data to compensate for the abrupt movement using the perturbation data. 
 
     
     
       8. The robotic system of  claim 7 , wherein the navigation unit is a remote base station for controlling the mobile reconnaissance unit. 
     
     
       9. The robotic system of  claim 7 , wherein the mobile reconnaissance unit is one of a plurality of mobile reconnaissance units communicatively coupled to the navigation unit, and wherein the navigation unit is a remote base station for all of the plurality of mobile reconnaissance units. 
     
     
       10. A robotic system comprising:
 a mobile reconnaissance unit having a color sensor, a depth sensor, and a graphics processing unit (GPU) with a GPU memory; 
 a navigation unit communicatively coupled to the mobile reconnaissance unit, the navigation unit having a central processing unit (CPU) with a CPU memory; 
 the robotic system configured to:
 begin a three-dimensional (3D) scan of an environment of the mobile reconnaissance unit using the color sensor and the depth sensor; 
 generate a mapping data for populating a volumetric representation of the environment produced using the GPU; 
 continue the 3D scan of the environment using the color sensor and the depth sensor; 
 update the mapping data based on the continued 3D scan; and 
 partition the volumetric representation between the GPU memory and the CPU memory based on a memory allocation criteria; 
 
 wherein the volumetric representation of the environment is stored partially in the GPU memory of the mobile reconnaissance unit and partially in the CPU memory of the navigation unit, and wherein the color sensor is an RGB camera, and the RGB camera and the depth sensor form an integrated RGB-D sensor of the mobile reconnaissance unit. 
 
     
     
       11. A method for use by a robotic system comprising a plurality of mobile reconnaissance units including a mobile reconnaissance unit having a color sensor, a depth sensor, and a graphics processing unit (GPU) with a GPU memory, and further including a navigation unit communicatively coupled to the plurality of mobile reconnaissance units and having a central processing unit (CPU) with a CPU memory, the navigation unit being a remote base station for all of the plurality of mobile reconnaissance units, the method comprising:
 beginning a three-dimensional (3D) scan of an environment of the mobile reconnaissance unit using the color sensor and the depth sensor; 
 generating a mapping data for populating a volumetric representation of the environment produced using the GPU; 
 continuing the 3D scan of the environment using the color sensor and the depth sensor; 
 updating the mapping data based on the continued 3D scan; and 
 partitioning the volumetric representation between the GPU memory and the CPU memory based on a memory allocation criteria; 
 wherein the volumetric representation of the environment is stored partially in the GPU memory of the mobile reconnaissance unit and partially in the CPU memory of the navigation unit. 
 
     
     
       12. The method of  claim 11 , wherein the navigation unit further includes a display screen, and wherein the method further comprises displaying a visual representation of the environment of the mobile reconnaissance unit on the display screen. 
     
     
       13. The method of  claim 11 , wherein the navigation unit is integrated with the mobile reconnaissance unit. 
     
     
       14. The method of  claim 11 , wherein the mobile reconnaissance unit comprises an aerial vehicle. 
     
     
       15. The method of  claim 11 , wherein the mobile reconnaissance unit comprises one of a wheeled vehicle, a legged vehicle, and a continuous track propulsion vehicle. 
     
     
       16. The method of  claim 11 , wherein the mobile reconnaissance unit comprises a submersible vehicle. 
     
     
       17. A method for use by a robotic system including a mobile reconnaissance unit having a color sensor, a depth sensor, and a graphics processing unit (GPU) with a GPU memory, and further including a navigation unit communicatively coupled to the mobile reconnaissance unit and having a central processing unit (CPU) with a CPU memory, the method comprising:
 beginning a three-dimensional (3D) scan of an environment of the mobile reconnaissance unit using the color sensor and the depth sensor; 
 generating a mapping data for populating a volumetric representation of the environment produced using the GPU; 
 continuing the 3D scan of the environment using the color sensor and the depth sensor; 
 updating the mapping data based on the continued 3D scan; and 
 partitioning the volumetric representation between the GPU memory and the CPU memory based on a memory allocation criteria; 
 wherein the volumetric representation of the environment is stored partially in the GPU memory of the mobile reconnaissance unit and partially in the CPU memory of the navigation unit, wherein the mobile reconnaissance unit further includes an inertial sensor, and wherein the method further comprises: 
 detecting a abrupt movement of the mobile reconnaissance unit during the second 3D scan, using the inertial sensor; 
 generating a perturbation data corresponding to the abrupt movement; and 
 correcting the mapping data to compensate for the abrupt movement using the perturbation data. 
 
     
     
       18. The method of  claim 17 , wherein the navigation unit is a remote base station for controlling the mobile reconnaissance unit. 
     
     
       19. The method of  claim 17 , wherein the mobile reconnaissance unit is one of a plurality of mobile reconnaissance units communicatively coupled to the navigation unit, and wherein the navigation unit is a remote base station for all of the plurality of mobile reconnaissance units. 
     
     
       20. A method for use by a robotic system including a mobile reconnaissance unit having a color sensor, a depth sensor, and a graphics processing unit (GPU) with a GPU memory, and further including a navigation unit communicatively coupled to the mobile reconnaissance unit and having a central processing unit (CPU) with a CPU memory, the method comprising:
 beginning a three-dimensional (3D) scan of an environment of the mobile reconnaissance unit using the color sensor and the depth sensor; 
 generating a mapping data for populating a volumetric representation of the environment produced using the GPU; 
 continuing the 3D scan of the environment using the color sensor and the depth sensor; 
 updating the mapping data based on the continued 3D scan; and 
 partitioning the volumetric representation between the GPU memory and the CPU memory based on a memory allocation criteria; 
 wherein the volumetric representation of the environment is stored partially in the GPU memory of the mobile reconnaissance unit and partially in the CPU memory of the navigation unit, and wherein the color sensor is an RGB camera, and the RGB camera and the depth sensor form an integrated RGB-D sensor of the mobile reconnaissance unit.