3d asset inspection

Abstract

Systems and methods for physical asset inspection are provided. According to one embodiment, a probe is positioned to multiple data capture positions with reference to a physical asset. For each position: odometry data is obtained from an encoder and/or an IMU; a 2D image is captured by a camera; a 3D sensor data frame is captured by a 3D sensor, having a view plane overlapping that of the camera; the odometry data, the 2D image and the 3D sensor data frame are linked and associated with a physical point in real-world space based on the odometry data; and switching between 2D and 3D views within the collected data is facilitated by forming a set of points containing both 2D and 3D data by performing UV mapping based on a known positioning of the camera relative to the 3D sensor.

What is claimed is: 
     
         1 . A method comprising:
 positioning a probe to a plurality of data capture positions with reference to a physical asset;   for each data capture position of the plurality of data capture positions:
 collecting data regarding the physical asset by performing a data collection process including:
 reading, by a central processing unit (CPU) of the probe, odometry data from one or more of an encoder and an inertial measurement unit (IMU) attached to or integrated with the probe; 
 capturing, by a camera attached to or integrated with the probe having a first view plane, one or more two-dimensional (2D) images; and 
 capturing, by a three-dimensional (3D) sensor attached to or integrated with the probe and having a second view plane overlapping that of the first view plane, one or more 3D sensor data frames; and 
 
 performing a data synthesis process including:
 linking, by the CPU, the odometry data, the one or more 2D images and the one or more 3D sensor data frames; and 
 associating, by the CPU, the one or more 2D images and the one or more 3D sensor data frames with a physical point in real-world space based on the odometry data; and 
 facilitating subsequent ability on behalf of a user navigating the collected data to switch between a 2D view and a 3D view by forming, by the CPU, a set of points each containing both 3D data and 2D data by performing UV mapping based on the one or more 2D images, the one or more 3D sensor data frames and based on a known physical geometry of positioning of the camera relative to the 3D sensor. 
 
   
     
     
         2 . The method of  claim 1 , wherein the data synthesis process is performed for a particular data capture position of the plurality of data capture positions after the data collection process has been completed for the particular data capture position. 
     
     
         3 . The method of  claim 1 , wherein the 3D sensor comprises a LiDAR sensor, a time of flight camera, a structured light camera or a laser displacement sensor and wherein the one or more 3D sensor data frames include information regarding distance between the 3D sensor and a surface of the physical asset. 
     
     
         4 . The method of  claim 1 , wherein the probe comprises a robot and wherein said positioning is guided or controlled by a computer program running on the CPU. 
     
     
         5 . The method of  claim 1 , wherein the probe comprises a tethered probe that is suspended by one or more cables or belts and wherein said positioning is via an external winch system. 
     
     
         6 . A probe comprising:
 one or more of an encoder and an inertial measurement unit (IMU);   a camera having a first view plane;   a three-dimensional (3D) sensor having a known physical geometry of positioning relative to the camera and having a second view plane overlapping that of the first view plane;   a non-transitory storage device having embodied therein one or more routines; and   a central processing unit (CPU) coupled to the non-transitory storage device, said one or more of an encode and an IMU, the camera and the 3D sensor and operable to execute the one or more routines to perform a method comprising:   for each data capture position of a plurality of data capture positions at which the probe is positioned with reference to a physical asset:
 collecting data regarding the physical asset by performing a data collection process including:
 reading odometry data from said one or more of the encoder and the IMU; 
 causing the camera to capture one or more two-dimensional (2D) images; and 
 causing the 3D sensor to capture one or more 3D sensor data frames; and 
 
 performing a data synthesis process including:
 linking the odometry data, the one or more 2D images and the one or more 3D sensor data frames; and 
 associating the one or more 2D images and the one or more 3D sensor data frames with a physical point in real-world space based on the odometry data; and 
 facilitating subsequent ability on behalf of a user navigating the collected data to switch between a 2D view and a 3D view by forming a set of points each containing both 3D data and 2D data by performing UV mapping based on the one or more 2D images, the one or more 3D sensor data frames and based on the known physical geometry of positioning of the 3D sensor relative to the camera. 
 
   
     
     
         7 . The probe of  claim 6 , wherein the data synthesis process is performed for a particular data capture position of the plurality of data capture positions after the data collection process has been completed for the particular data capture position. 
     
     
         8 . The probe of  claim 6 , wherein the 3D sensor comprises a LiDAR sensor, a time of flight camera, a structured light camera or a laser displacement sensor and wherein the one or more 3D sensor data frames include information regarding distance between the 3D sensor and a surface of the physical asset. 
     
     
         9 . The probe of  claim 6 , wherein the probe comprises a robot and wherein the robot is positioned at the plurality of data capture positions as a result of being guided or controlled at least in part by a computer program running on the CPU. 
     
     
         10 . The probe of  claim 9 , wherein the robot comprises a legged or a wheeled robot. 
     
     
         11 . The probe of  claim 6 , wherein the physical asset comprises a manhole and wherein the probe comprises a tethered probe that is suspended by a plurality of cables or belts and wherein the probe is positioned at the plurality of data capture positions within the manhole via an external winch system. 
     
     
         12 . A non-transitory computer-readable storage medium embodying a set of instructions, which when executed by one or more processors of a probe, cause the one or more processors to perform a method comprising:
 for each data capture position of a plurality of data capture positions at which the probe is positioned with reference to a physical asset:
 collecting data regarding the physical asset by performing a data collection process including:
 reading odometry data from one or more of an encoder and an inertial measurement unit (IMU) attached to or integrated with the probe; 
 causing a camera, attached to or integrated with the probe having a first view plane, to capture one or more two-dimensional (2D) images; and 
 causing a three-dimensional (3D) sensor, attached to or integrated with the probe and having a second view plane overlapping that of the first view plane, to capture one or more 3D sensor data frames; and 
 
 performing a data synthesis process including:
 linking the odometry data, the one or more 2D images and the one or more 3D sensor data frames; and 
 associating the one or more 2D images and the one or more 3D sensor data frames with a physical point in real-world space based on the odometry data; and 
 facilitating subsequent ability on behalf of a user navigating the collected data to switch between a 2D view and a 3D view by forming a set of points each containing both 3D data and 2D data by performing UV mapping based on the one or more 2D images, the one or more 3D sensor data frames and based on a known physical geometry of positioning of the camera relative to the 3D sensor. 
 
   
     
     
         13 . The non-transitory computer-readable storage medium of  claim 12 , wherein the data synthesis process is performed for a particular data capture position of the plurality of data capture positions after the data collection process has been completed for the particular data capture position. 
     
     
         14 . The non-transitory computer-readable storage medium of  claim 12 , wherein the 3D sensor comprises a LiDAR sensor, a time of flight camera, a structured light camera or a laser displacement sensor and wherein the one or more 3D sensor data frames include information regarding distance between the 3D sensor and a surface of the physical asset. 
     
     
         15 . The non-transitory computer-readable storage medium of  claim 12 , wherein the probe comprises a robot and wherein the robot is positioned at the plurality of data capture positions as a result of being guided or controlled at least in part by a computer program running on the one or more processors. 
     
     
         16 . The non-transitory computer-readable storage medium of  claim 12 , wherein the probe comprises a tethered probe that is suspended by a plurality of cables or belts and wherein the probe is positioned at the plurality of data capture positions via an external winch system.