System and method for robotic object detection using a convolutional neural network

Abstract

A system includes a mobile robot, the robot comprising a sensor; and a server operably connected to the robot over a network, the robot being configured to detect an object by processing sensor data using a convolutional neural network. A pipeline for robotic object detection using a convolutional neural network includes: a system comprising a mobile robot, the robot comprising a sensor, the system further comprising a server operably connected to the robot over a network, the robot being configured to detect an object by processing sensor data using a pipeline, the pipeline comprising a convolutional neural network, the pipeline configured to perform a data collection step, the pipeline further configured to perform a data transformation step, the pipeline further configured to perform a convolutional neural network step, the pipeline further configured to perform a network output transformation step, the pipeline further configured to perform a results output step.

What is claimed is: 
     
       1. A system comprising:
 a mobile robot, the robot comprising a two-dimensional (2D) sensor, the 2D sensor configured to provide spatial data about a frontward facing surface of one or more nearby objects, the 2D sensor further configured to provide infrared data related to one or more of a shape, a size, a type, a reflectivity, and a location of the object, 
 the robot further comprising a three-dimensional (3D) sensor, the 3D sensor configured to provide spatial data about the frontward facing surface of one or more nearby objects, the 3D sensor further configured to provide infrared data related to one or more of the shape, the size, the type, the reflectivity, and the location of the object; and 
 a server operably connected to the robot over a network, the server configured to manage the robot, the server further configured to provide the robot with location data regarding one or more of a location of the robot, a destination of the robot, and the location of the object; 
 the robot being configured to detect the object by processing sensor data using a convolutional neural network, the convolutional neural network being configured to determine one or more of information about the location of the object and information about the type of the object; 
 the server being configured to train the convolutional neural network while the server is not managing the robot by being in an offline status, 
 wherein the training comprises using a plurality of examples of an input to the convolutional neural network and a corresponding desired output from the convolutional neural network. 
 
     
     
       2. The system of  claim 1 , wherein the 2D sensor comprises a light detection and ranging (LIDAR) sensor. 
     
     
       3. The system of  claim 1 , wherein the object is selected from the group consisting of another robot, a forklift, a golf cart, an autonomous guided vehicles (AGV), a vehicle, and a shelf. 
     
     
       4. The system of  claim 1 , wherein the type of the object is selected from the group consisting of a vehicle, a robot, a cart, a landmark, a stationary object, a warehouse, and an inventory shelf. 
     
     
       5. The system of  claim 1 , wherein one or more of the 2D sensor and the 3D sensor is further configured to detect an object that is onboard the robot.