Navigating a Mobile Robot

Abstract

A method tor controlling a robot includes receiving image data from at least one image sensor. The image data corresponds to an environment about the robot. The method also includes executing a graphical user interface configured to display a scene of the environment based on the image data and receive an input indication indicating selection of a pixel location within the scene. The method also includes determining a pointing vector based on the selection of the pixel location. The pointing vector represents a direction of travel for navigating the robot in the environment. The method also includes transmitting a waypoint command to the robot. The waypoint command when received by the robot causes the robot to navigate to a target location. The target location is based on an intersection between the pointing vector and a terrain estimate of the robot.

What is claimed is: 
     
         1 . A computer-implemented method when executed by data processing hardware of an operator device in communication with a robot causes the data processing hardware to perform operations comprising:
 receiving image data from at least one image sensor, the image data corresponding to an environment of the robot; and   executing a graphical user interface for display on a screen of the operator device, the graphical user interface configured to:
 display a scene of the environment of the robot based on the image data; 
 receive a rotation input to rotate a field of view of the environment of the robot in a direction away from a current scene displayed in the graphical user interface; 
 display a preview scene by rotating the field of view of the environment of the robot in the direction away from the current scene; and 
 wherein the graphical user interlace is configured to display the preview scene without requiring physical movement by a body of the robot. 
   
     
     
         2 . The method of  claim 1 , wherein the scene of the environment of the robot is based on a virtual rectangle of image data representing the field of view of the environment. 
     
     
         3 . The method of  claim 2 , wherein rotating the field of view of the environment comprises adjusting the virtual rectangle of image data. 
     
     
         4 . The method of  claim 2 , wherein displaying the scene of the environment comprises tracing points of the virtual rectangle to corresponding pixels of the at least one image sensor to fill the points of the virtual rectangle with color based on the corresponding pixels and displaying the scene based on the filled points of the virtual rectangle. 
     
     
         5 . The method of  claim 4 , wherein, when points of the virtual rectangle are unable to he traced to corresponding pixels of the at least one image sensor, the points that are unable to be traced are filled with a default color. 
     
     
         6 . The method of  claim 4 , wherein a first portion of points of the virtual rectangle are traced to corresponding pixels of a first image sensor of the at least one image sensor and a second portion of points of the virtual rectangle are traced to corresponding pixels of a second image sensor of the at least one image sensor. 
     
     
         7 . The method of  claim 1 , wherein:
 with the current scene displayed in the graphical user interface, the body of the robot faces a first direction;   the preview scene represents the field of view of the environment with the body of the robot facing a second direction;   the rotation input includes a command for the robot to execute a turning maneuver to physically move the body of the robot from facing the first direction to face the second direction; and   the graphical user interface displays the preview scene before the body of the robot physically faces the second direction.   
     
     
         8 . The method of  claim 1 , wherein the graphical user interface is configured to receive the rotation input in response to receiving an input indication indicating selection of a rotation graphic displayed in the graphical user interface. 
     
     
         9 . The method of  claim 8 , wherein the rotation input causes the field of view of the environment to rotate proportional to a magnitude of the input indication. 
     
     
         10 . The method of  claim 1 , wherein the rotation of the field of view of the environment about the robot in the direction away from the current scene simulates the robot executing a turning maneuver in the direction away from the current scene and toward the preview scene. 
     
     
         11 . The method of  claim 1 , wherein the scene of the environment about the robot comprises any one of:
 a forward scene of the environment based on the image data, the image data captured by a forward-left camera and a forward-right camera disposed on the robot;   a left scene of the environment based on the image data, the image data captured by a left camera disposed on the robot;   a right scene of the environment based on the image data, the image data captured by a right camera disposed on the robot;   an aft scene of the environment based on the image data, the aft scene captured by an aft camera disposed on the robot; or   a top-down scene of the robot based on the image data, the image data captured by a payload camera, the forward-left camera, the forward-right camera, the left camera, the right camera, and the aft camera.   
     
     
         12 . The method of  claim 1 , wherein the robot comprises a quadruped robot. 
     
     
         13 . The method of  claim 1 , wherein:
 the at least one image sensor is disposed on the robot; and   the operator device is in communication with the image sensor via a network.   
     
     
         14 . A system comprising:
 data processing hardware of an operator device in communication with a robot; and   memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising:
 receiving image data from at least one image sensor, the image data corresponding to an environment of the robot; and 
 executing a graphical user interface for display on a screen of the operator device, the graphical user interface configured to:
 display a scene of the environment of the robot based on the image data; 
 receive a rotation input to rotate a field of view of the environment of the robot in a direction away from a current scene displayed in the graphical user interface; 
 display a preview scene by rotating the field of view of the environment of the robot in the direction away from the current scene; and 
 wherein the graphical user interface is configured to display the preview scene without requiring physical movement by a body of the robot. 
 
   
     
     
         15 . The system of  claim 14 , wherein the scene of the environment of the robot is based on a virtual rectangle of image data representing the field of view of the environment. 
     
     
         16 . The system of  claim 15 , wherein rotating the field of view of the environment comprises adjusting the virtual rectangle of image data. 
     
     
         17 . The system of  claim 15 , wherein displaying the scene of the environment comprises tracing points of the virtual rectangle to corresponding pixels of the at least one image sensor to fill the points of the virtual rectangle with color based on the corresponding pixels and displaying the scene based on the filled points of the virtual rectangle. 
     
     
         18 . The system of  claim 17 , wherein, when points of the virtual rectangle are unable to be traced to corresponding pixels of the at least one image sensor, the points that are unable to be traced are filled with a default color. 
     
     
         19 . The system of  claim 17 , wherein a first portion of points of the virtual rectangle are traced to corresponding pixels of a first image sensor of the at least one image sensor and a second portion of points of the virtual rectangle are traced to corresponding pixels of a second image sensor of the at least one image sensor. 
     
     
         20 . The system of  claim 14 , wherein:
 with the current scene displayed in the graphical user interface, the body of the robot faces a first direction;   the preview scene represents the field of view of the environment with the body of the robot facing a second direction;   the rotation input includes a command for the robot to execute a turning maneuver to physically move the body of the robot from facing the first direction to face the second direction; and   the graphical user interface displays the preview scene before the body of the robot physically faces the second direction.   
     
     
         21 . The system of  claim 14 , wherein the graphical user interface is configured to receive the rotation input in response to receiving an input indication indicating selection of a rotation graphic displayed in the graphical user interface. 
     
     
         22 . The system of  claim 21 , wherein the rotation input causes the field of view of the environment to rotate proportional to a magnitude of the input indication. 
     
     
         23 . The system of  claim 14 , wherein the rotation of the field of view of the environment about the robot in the direction away from the current scene simulates the robot executing a turning maneuver in the direction away from the current scene and toward the preview scene. 
     
     
         24 . The system of  claim 14 , wherein the scene of the environment about the robot comprises any one of:
 a forward scene of the environment based on the image data, the image data captured by a forward-left camera and a forward-right camera disposed on the robot;   a left scene of the environment based on the image data, the image data captured by a left camera disposed on the robot;   a right scene of the environment based on the image data, the image data captured by a right camera disposed on the robot;   an aft scene of the environment based on the image data, the aft scene captured by an aft camera disposed on the robot; or   a top-down scene of the robot based on the image data, the image data captured by a payload camera, the forward-left camera, the forward-right camera, the left camera, the right camera, and the aft camera.   
     
     
         25 . The system of  claim 14 , wherein the robot comprises a quadruped robot. 
     
     
         26 . The system of  claim 14 , wherein:
 the at least one image sensor is disposed on the robot, and   the operator device is in communication with the image sensor via a network.