Providing remote assistance to an autonomous vehicle

Abstract

An event is detected that impairs a confidence level of the autonomous vehicle in progressing through a current route. In response to detecting the event, the autonomous vehicle communicates information about the event to a remote source of guidance. The autonomous vehicle can receive instructions from the remote source of guidance on how to handle the event. The autonomous vehicle can then implement the instructions to handle the event while it operates.

What is claimed is: 
     
         1 . A method for remotely communicating with an autonomous vehicle, the method being implemented by one or more processors and comprising:
 obtaining data including sensor information, the data representing an alert from the autonomous vehicle, the alert indicating that the autonomous vehicle is uncertain in traversing a segment of roadway while on a trip;   providing an interface for a human operator that is remote from the autonomous vehicle, the interface including a visual representation of an interior or an exterior of the autonomous vehicle and a set of interactive features that is generated from the sensor information;   detecting a first selection input from the remote human operator on a selected region of the interface;   identifying a specific set of sensors based on the selected region of the interface and the visual representation of the autonomous vehicle;   determining a request for additional sensor data from the specific set of sensors of the autonomous vehicle; and   communicating the request to the autonomous vehicle, the request causing the autonomous vehicle to obtain the additional sensor data from the specific set of sensors.   
     
     
         2 . The method of  claim 1 , further comprising:
 obtaining the additional sensor data from the autonomous vehicle; and   providing the additional sensor data for display on the interface.   
     
     
         3 . The method of  claim 2 , further comprising:
 detecting a second selection input from the remote human operator of one or more interactive features from the set of interactive features, the second selection input specifying a response strategy comprising a condition and one or more actions for the autonomous vehicle to perform without additional input from the remote human operator.   
     
     
         4 . The method of  claim 3 , further comprising:
 communicating the response strategy to the autonomous vehicle, the response strategy causing the autonomous vehicle to perform at least an action of the one or more actions, to monitor for the condition while approaching or traversing the segment of roadway, and in response to detecting the condition, to perform one or more actions to change a direction, speed, or driving lane of the autonomous vehicle without additional input from the remote human operator.   
     
     
         5 . The method of  claim 3 , wherein providing the interface includes displaying a time remaining for the remote human operator to provide the second selection input before the autonomous vehicle performs a default action. 
     
     
         6 . The method of  claim 1 , wherein the autonomous vehicle is uncertain in traversing the segment of roadway when an uncertainly level exceeds a first threshold, or a confidence value is less than a second threshold, with respect to the autonomous vehicle's capability to safely respond to an event or condition on the segment of roadway. 
     
     
         7 . The method of  claim 1 , wherein providing the interface includes superimposing one or more interactive features of the interface over content that is generated from the sensor information. 
     
     
         8 . The method of  claim 1 , wherein providing the interface includes displaying image content captured by one or more cameras of the autonomous vehicle as the autonomous vehicle approaches the segment of roadway. 
     
     
         9 . The method of  claim 1 , wherein providing the interface includes providing an interactive feature to enable the remote human operator to request additional information from the autonomous vehicle. 
     
     
         10 . The method of  claim 9 , wherein the interactive feature enables the remote human operator to request additional images from any one of multiple cameras on the autonomous vehicle. 
     
     
         11 . The method of  claim 9 , wherein the interactive feature enables the remote human operator to directionally control one or more sensors of the autonomous vehicle to obtain additional information from the autonomous vehicle. 
     
     
         12 . The method of  claim 1 , wherein providing the interface includes providing an interactive feature to enable the remote human operator to communicate an instruction to the autonomous vehicle to take no action to avoid a detected object or event that is causing uncertainty for the autonomous vehicle. 
     
     
         13 . The method of  claim 1 , wherein providing the interface includes providing multiple interactive features to enable the remote human operator to communicate one or more instructions that correspond to a steering or braking operation that is to be performed by the autonomous vehicle. 
     
     
         14 . The method of  claim 13 , wherein the multiple interactive features enable the remote human operator to communicate one or more instructions that specify a magnitude of the steering or braking operation. 
     
     
         15 . The method of  claim 1 , wherein providing the interface for the remote human operator includes providing one or more features that enable the remote human operator to view images that are above the autonomous vehicle or next to the segment of roadway. 
     
     
         16 . A non-transitory computer-readable medium that stores instructions for remotely communicating with an autonomous vehicle, the instructions, when executed by one or more processors, causing the one or more processors to perform operations comprising:
 obtaining data including sensor information, the data representing an alert from the autonomous vehicle, the alert indicating that the autonomous vehicle is uncertain in traversing a segment of roadway while on a trip;   providing an interface for a human operator that is remote from the autonomous vehicle, the interface including a visual representation of an interior or an exterior of the autonomous vehicle and a set of interactive features that is generated from the sensor information;   detecting a first selection input from the remote human operator on a selected region of the interface;   identifying a specific set of sensors based on the selected region of the interface and the visual representation of the autonomous vehicle;   determining a request for additional sensor data from the specific set of sensors of the autonomous vehicle; and   communicating the request to the autonomous vehicle, the request causing the autonomous vehicle to obtain the additional sensor data from the specific set of sensors.   
     
     
         17 . The non-transitory computer-readable medium of  claim 16 , storing further instructions to cause the one or more processors to perform operations comprising:
 obtaining the additional sensor data from the autonomous vehicle; and   providing the additional sensor data for display on the interface.   
     
     
         18 . The non-transitory computer-readable medium of  claim 17 , storing further instructions to cause the one or more processors to perform operations comprising:
 detecting a second selection input from the remote human operator of one or more interactive features from the set of interactive features, the second selection input specifying a response strategy comprising a condition and one or more actions for the autonomous vehicle to perform without additional input from the remote human operator.   
     
     
         19 . The non-transitory computer-readable medium of  claim 18 , storing further instructions to cause the one or more processors to perform operations comprising:
 communicating the response strategy to the autonomous vehicle, the response strategy causing the autonomous vehicle to perform at least an action of the one or more actions, to monitor for the condition while approaching or traversing the segment of roadway, and in response to detecting the condition, to perform one or more actions to change a direction, speed, or driving lane of the autonomous vehicle without additional input from the remote human operator.   
     
     
         20 . A computer system comprising:
 a memory to store a set of instructions;   a network interface; and   one or more processors that access instructions from the memory, to remotely communicate with an autonomous vehicle using the network interface, wherein the one or more processors:
 obtain data including sensor information, the data representing an alert from the autonomous vehicle, the alert indicating that the autonomous vehicle is uncertain in traversing a segment of roadway while on a trip; 
 provide an interface for a human operator that is remote from the autonomous vehicle, the interface including a visual representation of an interior or an exterior of the autonomous vehicle and a set of interactive features that is generated from the sensor information; 
 detect a first selection input from the remote human operator on a selected region of the interface; 
 identify a specific set of sensors based on the selected region of the interface and the visual representation of the autonomous vehicle; 
 determine a request for additional sensor data from the specific set of sensors of the autonomous vehicle; and 
 communicate the request to the autonomous vehicle, the request causing the autonomous vehicle to obtain the additional sensor data from the specific set of sensors.