Automatic Imposition of Vehicle Speed Restrictions Depending on Road Situation Analysis

Abstract

In a vehicle teleoperation session, a speed limit is determined for which the vehicle can be safely teleoperated. A safety system senses data relating to the vehicle environment and generates a depth map from which obstacles in the vicinity of the vehicle can be identified. Based on the detected obstacles and a motion state of the vehicle, a speed limit is determined at which the teleoperator is predicted to be able to utilize an emergency braking command to avoid a collision. the speed limit may be automatically applied to the vehicle or may be provided to the teleoperator to enable the teleoperator to adjust the vehicle speed.

1 . A method for operating a vehicle comprising:
 establishing a connection between the vehicle and a remote support server for enabling a teleoperation session with a remote teleoperator;   during the teleoperation session, receiving sensor data from a sensor array of the vehicle indicative of a vehicle motion state and estimated locations of one or more obstacles in an environment of the vehicle, the sensor data including a video feed associated with the vehicle;   during the teleoperation session, obtaining control signals for controlling the vehicle from the remote support server that are generated by the remote teleoperator based on the video feed;   obtaining network latency information indicative of network latency introduced under detected or predicted network conditions that affect a time for the sensor data including the video feed to be transmitted to the remote support server and for the control signals to be transmitted from the remote support server to the vehicle during the teleoperation session;   modeling a teleoperator reaction time for reacting to the video feed based at least in part on configured video processing parameters of the video feed;   determining, based on the vehicle motion state, the network latency information, the modeled teleoperator reaction time, and the estimated locations of the one or more obstacles, a speed limit for limiting speed of the vehicle during the teleoperation session that enables safe teleoperation of the vehicle;   determining if the vehicle is operating above the speed limit; and   initiating a remedial action responsive to the vehicle operating above the speed limit.   
     
     
         2 . The method of  claim 1 , wherein modeling the teleoperator reaction time comprises:
 modeling at least one of a camera exposure time, a video frame onboard processing time, and a video frame transmission time.   
     
     
         3 . The method of  claim 1 , wherein initiating the remedial action comprises:
 sending a notification to the remote support server indicating that the vehicle is exceeding the speed limit.   
     
     
         4 . The method of  claim 1 , wherein initiating the remedial action comprises:
 determining, based on the vehicle motion state and the estimated locations of the one or more obstacles, a range of vehicle kinematic parameters that can be applied to avoid a collision with the one or more obstacles; and   providing the range to the remote support server to enable the remote support server to generate a representation of the range of vehicle kinematic parameters on a display of a workstation operated by the remote teleoperator.   
     
     
         5 . The method of  claim 1 , wherein initiating the remedial action comprises:
 automatically limiting the speed of the vehicle to the speed limit.   
     
     
         6 . The method of  claim 1 , wherein determining the speed limit is further based on an estimated deceleration time of the vehicle. 
     
     
         7 . The method of  claim 6 , wherein the estimated deceleration time is computed at least in part based on a predefined range of steering angles for different possible trajectories of the vehicle. 
     
     
         8 . A non-transitory computer-readable storage medium storing instructions for operating a vehicle, the instructions when executed by a processor causing the processor to perform steps including:
 establishing a connection between the vehicle and a remote support server for enabling a teleoperation session with a remote teleoperator;   during the teleoperation session, receiving sensor data from a sensor array of the vehicle indicative of a vehicle motion state and estimated locations of one or more obstacles in an environment of the vehicle, the sensor data including a video feed associated with the vehicle;   during the teleoperation session, obtaining control signals for controlling the vehicle from the remote support server that are generated by the remote teleoperator based on the video feed;   obtaining network latency information indicative of network latency introduced under detected or predicted network conditions that affect a time for the sensor data including the video feed to be transmitted to the remote support server and for the control signals to be transmitted from the remote support server to the vehicle during the teleoperation session;   modeling a teleoperator reaction time for reacting to the video feed based at least in part on configured video processing parameters of the video feed;   determining, based on the vehicle motion state, the network latency information, the modeled teleoperator reaction time, and the estimated locations of the one or more obstacles, a speed limit for limiting speed of the vehicle during the teleoperation session that enables safe teleoperation of the vehicle;   determining if the vehicle is operating above the speed limit; and   initiating a remedial action responsive to the vehicle operating above the speed limit.   
     
     
         9 . The non-transitory computer-readable storage medium of  claim 8 , wherein modeling the teleoperator reaction time comprises:
 modeling at least one of a camera exposure time, a video frame onboard processing time, and a video frame transmission time.   
     
     
         10 . The non-transitory computer-readable storage medium of  claim 8 , wherein initiating the remedial action comprises:
 sending a notification to the remote support server indicating that the vehicle is exceeding the speed limit.   
     
     
         11 . The non-transitory computer-readable storage medium of  claim 8 , wherein initiating the remedial action comprises:
 determining, based on the vehicle motion state and the estimated locations of the one or more obstacles, a range of vehicle kinematic parameters that can be applied to avoid a collision with the one or more obstacles; and   providing the range to the remote support server to enable the remote support server to generate a representation of the range of vehicle kinematic parameters on a display of a workstation operated by the remote teleoperator.   
     
     
         12 . The non-transitory computer-readable storage medium of  claim 8 , wherein initiating the remedial action comprises:
 automatically limiting the speed of the vehicle to the speed limit.   
     
     
         13 . The non-transitory computer-readable storage medium of  claim 8 , wherein determining the speed limit is further based on an estimated deceleration time of the vehicle. 
     
     
         14 . The non-transitory computer-readable storage medium of  claim 13 , wherein the estimated deceleration time is computed at least in part based on a predefined range of steering angles for different possible trajectories of the vehicle. 
     
     
         15 . A vehicle computing system comprising:
 one or more processors; and   a non-transitory computer-readable storage medium storing instructions for operating a vehicle, the instructions when executed by the one or more processors causing the one or more processors to perform steps including:
 establishing a connection between the vehicle and a remote support server for enabling a teleoperation session with a remote teleoperator; 
 during the teleoperation session, receiving sensor data from a sensor array of the vehicle indicative of a vehicle motion state and estimated locations of one or more obstacles in an environment of the vehicle, the sensor data including a video feed associated with the vehicle; 
 during the teleoperation session, obtaining control signals for controlling the vehicle from the remote support server that are generated by the remote teleoperator based on the video feed; 
 obtaining network latency information indicative of network latency introduced under detected or predicted network conditions that affect a time for the sensor data including the video feed to be transmitted to the remote support server and for the control signals to be transmitted from the remote support server to the vehicle during the teleoperation session; 
 modeling a teleoperator reaction time for reacting to the video feed based at least in part on configured video processing parameters of the video feed; 
 determining, based on the vehicle motion state, the network latency information, the modeled teleoperator reaction time, and the estimated locations of the one or more obstacles, a speed limit for limiting speed of the vehicle during the teleoperation session that enables safe teleoperation of the vehicle; 
 determining if the vehicle is operating above the speed limit; and 
 initiating a remedial action responsive to the vehicle operating above the speed limit. 
   
     
     
         16 . The vehicle computing system of  claim 15 , wherein modeling the teleoperator reaction time comprises:
 modeling at least one of a camera exposure time, a video frame onboard processing time, and a video frame transmission time.   
     
     
         17 . The vehicle computing system of  claim 15 , wherein initiating the remedial action comprises:
 sending a notification to the remote support server indicating that the vehicle is exceeding the speed limit.   
     
     
         18 . The vehicle computing system of  claim 15 , wherein initiating the remedial action comprises:
 determining, based on the vehicle motion state and the estimated locations of the one or more obstacles, a range of vehicle kinematic parameters that can be applied to avoid a collision with the one or more obstacles; and   providing the range to the remote support server to enable the remote support server to generate a representation of the range of vehicle kinematic parameters on a display of a workstation operated by the remote teleoperator.   
     
     
         19 . The vehicle computing system of  claim 15 , wherein initiating the remedial action comprises:
 automatically limiting the speed of the vehicle to the speed limit.   
     
     
         20 . The vehicle computing system of  claim 15 , wherein determining the speed limit is further based on an estimated deceleration time of the vehicle.