Real-time visualization of autonomous vehicle behavior in mobile applications

Abstract

Technologies for providing real-time visualizations of a behavior of an autonomous vehicle (AV) associated with a ride request. In some examples, a method for providing real-time visualizations of a behavior of an AV associated with a ride request can include receiving a user request for a ride from an AV, wherein the user request specifies a pick-up location associated with a user; receiving sensor data from one or more sensors associated with the AV; determining, based on the sensor data, a state and context of the AV while the AV is en route to the pick-up location; and presenting, at a display interface, a map depicting one or more visual indicators of the state and context of the AV, the state and context of the AV including a location of the AV and one or more AV operations.

What is claimed is: 
     
         1 . A method comprising:
 receiving a user request from a user device for a ride from an autonomous vehicle (AV), wherein the user request specifies a pick-up location associated with a user;   receiving a video feed captured by one or more camera sensors on the AV, the video feed depicting a local scene associated with the AV;   identifying a parking location for the user to enter the AV; and   providing, to the user device, the video feed and a message identifying the parking location for the user to enter the AV within the pick-up location, wherein the user device concurrently displays the video feed and the message.   
     
     
         2 . The method of  claim 1 , wherein one or more AV operations comprise at least one of driving, reversing, parking, braking, turning, activating a turn signal, activating a brake light, changing a gear, changing a velocity of the AV, and performing a u-turn. 
     
     
         3 . The method of  claim 1 , the method further comprising:
 presenting, at a display interface, a map depicting one or more visual indicators of a state and context of the AV,   wherein the map depicts the AV performing one or more AV operations, and wherein one or more visual indicators identify at least one of a first status of a turn signal on the AV, a second status of a brake light on the AV, a rotation of a steering wheel on the AV, a current gear active at the AV, a gear change at the AV, and a u-turn operation.   
     
     
         4 . The method of  claim 3 , wherein the state and context of the AV further comprises at least one of a temperature in an interior of the AV, an AV battery status, a velocity of the AV, a behavior of the AV, a path of the AV to the pick-up location, an environment associated with the AV, and one or more conditions in the environment associated with the AV, wherein the one or more conditions in the environment associated with the AV comprise at least one of traffic, a road closure, a traffic accident, a hazard, a traffic signal, an event in the environment associated with the AV, and an obstacle along the path of the AV. 
     
     
         5 . The method of  claim 4 , wherein the map depicts one or more aspects of the environment associated with the AV, the method further comprising:
 presenting, at a display interface of the user device, an information section providing at least one of a description of one or more aspects of a state and context of the AV, trip information, one or more notifications, and a status of one or more AV controls.   
     
     
         6 . The method of  claim 3 , wherein the one or more visual indicators identify at least one of a temperature in an interior of the AV, an AV battery status, a velocity of the AV, a behavior of the AV, a path of the AV to the pick-up location, one or more conditions in an environment associated with the AV, and a status of the one or more conditions in the environment associated with the AV. 
     
     
         7 . The method of  claim 1 , wherein an obstacle along a path of the AV comprises at least one of a stopped vehicle, an animal, a pot hole, a person, emergency personnel, emergency equipment, a construction zone, a street fair, and flooding, wherein one or more visual indicators provide a visual representation of the obstacle along the path of the AV and one or more circumstances associated with an event in an environment. 
     
     
         8 . A system comprising:
 one or more sensors of an autonomous vehicle; and   a processor for executing instructions stored in memory, wherein execution of the instructions by the processor executes:
 receiving a user request from a user device for a ride from an autonomous vehicle (AV), wherein the user request specifies a pick-up location associated with a user; 
 receiving a video feed captured by one or more camera sensors on the AV, 
   the video feed depicting a local scene associated with the AV;
 identifying a parking location for the user to enter the AV; and 
 providing, to the user device, the video feed and a message identifying the parking location for the user to enter the AV within the pick-up location, wherein the user device concurrently displays the video feed and the message. 
   
     
     
         9 . The system of  claim 8 , wherein one or more AV operations comprise at least one of driving, reversing, parking, braking, turning, activating a turn signal, activating a brake light, changing a gear, changing a velocity of the AV, and performing a u-turn. 
     
     
         10 . The system of  claim 8 , wherein execution of the instructions by the processor further executes:
 presenting, at a display interface, a map depicting one or more visual indicators of a state and context of the AV,   wherein the map depicts the AV performing one or more AV operations, and wherein one or more visual indicators identify at least one of a first status of a turn signal on the AV, a second status of a brake light on the AV, a rotation of a steering wheel on the AV, a current gear active at the AV, a gear change at the AV, and a u-turn operation.   
     
     
         11 . The system of  claim 10 , wherein the state and context of the AV further comprises at least one of a temperature in an interior of the AV, an AV battery status, a velocity of the AV, a behavior of the AV, a path of the AV to the pick-up location, an environment associated with the AV, and one or more conditions in the environment associated with the AV, wherein the one or more conditions in the environment associated with the AV comprise at least one of traffic, a road closure, a traffic accident, a hazard, a traffic signal, an event in the environment associated with the AV, and an obstacle along the path of the AV. 
     
     
         12 . The system of  claim 11 , wherein the map depicts one or more aspects of the environment associated with the AV, wherein execution of the instructions by the processor further executes:
 presenting, at a display interface of the user device, an information section providing at least one of a description of one or more aspects of a state and context of the AV, trip information, one or more notifications, and a status of one or more AV controls.   
     
     
         13 . The system of  claim 10 , wherein the one or more visual indicators identify at least one of a temperature in an interior of the AV, an AV battery status, a velocity of the AV, a behavior of the AV, a path of the AV to the pick-up location, one or more conditions in an environment associated with the AV, and a status of the one or more conditions in the environment associated with the AV. 
     
     
         14 . The system of  claim 8 , wherein an obstacle along a path of the AV comprises at least one of a stopped vehicle, an animal, a pot hole, a person, emergency personnel, emergency equipment, a construction zone, a street fair, and flooding, wherein one or more visual indicators provide a visual representation of the obstacle along the path of the AV and one or more circumstances associated with an event in an environment. 
     
     
         15 . A non-transitory computer readable medium comprising instructions, the instructions, when executed by a computing system, cause the computing system to:
 receive a user request from a user device for a ride from an autonomous vehicle (AV), wherein the user request specifies a pick-up location associated with a user;   receive a video feed captured by one or more camera sensors on the AV, the video feed depicting a local scene associated with the AV;   identify a parking location for the user to enter the AV; and   provide, to the user device, the video feed and a message identifying the parking location for the user to enter the AV within the pick-up location, wherein the user device concurrently displays the video feed and the message.   
     
     
         16 . The non-transitory computer readable medium of  claim 15 , wherein one or more AV operations comprise at least one of driving, reversing, parking, braking, turning, activating a turn signal, activating a brake light, changing a gear, changing a velocity of the AV, and performing a u-turn. 
     
     
         17 . The non-transitory computer readable medium of  claim 15 , the instructions further causing the computing system to:
 present, at a display interface, a map depicting one or more visual indicators of a state and context of the AV,   wherein the map depicts the AV performing one or more AV operations, and wherein one or more visual indicators identify at least one of a first status of a turn signal on the AV, a second status of a brake light on the AV, a rotation of a steering wheel on the AV, a current gear active at the AV, a gear change at the AV, and a u-turn operation.   
     
     
         18 . The non-transitory computer readable medium of  claim 17 , wherein the state and context of the AV further comprises at least one of a temperature in an interior of the AV, an AV battery status, a velocity of the AV, a behavior of the AV, a path of the AV to the pick-up location, an environment associated with the AV, and one or more conditions in the environment associated with the AV, wherein the one or more conditions in the environment associated with the AV comprise at least one of traffic, a road closure, a traffic accident, a hazard, a traffic signal, an event in the environment associated with the AV, and an obstacle along the path of the AV. 
     
     
         19 . The non-transitory computer readable medium of  claim 17 , wherein the one or more visual indicators identify at least one of a temperature in an interior of the AV, an AV battery status, a velocity of the AV, a behavior of the AV, a path of the AV to the pick-up location, one or more conditions in an environment associated with the AV, and a status of the one or more conditions in the environment associated with the AV. 
     
     
         20 . The non-transitory computer readable medium of  claim 15 , wherein an obstacle along a path of the AV comprises at least one of a stopped vehicle, an animal, a pot hole, a person, emergency personnel, emergency equipment, a construction zone, a street fair, and flooding, wherein one or more visual indicators provide a visual representation of the obstacle along the path of the AV and one or more circumstances associated with an event in an environment.