Vehicle dispatching method and system

Abstract

Vehicle dispatch system includes upper stage unit, lower stage unit and interface communication unit. The upper stage unit, configured to generate vehicle schedules, is communicatively connected to the interface communication unit. The lower stage unit, communicatively connected to the upper stage unit and the interface communication unit, has two storage units and a control unit. The first storage unit stores in a state representation multiple possible states having multiple possible actions. The control unit, which receives the schedule as a state representation, is configured to simulate states during an episode by selecting a state action and determining a reward value. The second storage unit stores the reward value and has a policy linked to one possible action for each state. The interface communication unit, operable to receive and transmit vehicle communications, is configured to access the policy and its associated action and communicate the action to a vehicle.

What is claimed is: 
     
         1 . A vehicle dispatch system comprising:
 an upper stage unit communicatively connected to the interface communication unit and configured to generate a schedule for vehicles;   a lower stage unit communicatively connected to the upper stage unit and the interface communication unit, and having a first storage unit for storing a plurality of possible states in a state representation, each possible state having a plurality of possible actions, a control unit that receives the schedule as a state representation, the control unit being configured to simulate the states during an episode by selecting an action of the state and determining a reward value for the selected action, and a second storage unit, the second storage unit storing the reward value for each action and having a policy linked to one of the possible actions for each state; and   an interface communication unit operable to receive and transmit communications from the vehicles, the interface communication unit being configured to access the policy and the action linked to the policy and communicate the action to one or more of the vehicles.   
     
     
         2 . The vehicle dispatch system of  claim 1 , wherein the control unit is configured to simulate the states continuously to maximize the reward value of the actions in each state. 
     
     
         3 . The vehicle dispatch system of  claim 1 , wherein the policy is linked to a maximum policy value for an action for each state and the maximum policy value is determined using the elapsed time in the episode. 
     
     
         4 . The vehicle dispatch system of  claim 3 , wherein the maximum policy value is further determined using a subsequent state with a subsequent action in the episode. 
     
     
         5 . The vehicle dispatch system of  claim 1 , wherein the action within each state that results in the maximum possible policy value becomes the policy for that state. 
     
     
         6 . The vehicle dispatch system of  claim 1 , wherein the upper stage unit is configured with linear programming that has an optimization function, an environmental constraint and a vehicle constraint as inputs to generate the schedule. 
     
     
         7 . The vehicle dispatch system of  claim 1 , wherein the policy and reward value are stored in a Q table. 
     
     
         8 . The vehicle dispatch system of  claim 1 , wherein the control unit is configured to continuously simulate the states during the episode and determining the reward value for the selected action.