Systems and methods for optimizing an online on-demand service

Abstract

Systems and methods for optimizing an online on-demand service are provided. A method may include: obtaining driver information associated with a plurality of historical drivers corresponding to the plurality of historical orders; for each historical driver during a predetermined period of time, determining a plurality of records based on the order information and the driver information according to a decision-making processes, each record includes a driver's space-time status, a driver's action, a driver's revenue, and a driver's subsequent space-time status; and determining a value function based on the plurality of records of each historical driver according to a reinforcement learning algorithm.

We claim: 
     
       1. A system for determining and optimizing an allocation strategy for an online on-demand transportation service, comprising:
 at least one storage medium including a set of instructions for determining a value function in an online on-demand transportation service; and 
 at least one processor in communication with the storage medium, wherein when executing the set of instructions, the at least one processor is directed to:
 store, into the at least one storage medium, a plurality of historical orders collected from a plurality of passenger terminals and a plurality of driver terminals; 
 obtain order information associated with the plurality of historical orders by accessing the at least one storage medium; 
 obtain driver information associated with a plurality of historical drivers corresponding to the plurality of historical orders by accessing the at least one storage medium; 
 for each historical driver during a predetermined period of time, determine a plurality of records based on the order information and the driver information according to a decision-making processes model, wherein each record includes a driver's space-time status, a driver's action, a driver's revenue, and a driver's subsequent space-time status, and the decision-making processes model is a Markov Decision Process (MDP) model, wherein the order information and the driver information are inputs of the MDP model, and the outputs of the MDP model are the plurality of records; 
 determine a value function based on the plurality of records of each historical driver according to a reinforcement learning model, wherein the plurality of records of each historical driver are inputs of the reinforcement learning model, and the output of the reinforcement learning model is the value function; and 
 utilize the value function to determine an allocation strategy for the online on-demand transportation service. 
 
 
     
     
       2. The system of  claim 1 , wherein the reinforcement learning model is a temporal-difference learning model or a dynamic programming model. 
     
     
       3. The system of  claim 1 , wherein for each historical driver during the predetermined period of time, to determine the plurality of records, the at least one processor is further directed to:
 determine one or more order records, each corresponding to a historical order; and 
 determine one or more spare-time records, each corresponding to a period of idle time not associated with any historical order. 
 
     
     
       4. The system of  claim 1 , wherein for each historical driver during the predetermined period of time, to determine the plurality of records, the at least one processor is further directed to:
 determine one or more order records, each corresponding to a historical order; 
 obtain an online time point and an offline time point of the historical driver in the predetermined period of time based on the driver information; 
 determine at least one of spare-time records that corresponds to a period of idle time between the online time point and a time point for accepting a first historical order; and 
 determine at least another one of spare-time records that corresponds to a period of idle time between a time point for finishing a last historical order and the offline time point. 
 
     
     
       5. The system of  claim 1 , wherein the plurality of records include at least one order record and at least one spare-time record,
 wherein an order record corresponds to a historical order and includes at least one of:
 a driver's space-time status including a time and a location of the historical driver when accepting the historical order, 
 a driver's action including accepting the historical order, 
 a driver's revenue including a value of the historical order, or 
 a driver's subsequent space-time status including a time and a location of the historical driver when finishing the historical order; and 
 
 wherein a spare-time record corresponds to a period of idle time not associated with any historical order and includes at least one of:
 a driver's space-time status including a time and a location of the historical driver during the period of idle time, 
 a driver's action including being idle during the period of idle time, 
 a driver's revenue including zero, or 
 a driver's subsequent space-time status including a subsequent time and a subsequent location of the historical driver at the end of the period of idle time, wherein the period of idle time is a predetermined duration in the idle time. 
 
 
     
     
       6. The system of  claim 5 , wherein for each historical driver during the predetermined period of time, to determine the plurality of records, the at least one processor is further directed to:
 identify an order type of a subsequent order based on the order information; 
 in response to identifying that the subsequent order is a reserving order, determine a reference time point before a preset buffer time from a start time point of the reserving order; and 
 determine at least one spare-time record that corresponds to a period of idle time between a time point for finishing the historical order and the reference time point. 
 
     
     
       7. The system of  claim 5 , wherein for each historical driver during the predetermined period of time, to determine the plurality of records, the at least one processor is further directed to:
 identify a service type of the historical driver; 
 identify an order type of a subsequent order based on the order information; 
 in response to identifying that the subsequent order is an upgrade order for the historical driver, identify a start time point for the historical driver to receive a normal order associated with the same service type; and 
 determine at least one spare-time record that corresponds to a period of idle time between finishing the historical order and the start time point. 
 
     
     
       8. The system of  claim 5 , wherein for each historical driver during the predetermined period of time, to determine the plurality of records, the at least one processor is further directed to:
 determine the order record that corresponds to the historical order, wherein
 the historical order is a general carpooling order that combines a plurality of carpooling orders having a same route ID, 
 the time and the location of the historical driver are respectively a time and a location of the historical driver when first accepting the general carpooling order, and 
 the value of the historical order is a sum of values of the carpooling orders. 
 
 
     
     
       9. The system of  claim 1 , wherein when executing the set of instructions, the at least one processor is further directed to:
 optimize allocations of the drivers to the incoming orders based on the value function. 
 
     
     
       10. A method for determining and optimizing an allocation strategy for an online on-demand transportation service implemented on a computing device having at least one processor, at least one storage medium, and a communication platform connected to a network, comprising:
 storing, into the at least one storage medium, a plurality of historical orders collected from a plurality of passenger terminals and a plurality of driver terminals; 
 obtain order information associated with the plurality of historical orders by accessing the at least one storage medium; 
 obtaining driver information associated with a plurality of historical drivers corresponding to the plurality of historical orders by accessing the at least one storage medium; 
 for each historical driver during a predetermined period of time, determining a plurality of records based on the order information and the driver information according to a decision-making processes model, wherein each record includes a driver's space-time status, a driver's action, a driver's revenue, and a driver's subsequent space-time status, and the decision-making processes model is a Markov Decision Process (MDP) model, wherein the order information and the driver information are inputs of the MDP model, and the outputs of the MDP model are the plurality of records; 
 determining a value function based on the plurality of records of each historical driver according to a reinforcement learning model, wherein the plurality of records of each historical driver are inputs of the reinforcement learning model, and the output of the reinforcement learning model is the value function; and 
 utilizing the value function to determine an allocation strategy for the online on-demand transportation service. 
 
     
     
       11. The method of  claim 10 , wherein the reinforcement learning model is a temporal-difference learning model or a dynamic programming model. 
     
     
       12. The method of  claim 10 , wherein for each historical driver during the predetermined period of time, the determining the plurality of records includes:
 determining one or more order records, each corresponding to a historical order; and 
 determining one or more spare-time records, each corresponding to a period of idle time not associated with any historical order. 
 
     
     
       13. The method of  claim 10 , wherein for each historical driver during the predetermined period of time, the determining the plurality of records includes:
 determining one or more order records, each corresponding to a historical order; 
 obtaining an online time point and an offline time point of the historical driver in the predetermined period of time based on the driver information; 
 determining at least one of spare-time records that corresponds to a period of idle time between the online time point and a time point for accepting a first historical order; and 
 determining at least another one of spare-time records that corresponds to a period of idle time between a time point for finishing a last historical order and the offline time point. 
 
     
     
       14. The method of  claim 10 , wherein the plurality of records include at least one order record and at least one spare-time record,
 wherein an order record corresponds to a historical order and includes at least one of:
 a driver's space-time status including a time and a location of the historical driver when accepting the historical order, 
 a driver's action including accepting the historical order, 
 a driver's revenue including a value of the historical order, and 
 a driver's subsequent space-time status including a time and a location of the historical driver when finishing the historical order; or 
 
 wherein a spare-time record corresponds to a period of idle time not associated with any historical order and includes at least one of:
 a driver's space-time status including a time and a location of the historical driver during the period of idle time, 
 a driver's action including being idle during the period of idle time, 
 a driver's revenue including zero, or 
 a driver's subsequent space-time status including a subsequent time and a subsequent location of the historical driver at the end of the period of idle time, wherein the period of idle time is a predetermined duration in the idle time. 
 
 
     
     
       15. The method of  claim 14 , wherein for each historical driver during the predetermined period of time, the determining the plurality of records includes:
 identifying an order type of a subsequent order based on the order information; 
 in response to identifying that the subsequent order is a reserving order, determining a reference time point before a preset buffer time from a start time point of the reserving order; and 
 determining at least one spare-time record that corresponds to a period of idle time between a time point for finishing the historical order and the reference time point. 
 
     
     
       16. The method of  claim 14 , wherein for each historical driver during the predetermined period of time, the determining the plurality of records includes:
 identifying a service type of the historical driver; 
 identifying an order type of a subsequent order based on the order information; 
 in response to identifying that the subsequent order is an upgrade order for the historical driver, identifying a start time point for the historical driver to receive a normal order associated with the same service type; and 
 determining at least one spare-time record that corresponds to a period of idle time between finishing the historical order and the start time point. 
 
     
     
       17. The method of  claim 14 , wherein for each historical driver during the predetermined period of time, the determining the plurality of records includes:
 determining the order record that corresponds to the historical order, wherein
 the historical order is a general carpooling order that combines a plurality of carpooling order having a same route ID, 
 the time and the location of the historical driver are respectively a time and a location of the historical drive when first accepting the general carpooling order, and 
 the value of the historical order is a sum of values of the carpooling orders. 
 
 
     
     
       18. A non-transitory computer readable medium, comprising at least one set of instructions for determining and optimizing an allocation strategy for an online on-demand transportation service, wherein when executing the set of instructions, the at least one set of instructions directs the at least one processor to:
 store, into at least one storage medium, a plurality of historical orders collected from a plurality of passenger terminals and a plurality of driver terminals; 
 obtain order information associated with the plurality of historical orders by accessing the at least one storage medium; 
 obtain driver information associated with a plurality of historical drivers corresponding to the plurality of historical orders by accessing the at least one storage medium; 
 for each historical driver during a predetermined period of time, determine a plurality of records based on the order information and the driver information according to a decision-making processes model, wherein each record includes a driver's space-time status, a driver's action, a driver's revenue, and a driver's subsequent space-time status, and the decision-making processes model is a Markov Decision Process (MDP) model, wherein the order information and the driver information are inputs of the MDP model, and the outputs of the MDP model are the plurality of records; 
 determine a value function based on the plurality of records of each historical driver according to a reinforcement learning model, wherein the plurality of records of each historical driver are inputs of the reinforcement learning model, and the output of the reinforcement learning model is the value function; and 
 utilize the value function to determine an allocation strategy for the online on-demand transportation service.