Probabilistic decision making system and methods of use

Abstract

Embodiments of this invention comprise modeling a team's state and the influence of training treatments, or actions, on that state to create a training policy. Both state and effects of actions are modeled as probabilistic using Partially Observable Markov Decision Process (POMDP) techniques. Utilizing this model and the resulting training policy with teams creates an effective decision aid for instructors to improve learning relative to a traditional scenario selection strategy.

What is claimed is: 
     
         1 . A computer based system for determining training treatments for a team, said system comprising:
 a memory to store at least one action comprising at least one training treatment;   a processor capable of executing machine instructions; and   the machine instructions configured to execute a POMDP model to create a training policy to determine the at least one training treatment to train a team on a topic.   
     
     
         2 . The computer based system of  claim 1  wherein the machine instructions configured to execute a POMDP model further comprises the POMDP model having at least one state, at least one transition function, at least one reward function, at least one observation and at least one observation function. 
     
     
         3 . The computer based system of  claim 2  wherein:
 the team comprises at least a first team member and a second team member; and 
 the at least one observation comprises a communication pattern between the first team member and the second team member. 
 
     
     
         4 . The computer based system of  claim 2  wherein:
 the team comprises at least a first team member and a second team member; and 
 the at least one state comprises a representation of an expertise state of the team with at least one team skill. 
 
     
     
         5 . The computer based system of  claim 4  wherein the at least one team skill is one selected from the group consisting of:
 a number of targets killed, 
 a type of targets killed, 
 a delay in information sharing, 
 a delay in target prosecution, and 
 a communication pattern between the first team member and the second team member. 
 
     
     
         6 . The computer based system of  claim 2  wherein:
 the at least one transition function comprises a representation of the probability of an expected changed expertise state of the team after training the team on the treatment; 
 the at least one reward function comprises a representation of at least one objective and at least one cost of training the team on the treatment; 
 the at least one observation comprises a representation of a measure; and 
 the at least one observation function comprises a representation of the probability of an expected observation of the team after training the team on the treatment. 
 
     
     
         7 . The computer based system of  claim 6  wherein:
 the representation of the expected changed expertise state of the team further comprises a probability of moving from the expertise state to the expected changed expertise state conditioned on the training treatment given to the team; 
 the representation of the objective further comprises at least one number, where each number represents a benefit of team attaining the expertise state given the training treatment; and 
 the representation of the expected observation of the team further comprises the probability of an observation given the expertise state of the team and the training treatment given to the team. 
 
     
     
         8 . The computer based system of  claim 3  wherein the POMDP model is further configured to create a training policy by linking each one of the at least one state to at least one of the at least one training treatment at a node and interconnecting each node to another node by one of the at least one observation. 
     
     
         9 . The computer based system of  claim 8  wherein the POMDP model is further configured to apply the training policy by obtaining the expertise state of the team, select the node having that expertise state and determine the linked training treatment at that node as the training treatment to train the team on the topic. 
     
     
         10 . The computer based system of  claim 9  wherein the POMDP model is further configured to:
 after applying the training policy to determine the training treatment, training the team on the training treatment and obtaining the observation for the team; 
 apply the training policy to select the interconnected node and the expected changed expertise state of the team based on the observation; and 
 determine a next training treatment to train the team. 
 
     
     
         11 . A computer based method for structuring training treatments for a team on a topic, said method comprising:
 defining at least one action comprising at least one training treatment;   utilizing a POMDP model to create a training policy to determine the at least one training treatment to train a team on a topic; and   the POMDP model having at least one state, at least one transition function, at least one reward function, at least one observation and at least one observation function.   
     
     
         12 . The computer based method of  claim 11  wherein:
 the team comprises at least a first team member and a second team member; and 
 the at least one observation comprises a communication pattern between the first team member and the second team member. 
 
     
     
         13 . The computer based method of  claim 12  wherein:
 the at least one state comprises a representation of an expertise state of the team; 
 the at least one transition function comprises a representation of the probability of an expected changed expertise state of the team after training the team on the treatment; 
 the at least one reward function comprises a representation of at least one objective and at least one cost of training the team on the treatment; 
 the at least one observation comprises a representation of a measure of the team; and 
 the at least one observation function comprises a representation of the probability of an expected observation of the team after training the team on the treatment. 
 
     
     
         14 . A program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to perform the method steps comprising:
 generating a decision making policy from a POMDP model;   the POMDP model comprising at least one state parameter, at least one observation parameter and at least one action parameter; and   the action parameter comprising training treatments.   
     
     
         15 . The program storage device of  claim 14  wherein the at least one state parameter comprises the state of expertise of a team and the at least one observation parameter comprises a measure of the expertise of the team. 
     
     
         16 . The program storage device of  claim 15  wherein the step of generating a decision making policy further comprises:
 defining the at least one state parameter, the at least one action parameter and the at least one observation parameter; 
 defining a plurality of functions comprising at least one transition function, at least one observation function and at least one utility function; and 
 generating the decision making policy based on said parameters and said functions. 
 
     
     
         17 . The program storage device of  claim 16  wherein the program of instructions executable by the machine to perform the method steps further comprises:
 determining a changed state of the team after applying an action parameter; 
 comparing the changed state of the team to a process threshold; 
 selecting the at least one action parameter from the decision making policy; 
 applying the at least one action parameter to the team; 
 determining a new changed state of the team; 
 comparing the new changed state of the team to the process threshold; and 
 repeating the steps of selecting the at least one action parameter, applying the at least one action parameter, determining a new changed team and comparing the new changed state until the process threshold is met.