System and method for flexible human-machine collaboration

Abstract

Methods and systems for enabling human-machine collaborations include a generalizable framework that supports dynamic adaptation and reuse of robotic capability representations and human-machine collaborative behaviors. Specifically, a method of feedback-enabled user-robot collaboration includes obtaining a robot capability that models a robot's functionality for performing task actions, specializing the robot capability with an information kernel that encapsulates task-related parameters associated with the task actions, and providing an instance of the specialized robot capability as a robot capability element that controls the robot's functionality based on the task-related parameters. The method also includes obtaining, based on the robot capability element's user interaction requirements, user interaction capability elements, via which the robot capability element receives user input and provides user feedback, controlling, based on the task-related parameters, the robot's functionality to perform the task actions in collaboration with the user input; and providing the user feedback including task-related information generated by the robot capability element in association with the task actions.

What is claimed is: 
     
       1. A computer-implemented method of feedback-enabled user-robot collaboration, comprising:
 obtaining a robot capability that models at least one functionality of a robot for performing one or more task actions; 
 specializing the robot capability with an information kernel to provide a specialized robot capability, wherein the information kernel encapsulates a set of task-related parameters associated with the one or more task actions; 
 providing an instance of the specialized robot capability as a robot capability element that controls the at least one functionality of the robot based on the set of task-related parameters; 
 obtaining one or more user interaction capability elements based on one or more user interaction requirements of the robot capability element, wherein the robot capability element receives user input and provides user feedback via the one or more user interaction capability elements; 
 controlling, based on the set of task-related parameters, the at least one functionality of the robot to perform the one or more task actions in collaboration with the user input, wherein the robot capability element generates task-related information associated with the one or more task actions; and 
 providing the user feedback including the task-related information. 
 
     
     
       2. The method of  claim 1 , further comprising:
 generating, via the one or more user interaction capability elements, one or more user interfaces (UIs); 
 receiving the user input via at least one input UI of the one or more UIs; and 
 providing the user feedback via at least one output UI of the one or more UIs. 
 
     
     
       3. The method of  claim 2 , wherein the user feedback includes at least one of robot motion information, a robot safety parameter, or a robot task completion status. 
     
     
       4. The method of  claim 2 , further comprising:
 identifying, based on the user input and the one or more task actions, an undefined parameter associated with the one or more task actions; and 
 providing, via the at least one output UI, information associated with the undefined parameter. 
 
     
     
       5. The method of  claim 2 , further comprising:
 determining that the user input fails to satisfy one or more parameterization requirements of the information kernel for specializing the robot capability; 
 indicating, via the at least one output UI, that the user input fails to satisfy the one or more parameterization requirements of the information kernel; 
 receiving additional user input via the at least one input UI; and 
 modifying the set of task-related parameters based on the additional user input. 
 
     
     
       6. The method of  claim 2 , further comprising:
 providing, via the at least one output UI subsequent to performing the one or more task actions, one or more task-related parameters in the set of task-related parameters that are modifiable to modify the one or more task actions. 
 
     
     
       7. The method of  claim 6 , further comprising:
 detecting a change in a robot state of the robot to a new robot state; 
 determining, in response to the change in the robot state, that the user input is valid for the new robot state; and 
 indicating, via the at least one output UI, that the user input is valid for the new state. 
 
     
     
       8. The method of  claim 6 , further comprising:
 detecting a change in a robot state of the robot to a new robot state; 
 determining, in response to the change in the robot state, that the user input is invalid for the new robot state; 
 indicating, via the at least one output UI, that the user input is invalid for the new robot state; 
 modifying, via the at least one input UI, the user input to provide modified user input; and 
 determining that the modified user input is valid for the new robot state prior to controlling the at least one functionality of the robot. 
 
     
     
       9. The method of  claim 6 , further comprising:
 unloading the robot capability element subsequent to performing the one or more task actions; 
 loading a new robot capability element in place of the robot capability element; 
 determining that the user input fails to satisfy one or more parameterization requirements of the new robot capability element; 
 indicating, via the at least one output UI, that the user input fails to satisfy the one or more parameterization requirements of the new robot capability element; and 
 receiving, via the at least one input UI, additional user input. 
 
     
     
       10. The method of  claim 6 , further comprising:
 unloading the robot capability element subsequent to performing the one or more task actions; 
 loading a new robot capability element in place of the robot capability element; 
 determining that the user input satisfies one or more parameterization requirements of the new robot capability element; 
 indicating, via the at least one output UI, that the user input satisfies the one or more parameterization requirements of the new robot capability element; and 
 providing, via the at least one output UI, information associated with a change in a robot state of the robot. 
 
     
     
       11. The method of  claim 1 , wherein the user feedback includes a set of robot capability elements available to perform the one or more task actions. 
     
     
       12. The method of  claim 1 , wherein providing the user feedback further comprises:
 providing the user feedback to a plurality of users. 
 
     
     
       13. The method of  claim 12 , wherein providing the user feedback further comprises:
 providing the user feedback to the plurality of users via one or more network connections. 
 
     
     
       14. The method of  claim 1 , wherein the user feedback includes visual feedback. 
     
     
       15. The method of  claim 1 , wherein the user feedback includes haptic feedback. 
     
     
       16. A system for feedback-enabled user-robot collaboration, comprising:
 a non-transitory memory storing instructions; and 
 a processor executing the instructions to cause the system to perform a method comprising:
 obtaining a robot capability that models at least one functionality of a robot for performing one or more task actions; 
 specializing the robot capability with an information kernel to provide a specialized robot capability, wherein the information kernel encapsulates a set of task-related parameters associated with the one or more task actions; 
 providing an instance of the specialized robot capability as a robot capability element that controls the at least one functionality of the robot based on the set of task-related parameters; 
 obtaining one or more user interaction capability elements based on one or more user interaction requirements of the robot capability element, wherein the robot capability element receives user input and provides user feedback via the one or more user interaction capability elements; 
 controlling, based on the set of task-related parameters, the at least one functionality of the robot to perform the one or more task actions in collaboration with the user input, wherein the robot capability element generates task-related information associated with the one or more task actions; and 
 providing the user feedback including the task-related information. 
 
 
     
     
       17. The system of  claim 16 , wherein the instructions cause the processor to further perform:
 generating, via the one or more user interaction capability elements, one or more user interfaces (UIs); 
 receiving the user input via at least one input UI of the one or more UIs; 
 providing the user feedback via at least one output UI of the one or more UIs; and 
 providing, via the at least one output UI subsequent to performing the one or more task actions, one or more task-related parameters in the set of task-related parameters that are modifiable to modify the one or more task actions. 
 
     
     
       18. The system of  claim 17 , wherein the instructions cause the processor to further perform:
 unloading the robot capability element subsequent to performing the one or more task actions; 
 loading a new robot capability element in place of the robot capability element; 
 determining that the user input fails to satisfy one or more parameterization requirements of the new robot capability element; 
 indicating, via the at least one output UI, that the user input fails to satisfy the one or more parameterization requirements of the new robot capability element; and 
 receiving, via the at least one input UI, additional user input. 
 
     
     
       19. The system of  claim 17 , wherein the instructions cause the processor to further perform:
 unloading the robot capability element subsequent to performing the one or more task actions; 
 loading a new robot capability element in place of the robot capability element; 
 determining that the user input satisfies one or more parameterization requirements of the new robot capability element; 
 indicating, via the at least one output UI, that the user input satisfies the one or more parameterization requirements of the new robot capability element; and 
 providing, via the at least one output UI, information associated with a change in a robot state of the robot. 
 
     
     
       20. A non-transitory computer-readable storage medium containing instructions which, when executed on a processor, perform a method comprising:
 obtaining a robot capability that models at least one functionality of a robot for performing one or more task actions; 
 specializing the robot capability with an information kernel to provide a specialized robot capability, wherein the information kernel encapsulates a set of task-related parameters associated with the one or more task actions; 
 providing an instance of the specialized robot capability as a robot capability element that controls the at least one functionality of the robot based on the set of task-related parameters; 
 obtaining one or more user interaction capability elements based on one or more user interaction requirements of the robot capability element, wherein the robot capability element receives user input and provides user feedback via the one or more user interaction capability elements; 
 controlling, based on the set of task-related parameters, the at least one functionality of the robot to perform the one or more task actions in collaboration with the user input, wherein the robot capability element generates task-related information associated with the one or more task actions; and 
 providing the user feedback including the task-related information.