System and method for controlling a robotic manipulator

Abstract

A controller for controlling robotic manipulator according to a task is provided. The controller is to collect data relating to a state and an object property of an object, and execute a state adapter model to produce a state correction to state of the object having the object property different from a unitary property of a unitary object. The controller is to execute a control policy using the state correction to produce an action for the unitary object, and execute an action adapter model to produce an action correction to the action produced by the control policy. The state correction and action correction are produced based on difference between object property and unitary property. The control policy is to map a state of the unitary object to the action of the robotic manipulator to manipulate the unitary object according to the task.

We claim: 
     
       1. A controller for controlling a robotic manipulator to manipulate an object according to a task, comprising:
 a memory configured to store
 a control policy configured to produce an action for the robotic manipulator to manipulate a unitary object having a predetermined unitary property, such that the control policy is configured to map a state of the unitary object to the action of the robotic manipulator to manipulate the unitary object according to the task, 
 a state adapter model configured to produce a state correction for a state of the object having an object property different from the unitary property, based on a difference between the object property and the unitary property, and 
 an action adapter model configured to produce an action correction to the action produced by the control policy based on the difference between the unitary property and the object property; 
 
 a processor configured to 
 collect data relating to the state and the object property of the object; 
 execute the state adapter model to produce the state correction for the state of the object to generate a corrected state of the object based on the collected data; 
 execute the control policy using the corrected state of the object to produce the action for the unitary object; 
 execute the action adapter model to produce the action correction to the action for the unitary object to generate a corrected action; and 
 control the robotic manipulator with the corrected action for the unitary object. 
 
     
     
       2. The controller of  claim 1 , wherein the action adapter model is configured to produce the action correction to the action for the unitary object based on the corrected state of the object and property encoding of the object property relating to the object. 
     
     
       3. The controller of  claim 2 , wherein the processor is further configured to:
 process, by executing an encoder, the object property of the object to produce the property encoding; 
 transmit the property encoding to the state adapter model and the action adapter model; and 
 produce, by executing the control policy, the state adapter model and the action adapter model, the corrected action based on the property encoding. 
 
     
     
       4. The controller of  claim 1 , wherein the object property is associated with at least one of: shape, size, density, weight, or material. 
     
     
       5. The controller of  claim 4 , wherein when the object property and the unitary property relates to the shape of the object and the unitary object, respectively, the processor is further configured to:
 produce, using the state adapter model, the state correction for the state of the object having an object shape different from a unitary shape, based on a difference between the object shape and the unitary shape; and 
 produce, using the action adapter model, the action correction to the action produced by the control policy based on the difference between the unitary shape and the object shape. 
 
     
     
       6. The controller of  claim 1 , wherein to train the control policy, the processor is further configured to:
 generate a dataset comprising a set of random objects, the set of random objects having corresponding random shapes different from a unitary shape of the unitary object; 
 learn the feature space for the random objects based on a difference between each of the random object shapes and the unitary shape; and 
 determine a predicted object shape for each of the set of random objects based on the learned feature space. 
 
     
     
       7. The controller of  claim 1 , wherein the object property is captured using one or more sensors. 
     
     
       8. The controller of  claim 1 , wherein the corrected action comprises action parameters for controlling an interaction between the robotic manipulator and the object. 
     
     
       9. The controller of  claim 1 , wherein the control policy comprises a neural network trained with reinforcement learning. 
     
     
       10. The controller of  claim 1 , wherein each of the state adapter model and the action adapter model comprises a neural network, wherein the state adapter model and the action adapter model are trained with machine learning. 
     
     
       11. The controller of  claim 1 , wherein, to train the controller, the processor is further configured to:
 cause the state adapter model to learn one or more linear transformations to produce a state correction to the state of the unitary object for a synthetic object; 
 cause the action adapter model to learn one or more linear transformations to produce an action correction to an action for the unitary object based on the corrected state; and 
 adapt the control policy for the synthetic object based on the learning. 
 
     
     
       12. The controller of  claim 1 , wherein the processor is further configured to:
 generate a simulation environment of the task based on the unitary property; and 
 produce the action correction based on the simulation environment. 
 
     
     
       13. A method for controlling a robotic manipulator to manipulate an object according to a task, the method comprising:
 collecting data relating to a state and an object property of the object; 
 producing, using a state adapter model, a state correction for the state of the object to generate a corrected state of the object based on a difference between the object property and a predetermined unitary property of a unitary object; 
 producing, using a control policy, an action for the unitary object using the corrected state of the object, wherein the action is for a robotic manipulator to manipulate the unitary object, such that the control policy is configured to map a state of the unitary object to the action of the robotic manipulator to manipulate the unitary object according to the task; 
 producing, using an action adapter model, an action correction to the action for the unitary object to generate a corrected action, based on the action for the unitary object produced by the control policy and the difference between the unitary property and the object property; and 
 controlling the robotic manipulator with the corrected action for the unitary object. 
 
     
     
       14. The method of  claim 13 , further comprising:
 processing, by executing an encoder, the object property of the object to produce a property encoding; 
 transmitting the property encoding to the state adapter model and the action adapter model; and 
 producing, by executing the state adapter model and the action adapter model, the corrected action based on the property encoding. 
 
     
     
       15. The method of  claim 13 , wherein the object property is associated with at least one of: shape, size, density, weight, or material. 
     
     
       16. The method of  claim 15 , wherein when the object property and the unitary property relates to the shape of the object and the unitary object, respectively, the method further comprises:
 producing, using the state adapter model, the state correction to the state of the object having an object shape different from a unitary shape based on a difference between the object shape and the unitary shape; and 
 producing, using the action adapter model, the action correction to the action produced by the control policy based on the difference between the unitary shape and the object shape. 
 
     
     
       17. The method of  claim 13 , wherein to train the control policy, the method further comprises:
 generating a dataset comprising a set of random objects, the set of random objects having corresponding random shapes different from a unitary shape of the unitary object; 
 learning the feature space for the random objects based on a difference between each of the random object shapes and the unitary shape; and 
 determining a predicted object shape for each of the set of random objects based on the learned feature space. 
 
     
     
       18. The method of  claim 13 , wherein the control policy comprises a neural network trained with reinforcement learning. 
     
     
       19. The method of  claim 13 , wherein each of the state adapter model and the action adapter model comprises a control policy, wherein the state adapter model and the action adapter model are trained with machine learning. 
     
     
       20. A non-transitory computer readable storage medium embodied thereon a program executable by a processor for performing a method, the method comprising:
 collecting data relating to a state and an object property of an object; 
 producing, using a state adapter model, a state correction for the state of the object to generate a corrected state of the object, based on a difference between the object property and a predetermined unitary property of a unitary object; 
 producing, using a control policy, an action for the unitary object using the corrected state of the object, wherein the action is for a robotic manipulator to manipulate the unitary object, such that the control policy is configured to map a state of the unitary object to the action of the robotic manipulator to manipulate the unitary object according to the task; 
 producing, using an action adapter model, an action correction to the action for the unitary object to generate a corrected action based on the action for the unitary object produced by the control policy and the difference between the unitary property and the object property; and 
 controlling the robotic manipulator with the corrected action for the unitary object.