Method and system for robot control using visual feedback

Abstract

A method for robot control using visual feedback including determining a generative model S 100 , training the generative model S 200 , and controlling the robot using the trained generative model S 300.

We claim: 
     
         1 . A method for system control, wherein the system comprises an end-effector connected to a robot, comprising:
 determining a generative model comprising a set of variables and a set of parameters, wherein the set of variables comprises observed variables and hidden variables, and wherein the set of parameters comprises camera parameters and structure parameters;   training the generative model, comprising:
 sampling a set of images, wherein each image depicts the robot, the end-effector, and a feature rigidly connected to the end effector; 
 determining feature pixel locations of the feature detected in the set of images; 
 initializing the camera parameters; 
 determining an action for the robot, wherein the action is a variable of the set of variables; 
 calculating optimized camera parameters, optimized structure parameters, and an end-effector pose based on the initialized camera parameters, the action, and the feature pixel locations, wherein the end-effector pose is a hidden variable of the set of variables; 
 and 
   controlling the robot using the trained generative model.