Systems and methods for robotic control under contact

Abstract

In variants, a method for robot control can include: receiving sensor data of a scene, modeling the physical objects within the scene, determining a set of potential grasp configurations for grasping a physical object within the scene, determining a reach behavior based on the potential grasp configuration, determining a trajectory for the reach behavior, and grasping the object using the trajectory.

We claim: 
     
         1 . A method comprising:
 receiving a set of sensor data;   with the sensor data, determining a set of physical objects in an environment;   determining a virtual model of a physical object of the set, wherein the virtual model comprises immutable properties of physical object;   determining a state estimate for the physical object;   based on the state estimate and virtual model, determining a set of potential grasp configurations for grasping the physical object;   based on the set of potential grasp configurations, determining a reach behavior associated with a collision-free path towards the physical object relative to a remainder of the set of physical objects in the environment; and   determining a trajectory for a robot based on the reach behavior.   
     
     
         2 . The method of  claim 1 , wherein the virtual model comprises an object mass and an object geometry. 
     
     
         3 . The method of  claim 2 , wherein the set of physical objects in the environment are determined with an object detector, wherein the object mass is estimated based on an object class associated with the object detector. 
     
     
         4 . The method of  claim 1 , wherein determining the trajectory comprises: fitting a set of splines to points of the collision-free path in a configuration space of the robot. 
     
     
         5 . The method of  claim 1 , further comprising: selecting the physical object as a grasp target from the set of physical objects based on a set of heuristics; and, based on the selection of the physical object as the grasp target, controlling the robot based on the trajectory. 
     
     
         6 . The method of  claim 5 , wherein the set of heuristics comprises: a grasp quality heuristic or a path length optimization. 
     
     
         7 . The method of  claim 1 , wherein the collision-free path is determined based on an environmental collision evaluation based on pre-computed parameter values for simulated robot behaviors. 
     
     
         8 . The method of  claim 7 , wherein the environmental collision evaluation is based on a lookup table. 
     
     
         9 . The method of  claim 7 , wherein the pre-computed parameter values are determined using reinforcement learning for discretized state and action spaces. 
     
     
         10 . The method of  claim 1 , wherein the virtual model comprises a rigid-body kinematic model for the physical object. 
     
     
         11 . The method of  claim 1 , further comprising: determining a set of robot behaviors based on the potential grasp configurations, the set of robot behaviors comprising at least one of: a pre-grasp behavior, a grasp behavior, a transport behavior, or a release behavior. 
     
     
         12 . The method of  claim 11 , wherein the reach behavior defines, based on the pre-grasp poses, how the robot should move toward the object in order to grasp it. 
     
     
         13 . The method of  claim 1 , wherein the reach behavior defines a set of motor forces and torques for joints of the robot. 
     
     
         14 . A method comprising:
 receiving a set of sensor data;   with the sensor data, identifying a physical object within a surrounding environment;   determining a virtual model and a state estimate for the physical object, the virtual model comprising an object mass and an object geometry;   based on the state estimate and virtual model, determining a set of potential grasp configurations for grasping the physical object; and   determining a pre-grasp trajectory for a robot to achieve at least one potential grasp configuration of the set, wherein determining the pre-grasp trajectory comprises enforcing a non-collision constraint between the robot and the surrounding environment.   
     
     
         15 . The method of  claim 14 , further comprising: identifying a plurality of physical objects within the surrounding environment based on the sensor data; and determining the surrounding environment for the physical object, wherein the surrounding environment is associated with a virtual model and a state estimate for each physical object of the plurality. 
     
     
         16 . The method of  claim 14 , wherein the object model comprises estimated values for a set of immutable object properties, wherein the object geometry and object mass are modeled as immutable object properties. 
     
     
         17 . The method of  claim 14 , wherein determining the pre-grasp trajectory comprises: fitting a set of splines to points of the collision-free path in a configuration space of the robot. 
     
     
         18 . The method of  claim 14 , wherein determining the pre-grasp trajectory comprises an environmental collision evaluation based on pre-computed parameter values for simulated robot behaviors. 
     
     
         19 . The method of  claim 18 , wherein the environmental collision evaluation is based on a lookup table. 
     
     
         20 . The method of  claim 18 , wherein the pre-computed parameter values are determined using reinforcement learning for discretized state and action spaces.