Robotic system with dynamic pack adjustment mechanism and methods of operating same

Abstract

A system and method for operating a robotic system to place objects into containers that have support walls is disclosed. The robotic system may detect an unexpected condition associated with a container during or before a real-time operation. Accordingly, the robotic system may dynamically adjust an existing packing plan based on detecting the unexpected condition.

We claim: 
     
       1. A tangible, non-transitory computer-readable medium having processor instructions stored thereon that, when executed by one or more processors, cause the one or more processors to perform a method, the method comprising:
 obtaining a real-time image that represents a container located at a task location and designated to receive a set of objects according to an existing packing plan; 
 dynamically generating an actual container model based on dividing the real-time image according to discretization units; 
 deriving a candidate pack plan for adjusting placements of objects in the existing packing plan within the container based on the actual container model and/or for adjusting a location of the existing packing plan within the container based on the actual container model; 
 adjusting the existing packing plan according to the candidate pack plan; and 
 implementing the adjusted packing plan for placing the set of objects in the container using a robotic arm and an end-effector. 
 
     
     
       2. The tangible, non-transitory computer-readable medium of  claim 1 , wherein the stored method further comprises:
 detecting an unexpected condition based on analyzing an outline of the container and/or one or more walls of the container as depicted in the real-time image, wherein
 dynamically generating the actual container model, deriving the candidate pack plan, adjusting the existing packing plan, and implementing the adjusted packing plan are responses to detecting the unexpected condition. 
 
 
     
     
       3. The tangible, non-transitory computer-readable medium of  claim 1 , wherein the stored method further comprises:
 computing a packing unit based on the existing packing plan, wherein the packing unit represents one or more physical attributes of the set of objects; and 
 determining a fit status based on comparing the packing unit at the candidate pack location with the actual container, wherein the existing packing plan is adjusted according to the fit status. 
 
     
     
       4. The tangible, non-transitory computer-readable medium of  claim 3 , wherein determining the fit status includes:
 deriving an updated approach path representing a motion plan associated with placement of a reference object within the set of objects at an adjusted location that corresponds to shifting the initial packing plan to the location of the candidate pack plan; and 
 validating the location of the candidate pack plan based on successfully deriving the updated approach path. 
 
     
     
       5. The tangible, non-transitory computer-readable medium of  claim 4 , wherein the updated approach path is for placing a first object in a placement sequence for the packing plan. 
     
     
       6. The tangible, non-transitory computer-readable medium of  claim 4 , wherein the updated approach path represents the motion plan associated with placement of the reference object that forms a perimeter edge or a corner for the packing plan. 
     
     
       7. The tangible, non-transitory computer-readable medium of  claim 1 , wherein:
 the existing packing plan includes a set of initial placement locations for placing a corresponding set of objects within the container; and 
 adjusting the existing packing plan includes shifting the set of placement locations based on the location of the candidate pack plan. 
 
     
     
       8. The tangible, non-transitory computer-readable medium of  claim 7 , wherein adjusting the existing packing plan includes:
 deriving an updated approach path representing a motion plan associated with placement of a reference object within the set of objects at an adjusted location that corresponds to shifting the initial packing plan to the location of the candidate pack plan; and 
 adjusting existing motion plans for remaining objects within the set of objects based on a difference between (1) the updated approach path for the reference object and (2) a previous approach path for the reference object according to the existing packing plan. 
 
     
     
       9. The tangible, non-transitory computer-readable medium of  claim 1 , wherein the stored method further comprises:
 determining whether the existing packing plan includes a placement location for placing an object within a threshold distance from a vertical wall of the container; 
 determining an updated placement location based on the candidate position for placing the object; and 
 wherein: 
 adjusting the existing packing plan includes validating the updated placement location according to one or more rules configured to analyze support from a vertically-oriented structure. 
 
     
     
       10. The tangible, non-transitory computer-readable medium of  claim 1 , wherein adjusting the existing packing plan includes dynamically re-deriving a new packing plan based on:
 obtaining object models representative of the set of objects; 
 determining candidate positions for placing the set of objects; 
 overlaying the object models over the actual container model according to the candidate positions; and 
 validating the candidate positions based on one or more placement rules. 
 
     
     
       11. A robotic system comprising:
 a communication device configured to obtain
 a real-time image that represents a container located at a task location and designated to receive a set of objects according to an existing packing plan; and 
 
 at least one processor coupled to the communication device, the at least one processor for:
 dynamically generating an actual container model based on dividing the real-time image according to discretization units; 
 deriving a candidate pack plan for adjusting placements of objects in the existing packing plan within the container based on the actual container model and/or for adjusting a location of the existing packing plan within the container based on the actual container model; 
 adjusting the existing packing plan according to the candidate pack plan; and 
 implementing the adjusted packing plan for placing the set of objects in the container using a robotic arm and an end-effector. 
 
 
     
     
       12. The robotic system of  claim 11 , wherein the at least one processor is for:
 detecting an unexpected condition based on analyzing an outline of the container and/or one or more walls of the container as depicted in the real-time image, wherein
 dynamically generating the actual container model, deriving the candidate pack plan, adjusting the existing packing plan, and implementing the adjusted packing plan are responses to detecting the unexpected condition. 
 
 
     
     
       13. The robotic system of  claim 11 , wherein the at least one processor is for:
 computing a packing unit based on the existing packing plan, wherein the packing unit represents one or more physical attributes of the set of objects; and 
 determining a fit status based on comparing the packing unit at the candidate pack plan with the actual container, wherein the existing packing plan is adjusted according to the fit status. 
 
     
     
       14. The robotic system of  claim 11 , wherein:
 the existing packing plan includes a set of initial placement locations for placing a corresponding set of objects within the container; and 
 the existing packing plan is adjusted based on shifting the set of placement locations based on the location of the candidate pack plan. 
 
     
     
       15. The robotic system of  claim 11 , wherein the at least one processor is for adjusting the existing packing plan by dynamically re-deriving a new packing plan based on:
 obtaining object models representative of the set of objects; 
 determining candidate positions for placing the set of objects; 
 overlaying the object models over the actual container model according to the candidate positions; and 
 validating the candidate positions based on one or more placement rules. 
 
     
     
       16. A method of operating a robotic system, the method comprising:
 obtaining a real-time image that represents a container located at a task location and designated to receive a set of objects according to an existing packing plan; 
 dynamically generating an actual container model based on dividing the real-time image according to discretization units; 
 deriving a candidate pack plan for adjusting placements of objects in the existing packing plan within the container based on the actual container model and/or for adjusting a location of the existing packing plan within the container based on the actual container model; 
 adjusting the existing packing plan according to the candidate pack plan; and 
 implementing the adjusted packing plan for placing the set of objects in the container using a robotic arm and an end-effector. 
 
     
     
       17. The method of  claim 16 , further comprising:
 detecting an unexpected condition based on analyzing an outline of the container and/or one or more walls of the container as depicted in the real-time image, wherein
 dynamically generating the actual container model, deriving the candidate pack plan, adjusting the existing packing plan, and implementing the adjusted packing plan are responses to detecting the unexpected condition. 
 
 
     
     
       18. The method of  claim 16 , further comprising:
 computing a packing unit based on the existing packing plan, wherein the packing unit represents one or more physical attributes of the set of objects; and 
 determining a fit status based on comparing the packing unit at the candidate pack plan with the actual container, wherein the existing packing plan is adjusted according to the fit status. 
 
     
     
       19. The method of  claim 16 , wherein adjusting the existing packing plan includes:
 deriving an updated approach path representing a motion plan associated with placement of a reference object within the set of objects at an adjusted location that corresponds to shifting the packing plan to the candidate pack location; and 
 validating the candidate pack plan based on successfully deriving the updated approach path. 
 
     
     
       20. The method of  claim 16 , further comprising:
 determining whether the existing packing plan includes a placement location for placing an object within a threshold distance a vertical wall of the container; 
 wherein adjusting the existing packing plan includes:
 determining an updated placement location for the object; and 
 validating the updated placement location according to one or more rules configured to analyze support from a vertically-oriented structure.