Systems, methods, and computer program products for populating environment models

Abstract

Systems, methods, and computer program products for managing and populating environment models are described. An environment model is accessed which represents an environment, and the environment model is populated with instances of object models. Locations where the instances of object models should be positioned in the environment model are identified, by determining where in the environment model a respective size of each instance when viewed from a vantage point at the environment model matches a size of the object represented by the respective instance when viewed from a corresponding vantage point at the environment.

The invention claimed is: 
     
         1 . A computer program product comprising at least one non-transitory processor-readable storage medium storing processor-executable instructions and/or data that, when executed by at least one processor of a processor-based system, cause the processor-based system to:
 access, by the at least one processor, an environment model representation of an environment;   access, by the at least one processor, a first view of the environment from a first vantage point, the first vantage point having a position and a perspective in relation to the environment, wherein the first view includes an object in the environment;   access, in a library of object models, an object model representation of the object, the object model including dimension data indicative of spatial dimensions of the object; and   populate the environment model with an instance of the object model at a location in the environment model, wherein the processor-executable instructions which cause the processor-based system to populate the environment model with the instance of the object model at the location cause the processor-based system to:   generate a second view of the environment model from a second vantage point, wherein a position and a perspective of the second vantage point in relation to the environment model substantially match the position and the perspective of the first vantage point in relation to the environment;   identify the location in the environment model where at least one spatial dimension of the instance of the object model in the second view of the environment model from the second vantage point substantially matches a corresponding spatial dimension of the object in the first view of the environment from the first vantage point; and   position the instance of the object model at the location, wherein:   the at least one processor is carried by a robot body positioned at the environment;   the at least one non-transitory processor-readable storage medium is carried by the robot body;   the at least one non-transitory processor-readable storage medium stores the library of object models; and   the at least one non-transitory processor-readable storage medium stores the environment model.   
     
     
         2 . The computer program product of  claim 1 , wherein:
 the first view comprises first image data having a first resolution;   the second view comprises second image data having a second resolution;   the processor-executable instructions which cause the processor-based system to identify the location in the environment model where at least one spatial dimension of the instance of the object model in the second view of the environment model from the second vantage point substantially matches a corresponding spatial dimension of the object in the first view of the environment from the first vantage point cause the processor-based system to:   identify the location in the environment model where a number of pixels occupied by the instance of the object model in the second image data corresponds to a number of pixels occupied by the object in the first image data.   
     
     
         3 . The computer program product of  claim 2 , wherein:
 the first resolution is equal to the second resolution; and   the processor-executable instructions which cause the processor-based system to identify the location in the environment model where a number of pixels occupied by the instance of the object model in the second image data corresponds to a number of pixels occupied by the object in the first image data cause the processor-based system to: identify the location in the environment model where a number of pixels occupied by the instance of the object model in the second image data is equal to a number of pixels occupied by the object in the first image data.   
     
     
         4 . The computer program product of  claim 2 , wherein:
 the first resolution is different from the second resolution by a fixed ratio;   the processor-executable instructions which cause the processor-based system to identify the location in the environment model where a number of pixels occupied by the instance of the object model in the second image data corresponds to a number of pixels occupied by the object in the first image data cause the processor-based system to: identify the location in the environment model where a number of pixels occupied by the instance of the object model in the second image data is equal to a number of pixels occupied by the object in the first image data multiplied by the fixed ratio.   
     
     
         5 . The computer program product of  claim 1 , wherein the processor-executable instructions further cause the processor-based system to generate the object model representing the object in the library of object models. 
     
     
         6 . The computer program product of  claim 5 , wherein the processor-executable instructions which cause the processor-based system to generate the object model representing the object cause the processor-based system to: generate the object model representing the object, including the dimension data indicative of the spatial dimensions of the object. 
     
     
         7 . The computer program product of  claim 5 , wherein:
 the processor-executable instructions further cause the processor-based system to capture, by at least one image sensor, image data representing the object from multiple viewpoints; and   the processor-executable instructions which cause the processor-based system to generate the object model representing the object in the library of object models cause the processor-based system to generate the object model based on the captured image data from multiple viewpoints.   
     
     
         8 . The computer program product of  claim 5  wherein:
 the processor-executable instructions further cause the processor-based system to capture, by at least one haptic sensor positioned at an actuatable member which contacts the object, haptic data representing the object; and 
 the processor-executable instructions which cause the processor-based system to generate the object model representing the object in the library of object models cause the processor-based system to generate the object model based on the captured haptic data. 
 
     
     
         9 . The computer program product of  claim 1 , wherein:
 the environment is a three-dimensional environment;   the environment model is a three-dimensional environment model;   the first view comprises first two-dimensional image data representing the environment from the first vantage point;   the second view comprises second two-dimensional image data representing the environment model from the second vantage point; and   the processor-executable instructions which cause the processor-based system to populate the environment model with the instance of the object model at the location further cause the processor-based system to, prior to identifying the location, position the instance of the object model in the second image data to correspond to a position of the object in the first image data.   
     
     
         10 . The computer program product of  claim 9 , wherein the processor-executable instructions which cause the processor-based system to populate the environment model with the instance of the object model at the location further cause the processor-based system to, prior to identifying the location, orient the instance of the object model in the second image data to correspond to an orientation of the object in the first image data. 
     
     
         11 . The computer program product of  claim 9 , wherein the processor-executable instructions further cause the processor-based system to determine a distance in the environment model between the second vantage point and the instance of the object model at the location. 
     
     
         12 . The computer program product of  claim 1 , wherein the environment is a physical environment, and the environment model is a representation of the physical environment. 
     
     
         13 . The computer program product of  claim 1 , wherein the environment is a virtual environment, and the environment model is a representation of the virtual environment. 
     
     
         14 . The computer program product of  claim 1 , wherein:
 the at least one non-transitory processor-readable storage medium includes a first at least one non-transitory processor-readable storage medium carried by the robot body and a second at least one non-transitory processor-readable storage medium positioned at a robot controller remote from the robot body, the robot controller operable to provide control data to the robot body;   the first at least one non-transitory processor-readable storage medium stores the environment model;   the second at least one non-transitory processor-readable storage medium stores the library of object models;   the processor-executable instructions which cause the processor-based system to access the environment model representation of the environment cause the processor-based system to: access, by the at least one processor, the environment model stored at the first at least one non-transitory processor-readable storage medium;   the processor-executable instructions which cause the system to access the first view of the environment cause the processor-based system to access, by the at least one processor, the first view of the environment stored at the first at least one non-transitory processor-readable storage medium;   the processor-executable instructions which cause the processor-based system to access, in the library of object models, the object model cause the processor-based system to: access the object model in the library of models stored at the second at least one non-transitory processor-readable storage medium, via a communication interface which communicatively couples the robot body and the robot controller;   the processor-executable instructions which cause the processor-based system to generate a second view of the environment model from the second vantage point cause the processor-based system to: generate, by the at least one processor, the second view of the environment model;   the processor-executable instructions which cause the processor-based system to identify the location in the environment model cause the processor-based system to: identify, by the at least one processor, the location in the environment model; and   the processor-executable instructions which cause the processor-based system to position the instance of the object model at the location cause the processor-based system to: update, by the at least one processor, the environment model stored at the first at least one non-transitory processor-readable storage medium to include the instance of the object model at the location.   
     
     
         15 . The computer program product of  claim 1 , wherein the processor-executable instructions further cause the processor-based system to capture, by at least one image sensor, image data representing the first view of the environment from the first vantage point. 
     
     
         16 . A computer program product comprising at least one non-transitory processor-readable storage medium storing processor-executable instructions and/or data that, when executed by at least one processor of a processor-based system, cause the processor-based system to:
 access, by the at least one processor, an environment model representation of an environment;   access, by the at least one processor, a first view of the environment from a first vantage point, the first vantage point having a position and a perspective in relation to the environment, wherein the first view includes an object in the environment;   access, in a library of object models, an object model representation of the object, the object model including dimension data indicative of spatial dimensions of the object; and   populate the environment model with an instance of the object model at a location in the environment model, wherein the processor-executable instructions which cause the processor-based system to populate the environment model with the instance of the object model at the location cause the processor-based system to:   generate a second view of the environment model from a second vantage point, wherein a position and a perspective of the second vantage point in relation to the environment model substantially match the position and the perspective of the first vantage point in relation to the environment;   identify the location in the environment model where at least one spatial dimension of the instance of the object model in the second view of the environment model from the second vantage point substantially matches a corresponding spatial dimension of the object in the first view of the environment from the first vantage point; and   position the instance of the object model at the location, wherein:   the at least one processor is positioned at a robot controller remote from the environment;   the at least one non-transitory processor-readable storage medium is positioned at the robot controller;   the at least one non-transitory processor-readable storage medium stores the library of object models; and   the at least one non-transitory processor-readable storage medium stores the environment model.   
     
     
         17 . A computer program product comprising at least one non-transitory processor-readable storage medium storing processor-executable instructions and/or data that, when executed by at least one processor of a processor-based system, cause the processor-based system to:
 access, by the at least one processor, an environment model representation of an environment;   access, by the at least one processor, a first view of the environment from a first vantage point, the first vantage point having a position and a perspective in relation to the environment, wherein the first view includes an object in the environment and wherein the first view comprises first image data having a first resolution;   access, in a library of object models, an object model representation of the object, the object model including dimension data indicative of spatial dimensions of the object; and   populate the environment model with an instance of the object model at a location in the environment model, wherein the processor-executable instructions which cause the processor-based system to populate the environment model with the instance of the object model at the location cause the processor-based system to:   generate a second view of the environment model from a second vantage point, wherein a position and a perspective of the second vantage point in relation to the environment model substantially match the position and the perspective of the first vantage point in relation to the environment and wherein the second view comprises second image data having a second resolution, the second resolution different from the first resolution by a fixed ratio;   identify the location in the environment model where a number of pixels occupied by the instance of the object model in the second image data is equal to a number of pixels occupied by the object in the first image data multiplied by the fixed ratio; and   position the instance of the object model at the location.