Semantic Zone Separation for Map Generation

Abstract

Methods, systems, and apparatus for receiving a mapping of a property that includes a three-dimensional representation of the property, receiving observations of the property that each depict a portion of the property, providing the mapping and the observations to an object mapping engine, receiving an object mapping of the property, wherein the object mapping includes a plurality of object labels that each identify an object that was recognized from the observations and a location of the object within the three-dimensional representation that corresponds to a physical location of the object in the property, and obtaining a semantic mapping of the property that identifies semantic zones of the property with respect to the three-dimensional representation, wherein the semantic mapping is generated based on an output that results from a semantic mapping model processing the object mapping.

What is claimed is: 
     
         1 . A computer-implemented method comprising:
 receiving a first set of data that represents a three-dimensional (3D) mapping of an environment;   receiving a second set of data that represents an observation of the environment separate from the 3D mapping, the observation including representations of a plurality of objects in the environment observed by a sensing system;   processing the first set of data and the second set of data to generate an object mapping of the environment;   generating, based on the object mapping, a semantic mapping of the environment that includes semantic labels for a plurality of semantic zones in the environment;   determining, based on the semantic mapping and for a robotic device instructed to locate a particular object type, one or more areas of the environment to prioritize searching for the particular object type; and   causing the robotic device to search the one or more determined areas of the environment to locate the particular object type.   
     
     
         2 . The computer-implemented method of  claim 1 , further comprising:
 determining, based on the semantic mapping and for the robotic device, one or more other areas of the environment to avoid searching for the particular object type; and   causing the robotic device to avoid the one or more other determined areas of the environment when searching for the particular object type.   
     
     
         3 . The computer-implemented method of  claim 1 , further comprising using the semantic mapping to modify a language model used by the robotic device to perform speech recognition. 
     
     
         4 . The computer-implemented method of  claim 3 , wherein using the semantic mapping to modify the language model comprises causing the language model to favor one or more terms commonly heard in a particular semantic zone over one or more other terms less commonly heard in the particular semantic zone. 
     
     
         5 . The computer-implemented method of  claim 3 , further comprising using the language model to interpret language describing one or more objects in a particular semantic zone in order to control the robotic device to interact with the one or more objects. 
     
     
         6 . The computer-implemented method of  claim 3 , further comprising using the language model to interpret language spoken by a user in proximity to the robotic device. 
     
     
         7 . The computer-implemented method of  claim 1 , further comprising using the semantic mapping to select an object from among two or more objects for the robotic device to retrieve in response to a user command. 
     
     
         8 . The computer-implemented method of  claim 1 , further comprising using the semantic mapping to control the robotic device to avoid one or more of the semantic zones at one or more particular times of day. 
     
     
         9 . The computer-implemented method of  claim 1 , further comprising using the semantic mapping to control the robotic device to enter a silent mode in which the robotic device avoids emitting sounds in one or more of the semantic zones at one or more particular times of day. 
     
     
         10 . The computer-implemented method of  claim 1 , further comprising using the semantic mapping to determine an area in the environment for the robotic device to position itself when not performing other tasks in the environment. 
     
     
         11 . The computer-implemented method of  claim 10 , wherein the determined area comprises a semantic zone in which the robotic device expects to encounter one or more users requesting assistance. 
     
     
         12 . The computer-implemented method of  claim 1 , further comprising using the semantic mapping to determine a path for the robotic device to navigate from one location to another location in the environment. 
     
     
         13 . The computer-implemented method of  claim 1 , further comprising using the semantic mapping to update the object mapping of the environment. 
     
     
         14 . The computer-implemented method of  claim 1 , wherein the sensing system is part of the robotic device. 
     
     
         15 . A system comprising:
 one or more processors configured to execute computer program instructions; and   one or more computer-storage media encoded with one or more computer programs that, when executed by the one or more processors, cause the system to perform operations comprising:   receiving a first set of data that represents a three-dimensional (3D) mapping of an environment;   receiving a second set of data that represents an observation of the environment separate from the 3D mapping, the observation including representations of a plurality of objects in the environment observed by a sensing system;   processing the first set of data and the second set of data to generate an object mapping of the environment;   generating, based on the object mapping, a semantic mapping of the environment that includes semantic labels for a plurality of semantic zones in the environment;   determining, based on the semantic mapping and for a robotic device instructed to locate a particular object type, one or more areas of the environment to prioritize searching for the particular object type; and   causing the robotic device to search the one or more determined areas of the environment to locate the particular object type.   
     
     
         16 . The system of  claim 15 , wherein the operations further comprise using the semantic mapping to select an object from among two or more objects for the robotic device to retrieve in response to a user command. 
     
     
         17 . The system of  claim 15 , wherein the operations further comprise using the semantic mapping to modify a language model used by the robotic device to perform speech recognition. 
     
     
         18 . The system of  claim 15 , wherein the operations further comprise using the semantic mapping to control the robotic device to avoid one or more of the semantic zones at one or more particular times of day. 
     
     
         19 . The system of  claim 15 , wherein the operations further comprise using the semantic mapping to determine an area in the environment for the robotic device to position itself when not performing other tasks in the environment. 
     
     
         20 . One or more computer-readable devices storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:
 receiving a first set of data that represents a three-dimensional (3D) mapping of an environment;   receiving a second set of data that represents an observation of the environment separate from the 3D mapping, the observation including representations of a plurality of objects in the environment observed by a sensing system;   processing the first set of data and the second set of data to generate an object mapping of the environment;   generating, based on the object mapping, a semantic mapping of the environment that includes semantic labels for a plurality of semantic zones in the environment;   determining, based on the semantic mapping and for a robotic device instructed to locate a particular object type, one or more areas of the environment to prioritize searching for the particular object type; and   causing the robotic device to search the one or more determined areas of the environment to locate the particular object type.