System and method for generating simulated scenes from open map data for machine learning

Abstract

Systems and methods for generating simulated scenes from open map data for machine learning are presented. The system includes an automatic scene generative pipeline that uses freely-available map information and random texture maps to create large-scale 3D urban scene layouts for supervised learning methods. The system generates synthetic datasets that have improved generalization capabilities with respect to a given target domain of interest using data from open maps and texture map from the same geographic locations. Data from the generation pipeline of the system improves a model's generalization to real image sets beyond arbitrarily-simulated sets or labeled real data from other geographical regions.

What is claimed is: 
     
       1. A system for generating a simulated scene from open map data for training a machine learning system, comprising:
 a simulation computer system; 
 means for receiving open map data; 
 means for receiving texture map data; and 
 a scene generation software engine executed by the simulation computer system, the scene generation software engine causing the simulation computer system to:
 generate a three-dimensional (3D) scene layout using the open map data and the texture map data; 
 capture the scene layout from a plurality of different camera view points; and 
 generate machine learning training data using the captured scene layout. 
 
 
     
     
       2. The system of  claim 1 , wherein the scene generation software engine further causes the simulation computer system to present a bounding box to a user of the simulation computer system. 
     
     
       3. The system of  claim 2 , wherein the bounding box allows the user to define a geographical region of interest. 
     
     
       4. The system of  claim 1 , wherein the scene generation software engine further causes the simulation computer system to perform a stochastic process on the scene layout to add virtual objects to the scene layout. 
     
     
       5. The system of  claim 1 , wherein the scene generation software engine further causes the simulation computer system to add simulated lighting and weather conditions to the scene layout. 
     
     
       6. The system of  claim 1 , wherein the scene generation software engine further causes the simulation computer system to project buildings, roads, vegetation, and bodies of water as two-dimensional planes. 
     
     
       7. The system of  claim 6 , wherein the scene generation software engine further causes the simulation computer system to add heights to each building plane and each building polygon. 
     
     
       8. The system of  claim 7 , wherein the scene generation software engine further causes the simulation computer system to store road lanes as polygons. 
     
     
       9. The system of  claim 8 , wherein the scene generation software engine further causes the simulation computer system to extrude road planes having lane-width parameters. 
     
     
       10. The system of  claim 9 , wherein the scene generation software engine further causes the simulation computer system to create a final 3D mesh scene layout using the two-dimensional planes, heights, polygons, and the extruded road planes. 
     
     
       11. The system of  claim 9 , wherein the scene generation software engine further causes the simulation computer system to apply the texture map data to the 3D mesh to create the scene layout. 
     
     
       12. The system of  claim 1 , wherein the scene generation software engine further causes the simulation computer system to obtain a 3D computer-aided design (CAD) file of virtual objects and add the virtual objects to the scene layout. 
     
     
       13. The system of  claim 1 , wherein the scene generation software engine further causes the simulation computer system to capture aerial and ground views using different camera viewpoints and extract images or videos including semantic labels associated with the captured aerial and ground views. 
     
     
       14. A method for generating a simulated scene from open map data for training a machine learning system, comprising:
 receiving open map data; 
 receiving texture map data 
 generating at a simulation computer system a three-dimensional (3D) scene layout using the open map data and the texture map data; 
 capturing the scene layout from a plurality of different camera view points; and 
 generating machine learning training data using the captured scene layout. 
 
     
     
       15. The method of  claim 14 , further comprising presenting a bounding box to a user of the simulation computer system and allowing the user to define a geographical region of interest. 
     
     
       16. The method of  claim 14 , further comprising performing a stochastic process on the scene layout to add virtual objects to the scene layout. 
     
     
       17. The method of  claim 14 , further comprising adding simulated lighting and weather conditions to the scene layout. 
     
     
       18. The method of  claim 14 , further comprising projecting buildings, roads, vegetation, and bodies of water as two-dimensional planes. 
     
     
       19. The method of  claim 18 , further comprising adding heights to each building plane and each building polygon. 
     
     
       20. The method of  claim 19 , further comprising storing road lanes as polygons. 
     
     
       21. The method of  claim 20 , further comprising extruding road planes having lane-width parameters. 
     
     
       22. The method of  claim 21 , further comprising generating a final 3D mesh scene layout using the two-dimensional planes, heights, polygons, and the extruded road planes. 
     
     
       23. The method of  claim 22 , further comprising applying the texture map data to the 3D mesh to create the scene layout. 
     
     
       24. The method of  claim 14 , further comprising obtaining a 3D computer-aided design (CAD) file of virtual objects and add the virtual objects to the scene layout. 
     
     
       25. The method of  claim 14 , further comprising capturing aerial and ground views using different camera viewpoints and extract images or videos including semantic labels associated with the captured aerial and ground views.