Methods and systems for automatically detecting design elements in a two-dimensional design document

Abstract

Systems and methods are disclosed for automatically detecting a design element in a design document. One method comprises receiving a design document and generating an enhanced design document based on the received design document. The enhanced design document may be generated by augmenting additional information to the design document using machine learning techniques. In response to receiving a user input, one or more design elements in the enhanced design document may be determined, and additional information associated with the determined one or more design elements may be displayed to the user.

1 .- 20 . (canceled) 
     
     
         21 . A computer-implemented method for detecting two-dimensional (2D) elements in 2D documents, the method comprising:
 receiving a plurality of 2D sample documents;   extracting training data from the plurality of 2D sample documents, the training data including i) a plurality of 2D sample elements visually displayed on the plurality of 2D sample documents and ii) a plurality of designations stored on the plurality of 2D sample documents and associated with the plurality of 2D sample elements; and   training a machine learning model with the training data by:   detecting visual objects in the plurality of 2D sample documents indicative of the plurality of 2D sample elements for identifying 2D elements in 2D documents; and   distinguishing visual features between the plurality of 2D sample elements displayed on the plurality of 2D sample documents for classifying each of the 2D elements identified in the 2D documents with at least one of the plurality of designations.   
     
     
         22 . The method of  claim 21 , wherein each of the plurality of 2D sample elements includes an image of a drawing symbol displayed on the plurality of 2D sample documents, and the designation includes a classification type labeled to the drawing symbol on the plurality of 2D sample documents. 
     
     
         23 . The method of  claim 21 , wherein each of the 2D documents includes a visual content document, an image, a picture, a drawing, or a media file; and
 wherein each of the 2D elements includes one or more lines, one or more geometrical shapes, or one or more polygons representing an object visually displayed on the 2D documents.   
     
     
         24 . The method of  claim 21 , wherein extracting the training data from the plurality of 2D sample documents comprises:
 retrieving images of each of the plurality of 2D sample elements displayed on the plurality of 2D sample documents; and   generating a thumbnail picture for each of the images retrieved from the plurality of 2D sample documents.   
     
     
         25 . The method of  claim 24 , wherein extracting the training data from the plurality of 2D sample documents further comprises:
 extracting information stored on the plurality of 2D sample documents indicative of the designation of each of the plurality of 2D sample elements displayed on the plurality of 2D sample documents;   pairing the designation of each of the plurality of 2D sample elements with the respective thumbnail picture generated for each of the images retrieved from the plurality of 2D sample documents, wherein each of the thumbnail pictures and a respective label representative of the paired designation are provided to the machine learning model as the training data.   
     
     
         26 . The method of  claim 21 , wherein each of the plurality of 2D sample documents includes one or more sources of image data of the plurality of 2D sample elements labeled with the plurality of designations. 
     
     
         27 . The method of  claim 21 , further comprising:
 splitting the training data into a first part and a second part, wherein the visual objects in the 2D sample documents indicative of the plurality of 2D sample elements are detected and the visual features are distinguished between the plurality of 2D sample elements using the first part of the training data; and   validating the detected visual objects and the distinguished visual features using the second part of the training data to determine an accuracy of the machine learning model in identifying the 2D elements in the 2D documents and classifying each of the 2D elements to at least one of the plurality of designations.   
     
     
         28 . A method for automatically generating an enhanced 2D document, the method comprising:
 receiving a 2D document;   detecting, using a trained machine learning model, one or more 2D elements in the 2D document, wherein the trained machine learning model is trained using training data including i) a plurality of 2D sample elements visually displayed on a plurality of 2D sample documents and ii) a plurality of designations stored on the plurality of 2D sample documents and associated with the plurality of 2D sample elements;   determining a location and a designation associated with each of the detected one or more 2D elements; and   generating the enhanced 2D document by augmenting the 2D document with the determined location and the designation of each of the detected one or more 2D elements.   
     
     
         29 . The method of  claim 28 , further comprising:
 receiving, via a graphical user interface of a document review application, a user selection of a 2D element in the 2D document;   determining, using the enhanced 2D document, information associated with the user-selected 2D element, the information including at least one of the determined location or the designation of the user-selected 2D element; and   displaying, via the graphical user interface, the information associated with the user-selected 2D element.   
     
     
         30 . The method of  claim 28 , further comprising:
 receiving, via a graphical user interface of a document review application, a search query provided by a user in relation to the 2D document, the search query including a keyword;   determining, using the enhanced 2D document, that a designation of a 2D element in the 2D document matches the keyword;   visually distinguishing, via the graphical user interface, the 2D element associated with the designation matching the keyword.   
     
     
         31 . The method of  claim 28 , wherein the trained machine learning model is trained by:
 detecting visual objects in the plurality of 2D sample documents indicative of the plurality of 2D sample elements for identifying 2D elements in 2D documents; and   distinguishing visual features between the plurality of 2D sample elements displayed on the plurality of 2D sample documents for classifying each of the 2D elements identified in the 2D documents with at least one of the plurality of designations.   
     
     
         32 . The method of  claim 28 , wherein the plurality of 2D sample documents include a plurality of enhanced 2D documents that are augmented using data automatically extracted from a three-dimensional (3D) model, and the training data is extracted from, at least in part, the plurality of enhanced 2D documents. 
     
     
         33 . The method of  claim 28 , wherein determining the designation associated with each of the detected one or more 2D elements comprises classifying, using the trained machine learning model, each of the detected one or more 2D elements with at least one of the plurality of designations. 
     
     
         34 . The method of  claim 28 , wherein determining the location associated with each of the detected one or more 2D elements comprises determining x and y coordinates representative of a distance of the corresponding detected 2D element from an edge of the 2D document. 
     
     
         35 . A system for automatically generating an enhanced 2D document, the system comprising:
 one or more processors; and   one or more storage devices storing instructions that, when executed by the one or more processors, cause the one or more processors to perform a method comprising:   receiving a 2D document;   detecting, using a trained machine learning model, one or more 2D elements in the 2D document, wherein the trained machine learning model is trained using training data including i) a plurality of 2D sample elements visually displayed on a plurality of 2D sample documents and ii) a plurality of designations stored on the plurality of 2D sample documents and associated with the plurality of 2D sample elements;   determining a location and a designation associated with each of the detected one or more 2D elements; and   generating the enhanced 2D document by augmenting the 2D document with the determined location and the designation of each of the detected one or more 2D elements.   
     
     
         36 . The system of  claim 35 , wherein the method comprises:
 receiving, via a graphical user interface of a document review application, a user selection of a 2D element in the 2D document;   determining, using the enhanced 2D document, information associated with the user-selected 2D element, the information including at least one of the determined location or the designation of the user-selected 2D element; and   displaying, via the graphical user interface, the information associated with the user-selected 2D element.   
     
     
         37 . The system of  claim 35 , wherein the method comprises:
 receiving, via a graphical user interface of a document review application, a search query provided by a user in relation to the 2D document, the search query including a keyword;   determining, using the enhanced 2D document, that a designation of a 2D element in the 2D document matches the keyword;   visually distinguishing, via the graphical user interface, the 2D element associated with the designation matching the keyword.   
     
     
         38 . The system of  claim 35 , wherein the trained machine learning model is trained by:
 detecting visual objects in the plurality of 2D sample documents indicative of the plurality of 2D sample elements for identifying 2D elements in 2D documents; and   distinguishing visual features between the plurality of 2D sample elements displayed on the plurality of 2D sample documents for classifying each of the 2D elements identified in the 2D documents with at least one of the plurality of designations.   
     
     
         39 . The system of  claim 35 , wherein the plurality of 2D sample documents include a plurality of enhanced 2D documents that are augmented using data automatically extracted from a three-dimensional (3D) model, and the training data is extracted from, at least in part, the plurality of enhanced 2D documents. 
     
     
         40 . The system of  claim 35 , wherein determining the location associated with each of the detected one or more 2D elements comprises determining x and y coordinates representative of a distance of the corresponding detected 2D element from an edge of the 2D document; and
 wherein determining the designation associated with each of the detected one or more 2D elements comprises classifying, using the trained machine learning model, each of the detected one or more 2D elements with at least one of the plurality of designations.