Convolutional neural network-based mode selection and defect classification for image fusion

Abstract

Systems and methods for classifying defects using hot scans and convolutional neural networks (CNNs) are disclosed. Primary scanning modes are identified by a processor and a hot scan of a wafer is performed. Defects of interest and nuisance data are selected and images of those areas are captured usa7ing one or more secondary scanning modes. Image sets are collected and divided into subsets. CNNs are trained using the image subsets. An ideal secondary scanning mode is determined and a final hot scan is performed. Defects are filtered and classified according to the final hot scan and the ideal secondary scanning mode. Disclosed systems for classifying defects utilize image data acquisition subsystems such as a scanning electron microscope as well as processors and electronic databases.

What is claimed is: 
     
       1. A method for classifying defects comprising:
 identifying, using a processor, a primary scanning mode; 
 performing, using an image data acquisition subsystem, a first hot scan using the identified primary scanning mode; 
 selecting, using the processor, a plurality of defects of interest and nuisance data in the first hot scan; 
 performing, using the image data acquisition subsystem, one or more additional scans using one or more secondary scanning modes; 
 collecting, using the processor, one or more image sets, each image set comprising a primary scanning mode image and one or more secondary scanning mode images corresponding to a selected defect of interest or nuisance data; 
 dividing, using the processor, each collected image set into a first image subset and a second image subset; 
 training, using the processor, for each pair of primary scanning mode and secondary scanning mode, a convolutional neural network (CNN) with the corresponding first image subset; 
 determining an ideal secondary scanning mode, using the processor, by applying each CNN to the corresponding second image subset; 
 performing, using the image data acquisition subsystem, a final hot scan using the ideal secondary scanning mode; and 
 classifying, using the processor, defects from the final hot scan by using the CNN corresponding to the ideal secondary scanning mode to filter out nuisance data in the final hot scan. 
 
     
     
       2. The method of  claim 1 , wherein the primary scanning mode is identified by running a hot scan to detect a defect. 
     
     
       3. The method of  claim 1 , wherein the one or more secondary scanning modes deviate from the primary scanning mode based on focus offset. 
     
     
       4. The method of  claim 1 , wherein the one or more secondary scanning modes deviate from the primary scanning mode based on aperture. 
     
     
       5. The method of  claim 1 , wherein the one or more secondary scanning modes deviate from the primary scanning mode based on spectrum. 
     
     
       6. The method of  claim 1 , wherein the one or more secondary scanning modes deviate from the primary scanning mode based on polarization. 
     
     
       7. The method of  claim 1 , wherein the step of training the CNN includes using transfer learning to create hyperparameters for each CNN. 
     
     
       8. The method of  claim 1 , wherein each CNN is evaluated based on a separation between the plurality of defects of interest and nuisance data. 
     
     
       9. The method of  claim 1 , wherein each additional scan uses a different secondary scanning mode. 
     
     
       10. The method of  claim 1 , wherein input to the CNN is six images per selected defect of interest and nuisance data. 
     
     
       11. The method of  claim 10 , wherein the images are 32×32 pixels in size. 
     
     
       12. The method of  claim 10 , wherein the six images comprise a test image, a reference image, a difference image for the primary scanning mode and a test image, a reference image, and a difference image for one of the secondary scanning modes. 
     
     
       13. The method of  claim 1 , wherein input images are processed through one or more rectified linear unit layers. 
     
     
       14. The method of  claim 13 , wherein the rectified linear unit layer utilizes one or more filters. 
     
     
       15. The method of  claim 13 , wherein an end result is a fully connected layer. 
     
     
       16. The method of  claim 13 , wherein one or more pooling layers are utilized. 
     
     
       17. A system for classifying defects comprising:
 an image data acquisition subsystem; and 
 a processor in electronic communication with the image data acquisition subsystem, the processor configured to:
 identify a primary scanning mode; 
 instruct the image data acquisition subsystem to return a first hot scan using the identified primary scanning mode; 
 identify a plurality of defects of interest and nuisance data in the returned first hot scan; 
 instruct the image data acquisition subsystem to return one or more scans using one or more secondary scanning modes; 
 collect one or more image sets, each image set comprising a primary scanning mode image and one or more secondary scanning mode images corresponding to a selected defect of interest or nuisance data; 
 divide each collected image set into a first image subset and a second image subset; 
 train, for each pair of primary scanning mode and secondary scanning mode, a convolutional neural network (CNN) with the corresponding first image subset; 
 determine an ideal secondary scanning mode by applying each CNN to the corresponding second image subset; 
 instruct the image data acquisition subsystem to return a final hot scan using the ideal secondary scanning mode; and 
 classify defects from the final hot scan by using the CNN corresponding to the ideal secondary scanning mode to filter out nuisance data in the final hot scan. 
 
 
     
     
       18. The system of  claim 17  further comprising a database in electronic communication with the processor and the image data acquisition subsystem, the database configured to store classified defects from the final hot scan. 
     
     
       19. The system of  claim 18 , wherein the database is also configured to store one or more CNNs.