Machine learning model for automatically detecting faults using optical inspection

Abstract

A computer-implemented method for generating a trained machine learning model for automatically detecting faults in a manufactured product using optical inspection is provided, including receiving a multitude of image data of images taken from manufactured products to be inspected, establishing a segmentation machine learning model which inputs the image data of one image and out-puts a first number of image data segments, establishing a detection machine learning model including a separate detection machine learning model for each of the image data segments, combining each of the separate detection machine learning models with the segmentation machine learning model generating a paired machine learning model for each of the image data segments, coupling the paired machine learning models of all image data segments according to a fault criterion, and generating a trained machine learning model by optimizing the coupled paired machine learning models for all image data segments of the received images.

1 . A computer-implemented method for generating a trained machine learning model for automatically detecting faults in a manufactured product using optical inspection, comprising:
 receiving a multitude of image data of images taken from manufactured products to be inspected;   establishing a segmentation machine learning model which inputs the image data of one image and outputs a first number of image data segments, each image data segment covering a coherent subset of image data;   establishing a detection machine learning model comprising a separate detection machine learning model for each of the image data segments, wherein each of the separate detecting machine learning model inputs the image data of the image data segment and outputs a fault probability for the respective image data segment;   combining each of the separate detection machine learning models with the segmentation machine learning model generating a paired machine learning model for each of the image data segments;   coupling the paired machine learning models of all image data segments according to a fault criterion; and   generating the trained machine learning model by optimizing the coupled paired machine learning models for all image data segments of the received images.   
     
     
         2 . The method according to  claim 1 , wherein the first number of image data segments is predefined, based on domain knowledge. 
     
     
         3 . The method according to  claim 1 , wherein the trained machine learning model outputs:
 an indication of a location for each of the determined image data segments related to the image of the manufactured product, and   a fault probability for each determined image data segment, and/or   a fault value for the manufactured product as a whole.   
     
     
         4 . The method according to  claim 1 , wherein the fault criterion is an accumulated fault adding the fault probabilities across all image data segments. 
     
     
         5 . The method according to  claim 1 , wherein the fault criterion is an exclusive fault, wherein the manufactured product is not faulty if none of the image data segments are faulty. 
     
     
         6 . The method according to  claim 1 , wherein a label is associated to the image data indicating that the manufactured product as a whole is faulty, or that the manufactured product as a whole is not faulty. 
     
     
         7 . The method according to  claim 6 , wherein the label indicates a type of fault of the manufactured product as a whole. 
     
     
         8 . The method according to  claim 1 , wherein each separate detection machine learning model is a neural network which outputs a probability value based only on the image data of the image data segment determined by the segmentation machine learning model. 
     
     
         9 . The method according to  claim 1 , wherein the segmentation machine learning model is a neural network which outputs a random one hot vector containing a single value of one at a specific position and zeros everywhere else, wherein the position of the value of one follows a discrete distribution that is reparametrized to be continuous and differentiable. 
     
     
         10 . The method according to  claim 1 , wherein the manufactured product is a printed circuit board, a produced part in automotive industries or in food and beverage industry. 
     
     
         11 . A computer-implemented method for automatically detecting faults in a manufactured product using optical inspection, comprising:
 receiving an image data of an image taken from a manufactured product to be inspected,   inputting the image data into the trained machine learning model generated according to  claim 1 ,   outputting an indication of a location of the deter-mined image data segments related to the image of the manufactured product, and   outputting a fault probability for each determined image data segment, and/or   outputting a fault value for the manufactured product.   
     
     
         12 . A training apparatus for generating a trained machine learning model, comprising:
 an input module, configured to receive a multitude of image data of images taken from manufactured products to be inspected;   a segmentation module, configured to establish a seg-mentation machine learning model which inputs the image data of one image and outputs a first number of image data segments, each image data segment covering a coherent subset of image data;   a detection module, configured to establish a detection machine learning model comprising a separate detection machine learning model for each of the image data segments, wherein each of the separate detecting machine learning model inputs the image data of the image data segment and outputs a fault probability for the respective image data segment;   a combiner module, configured to combine each of the separate detecting machine learning models with the segmentation machine learning model generating a paired machine learning model for each of the image data segments;   a coupler module, configured to couple the paired machine learning models of all image data segments according to a fault criterion; and   an optimization module, configured to generate a trained machine learning model by optimizing the coupled paired machine learning models for all image data segments of the received images.   
     
     
         13 . A detection apparatus for automatically detecting faults in a manufactured product using optical inspection, comprising:
 an input module, configured to receive an image data of an image taken from a manufactured product to be inspected;   a detection module, configured to input the image data into a trained machine learning model generated according to  claim 1 ; and   an output module, configured to output an indication of a location of the determined image data segments related to the image of the manufactured product, and to output a fault probability for each determined image data segments, and/or to output a fault value for the manufactured product.   
     
     
         14 . A computer program product, comprising a computer readable hardware storage device having computer readable program code stored therein, said program code executable by a processor of a computer system to implement a method of  claim 1  when the product is run on the digital computer. 
     
     
         15 . The computer program product directly loadable into the in-ternal memory of a digital computer, comprising software code portions for performing the steps of  claim 11  when the product is run on the digital computer.