Method and system of judgment record monitoring for outlier judgment guide

Abstract

A method for monitoring judgment records for an outlier judgment guide according to an embodiment of the inventive concepts is configured to detect at least one similarity image having a similarity greater than or equal to a predetermined reference with a predetermined inspection image, provide judgment detailed information, which is information specifying a judgement content on the presence or absence of an outlier, acquire first judgment result information, which is information specifying a judgement result on the presence or absence of an outlier for the inspection image, and then generates the inspection image and a first judgment result depending on a similarity between each of the inspection image and the similarity image.

1 . A method performed by a computing system for monitoring judgment records for an outlier judgment guide, the method comprising:
 extracting a previously inspected feature vector for each of a plurality of previously inspected overkill images pre-stored in a database using an image feature extraction model trained based on a patch feature;   acquiring a predetermined inspection image;   extracting an inspection feature vector for the inspection image using the image feature extraction model;   measuring a similarity between feature vectors of each of the inspection feature vector and a plurality of previously inspected feature vectors;   detecting at least one similarity image having a similarity greater than or equal to a predetermined reference with the acquired inspection image;   detecting a first previously inspected image among the plurality of previously inspected images, wherein the similarity between the measured feature vectors is greater than or equal to a preset similarity reference threshold, as a similarity image;   controlling an output device to output judgment detailed information, which is information specifying a judgment content for the presence or absence of an outlier for each of the detected similarity image and the inspection image;   acquiring first judgment result information, which is information specifying a determination result on the presence or absence of the outlier for the inspection image; and   storing the inspection image and the first judgment result information in the database according to the similarity between the feature vectors of each of the inspection image and the similarity image.   
     
     
         2 . The method of  claim 1 , wherein a provision of the judgment detailed information comprises providing at least one piece of information among worker-in-charge information, wherein worker-in-charge information is information specifying a worker-in-charge who performed a determination on the presence or absence of the outlier for the similarity image, and second judgement result information, and wherein second judgment information is information specifying a determination result on the presence or absence of the outlier for the similarity image. 
     
     
         3 . The method of  claim 1 , further comprising pre-training the image feature extraction model to output an integrated similarity comprising a pairwise similarity and a contextual similarity for a plurality of patch feature pairs. 
     
     
         4 . The method of  claim 3 , further comprising performing feature representation learning based on the integrated similarity for the inspection image using the image feature extraction model. 
     
     
         5 . The method of  claim 4 , wherein performance of the feature representation learning based on the integrated similarity comprises training a second network (SN) by applying the integrated similarity to a relaxed contrastive loss in a model comprising a first network (TN) and the SN. 
     
     
         6 . The method of  claim 4 , wherein the detection of the at least one similarity image further comprises calculating a decision similarity that specifies a final similarity between the inspection image and the first previously inspected image based on a raw data similarity and a feature vector similarity. 
     
     
         7 . The method of  claim 6 , wherein the detection of the similarity image further comprises detecting the similarity image based on the decision similarity and the preset similarity reference threshold. 
     
     
         8 . The method of  claim 7 , wherein the detection of the similarity image further comprises:
 determining whether the number of detected similarity images meets a preset number thereof; and   additionally detecting the similarity image when the preset number thereof is not met.   
     
     
         9 . The method of  claim 8 , wherein the additional detection of the similarity image comprises:
 adjusting the similarity reference threshold; and   detecting the similarity image based on the adjusted similarity reference threshold.   
     
     
         10 . The method of  claim 6 , wherein a provision of the similarity image and the judgement detailed information comprises further providing the decision similarity. 
     
     
         11 . The method of  claim 2 , wherein the provision of the similarity image and the judgement detailed information comprises:
 additionally detecting the similarity image according to a ratio of second judgment result information for each of the at least one similarity image; and   further providing additional provision images, which are the additionally detected similarity images.   
     
     
         12 . The method of  claim 2 , wherein the provision of the similarity image and the judgement detailed information comprises aligning the detected similarity images based on at least one of the worker-in-charge information or the similarity between each of the inspection image and the similarity image. 
     
     
         13 . The method of  claim 2 , wherein the storage in the database further comprises storing the inspection image and the first judgment result information in the database according to whether the first judgment result information and second judgment result information are identical. 
     
     
         14 . The method of  claim 1 , further comprising filtering the detected similarity images according to a preset condition. 
     
     
         15 . The method of  claim 1 , wherein the acquisition of the inspection image comprises:
 performing, by a vision inspection unit, first automatic outlier judgment using a machine learning-based defect judgment model on a first inspection image captured of a product on a production line;   deciding that reliability of a first automatic outlier detection result is less than a preset reference value; and   providing a worker with the first inspection image to decide and judge the inspection image.   
     
     
         16 . A system for monitoring judgment records for an outlier judgment guide, the system comprising:
 at least one inspection database;   at least one monitoring interface unit;   a memory; and   at least one processor configured to execute instructions stored in the memory, wherein the at least one processor is configured to:   acquire a predetermined inspection image;   extract an inspection feature vector for the inspection image and a previously inspected feature vector for each of a plurality of previously inspected images pre-stored in the inspection database using an image feature extraction model trained based on a patch feature;   measure a similarity between feature vectors of each of the inspection feature vector and a plurality of previously inspected feature vectors to detect, among the plurality of previously inspected images, a first previously inspected image whose measured similarity between the feature vectors is greater than or equal to a preset similarity reference threshold, as a similarity image;   control the monitoring interface unit to output the detected similarity image and judgement detailed information;   acquire first judgment result information through the monitoring interface unit; and   store the inspection image and the first judgment result information in the inspection database according to the similarity between the feature vectors.   
     
     
         17 . The system of  claim 16 , wherein the image feature extraction model is pre-trained to output an integrated similarity comprising a pairwise similarity and a contextual similarity for a plurality of patch feature pairs. 
     
     
         18 . The system of  claim 17 , wherein the image feature extraction model comprises a teacher network (TN) and a student network (SN) and performs feature representation learning by applying the integrated similarity to a relaxed contrastive loss. 
     
     
         19 . The system of  claim 16 , wherein the at least one processor is further configured to:
 additionally measure a raw data similarity between the inspection image and the first previously inspected image; and   combine the raw data similarity and the similarity between the feature vectors to calculate a decision similarity and to detect the similarity image based on the decision similarity.   
     
     
         20 . A vision inspection system providing an outlier judgment guide, the system comprising:
 a vision inspection unit that acquires a product image, performs a first automatic judgment using a patch feature-based outlier detection model, and decides an image with judgment reliability less than a preset reference value as an inspection image;   an inspection database storing past judgment records;   an inspection monitoring processing unit that measures a similarity between feature vectors of the inspection image and previously inspected images in the inspection database to detect a similarity image, and decides whether to store the inspection image in a database based on a judgment result from a worker; and   a monitoring interface unit that displays the inspection image, the detected similarity image, and judgment detailed information thereof to the worker under the control of the inspection monitoring processing unit,   wherein the inspection monitoring processing unit extracts each of an inspection feature vector of the inspection image and a previously inspected feature vector of the previously inspected images using the outlier detection model, and compares the similarity between the feature vectors to detect the similarity image among a plurality of previously inspected images in the inspection database.