Mislabeled product detection

Abstract

Aspects of this disclosure include technologies for detecting mislabeled products. In one embodiment, the disclosed system will capture an image of a product when the MRL of the product is scanned or being scanned. After recognizing the product in the image, the size of the area containing the product may be calculated. Subsequently, the disclosed system can determine whether the MRL mismatches the product in the image if this size of the area containing the product does not match the standard size associated with the MRL.

What is claimed is: 
     
         1 . A computer-implemented method for detecting mislabeled products, comprising:
 identifying a region of interest of an image based on a source device having scanned a product label, the source device being one of a plurality of source devices that configured to scan product labels;   detecting, via an object detection model, a product in the region of interest of the image; and   generating an alert in response to a mismatch between an attribute of the product in the region of interest of the image and a corresponding attribute associated with the product label.   
     
     
         2 . The computer-implemented method of  claim 1 , wherein the object detection model is a convolutional neural network based model, the computer-implemented method further comprising:
 selecting a part of the image containing the region of interest;   fetching only the part of the image to the convolutional neural network based model; and   output a bounding box of the product based on the convolutional neural network based model.   
     
     
         3 . The computer-implemented method of  claim 1 , further comprising:
 capturing the image in a top-down view, where identifying the region of interest comprises identifying a marking of the source device, and identifying the region of interest based on a location and a direction of the marking of the source device.   
     
     
         4 . The computer-implemented method of  claim 1 , wherein the attribute comprises a size attribute, the computer-implemented method further comprising:
 retrieving a standard size associated with the product label based on a product identifier associated with the product label;   determining a size of a bounding box of the product in the region of interest of the image; and   comparing the size of the bounding box of the product and the standard size associated with the product label.   
     
     
         5 . The computer-implemented method of  claim 4 , wherein determining the size of the bounding box comprises identifying a size reference in the image, and determining the size of the bounding box based on a known measurement of the size reference. 
     
     
         6 . The computer-implemented method of  claim 4 , wherein determining the size of the bounding box comprises identifying pixels in the bounding box, and determining the size of the bounding box based on the pixels in the bounding box in relation to pixels in the image. 
     
     
         7 . The computer-implemented method of  claim 4 , wherein the standard size associated with the product label comprises a minimum size and a maximum size, the computer-implemented method further comprising:
 determining the size of the bounding box being outside of a range of the minimum size and the maximum size.   
     
     
         8 . The computer-implemented method of  claim 4 , further comprising:
 adjusting the standard size associated with the product label based on the size of the bounding box of the product in the region of interest of the image.   
     
     
         9 . The computer-implemented method of  claim 1 , wherein the attribute of the product comprises a combination of size and content, the computer-implemented method further comprising:
 detecting the mismatch based on a content dissimilarity between the product in the region of interest and content data associated with the product label, and a size dissimilarity between the product in the region of interest of the image and size data associated with the product label.   
     
     
         10 . An apparatus to detect mislabeled products, the apparatus comprising a memory having computer programs stored thereon and a processor configured to perform, when executing the computer programs, operations comprising:
 identifying a region of interest of an image based on a source device having scanned a product label, the source device being one of a plurality of source devices that configured to scan product labels;   detecting, via an object detection model, a product in the region of interest of the image; and   generating an alert in response to a mismatch between an attribute of the product in the region of interest of the image and a corresponding attribute associated with the product label.   
     
     
         11 . The apparatus of  claim 10 , wherein the identifying comprises identifying a marking of the source device, and identifying the region of interest based on a location and a direction of the marking in the image. 
     
     
         12 . The apparatus of  claim 10 , wherein the detecting comprises generating a bounding box of the product by fetching only the region of interest to a convolutional neural network based object detection model. 
     
     
         13 . The apparatus of  claim 12 , the operations further comprising:
 retrieving a standard size associated with the product label based on a product identifier associated with the product label;   determining a size of the bounding box of the product in the region of interest of the image; and   comparing the size of the bounding box of the product and the standard size associated with the product label.   
     
     
         14 . The apparatus of  claim 13 , the operations further comprising:
 determining the mismatch based on the size of the bounding box being outside of a range of the standard size that comprises a minimum size and a maximum size.   
     
     
         15 . The apparatus of  claim 13 , wherein the standard size associated with the product label comprises a maximum size, the operations further comprising:
 determining the maximum size based on a longest internal diagonal of a three-dimensional bounding box associated with the product; and   adjusting the maximum size by a tolerance coefficient.   
     
     
         16 . The apparatus of  claim 13 , the operations further comprising:
 adjusting the standard size associated with the product label based on the size of the bounding box of the product in the region of interest of the image.   
     
     
         17 . The apparatus of  claim 10 , the operations further comprising:
 showing the alert to a user associated with the source device.   
     
     
         18 . A system for detecting mismatched products, comprising:
 an object assessor configured to identify a region of interest of an image based on a source device having scanned a product label, the source device being one of a plurality of source devices that configured to scan product labels;   the object assessor further configured to detecting, via an object detection model, a product in the region of interest of the image; and   another assessor, operationally coupled to the object assessor, configured to detect a mismatch between an attribute of the product in the region of interest of the image and a corresponding attribute associated with the product label.   
     
     
         19 . The system of  claim 18 , wherein the another assessor is further configured to detect the mismatch based on a size of a bounding box of the product in the region of interest of the image being outside of a range of a standard size associated with the product label, the standard size having a minimum size and a maximum size. 
     
     
         20 . The system of  claim 19 , wherein the another assessor is further configured to determine the standard size associated with the product label based on a distribution of sizes corresponding to a plurality of bounding boxes of the product in respective images of the product.