US12125189B2ActiveUtilityA1
Method of detecting product defects, electronic device, and storage medium
Est. expiryJun 15, 2041(~14.9 yrs left)· nominal 20-yr term from priority
G06T 7/11G06T 7/70G06T 2207/20104G06V 10/243G06V 10/25G06T 2207/30108G06T 2207/20021G06T 2207/30164G06T 7/12G06T 7/13G06T 7/0004
63
PatentIndex Score
0
Cited by
8
References
20
Claims
Abstract
A method of detecting product defects obtains an image of a product and sets a region of interest (ROI) of the image. A first contour of a first target object is detected in the region of interest. The image is detected according to the first contour to obtain a corrected image. A position difference between the first contour and a second target object in the region of interest is obtained. A second contour of the second target object is detected in the corrected image according to the position difference. A first image area corresponding to the first contour and a second image area corresponding to the second contour are segmented and input into an autoencoder. According to outputs of the autoencoder, whether the product is defective is determined. A detection result of the product is output. The method can detect defects on products quickly and accurately.
Claims
exact text as granted — not AI-modifiedWe claim:
1. A method of detecting product defects, comprising:
obtaining an image of a product, and setting a region of interest of the image;
detecting a first contour of a first target object in the region of interest;
correcting the image according to the first contour to obtain a corrected image;
obtaining a position difference between the first contour and a second target object in the region of interest;
detecting a second contour of the second target object in the corrected image according to the position difference;
segmenting a first image area corresponding to the first contour and a second image area corresponding to the second contour; and
inputting the first image area into an autoencoder to obtain a first output, inputting the second image area into the autoencoder to obtain a second output, determining whether the product is defective according to the first output and the second output, and outputting a detection result of the product.
2. The method of claim 1 , wherein correcting the image according to the first contour comprises:
establishing a coordinate system, an origin of the coordinate system being a lower left corner of the image when the image is placed in a forward direction, an X-axis of the coordinate system being a horizontal direction of the image, and a Y-axis of the coordinate system being a vertical direction of the image;
determining coordinates of four vertices of the first contour in the coordinate system, the four vertices coordinates denoted as A 1 (x1,y1), A 2 (x2,y2), A 3 (x3,y3), and A 4 (x4,y4), wherein A 1 is a vertex closest to the X-axis, and the vertex A 1 , the vertex A 2 , the vertex A 3 , and the vertex A 4 are distributed in a counterclockwise order;
calculating coordinates of a center of the first contour according to the coordinates of the four vertices;
calculating an angle α between the first contour and the X-axis; and
correcting the image with the coordinate of the center of the first contour as a rotation center and the angle α as a rotation angle.
3. The method of claim 2 , wherein the coordinate of the center of the first contour is
M
(
x
1
-
(
x
1
-
x
3
)
2
,
y
3
-
(
y
3
-
y
1
)
2
)
.
4. The method of claim 2 , wherein a formula of calculating the angle α between the first contour and the X-axis is:
α
=
arc
tan
(
y
2
-
y
1
x
2
-
x
1
)
.
5. The method of claim 1 , wherein the first output indicates whether the first image area corresponds to a first defective region of the product, the second output indicates whether the second image area corresponds to a second defective region of the product, and determining whether the product is defective according to the first output and the second output comprises:
determining that the product is defective when the first image area corresponds to the first defective region or the second image area corresponds to the second defective region; and
determining that the product is flawless when neither the first image area corresponds to the first defective region nor the second image area corresponds to the second defective region.
6. The method of claim 1 , wherein inputting the first image area into an autoencoder to obtain a first output, inputting the second image area into the autoencoder to obtain a second output, determining whether the product is defective according to the first output and the second output comprises:
performing vectorization processing on the first image area to obtain a first feature vector, and performing vectorization processing on the second image area to obtain a second feature vector;
inputting the first feature vector into the autoencoder to obtain a first reconstructed image, and inputting the second feature vector into the autoencoder to obtain a second reconstructed image;
calculating a first reconstruction error between the first image area and the first reconstructed image, and calculating a second reconstruction error between the second image area and the second reconstructed image; and
determining whether the product is defective according to the first reconstruction error and the second reconstruction error.
7. The method of claim 6 , wherein calculating a first reconstruction error between the first image area and the first reconstructed image comprises:
calculating a first mean square error between the first image area and pixels in the first reconstructed image as the first reconstruction error; and
calculating a second reconstruction error between the second image area and the second reconstructed image comprises:
calculating a second mean square error between the second image area and pixels in the second reconstructed image as the second reconstruction error.
8. The method of claim 6 , wherein determining whether the product is defective according to the first reconstruction error and the second reconstruction error comprises:
determining that the product is defective when the first reconstruction error or the second reconstruction error is greater than a preset threshold; and
determining that the product is flawless when neither the first reconstruction error nor the second reconstruction error is greater than the threshold.
9. An electronic device comprising:
at least one processor; and
a storage device storing computer-readable instructions, which when executed by the at least one processor, cause the at least one processor to:
obtain an image of a product, and set a region of interest of the image;
detect a first contour of a first target object in the region of interest;
correct the image according to the first contour to obtain a corrected image;
obtain a position difference between the first contour and a second target object in the region of interest;
detect a second contour of the second target object in the corrected image according to the position difference;
segment a first image area corresponding to the first contour and a second image area corresponding to the second contour; and
input the first image area into an autoencoder to obtain a first output, input the second image area into the autoencoder to obtain a second output, determine whether the product is defective according to the first output and the second output, and output a detection result of the product.
10. The electronic device of claim 9 , wherein the at least one processor is further caused to:
establish a coordinate system, an origin of the coordinate system being a lower left corner of the image when the image is placed in a forward direction, an X-axis of the coordinate system being a horizontal direction of the image, and a Y-axis of the coordinate system being a vertical direction of the image;
determine coordinates of four vertices of the first contour in the coordinate system, the coordinates of four vertices denoted as A 1 (x1,y1), A 2 (x2,y2), A 3 (x3,y3), and A 4 (x4,y4), wherein A 1 is a vertex closest to the X-axis, and the vertex A 1 , the vertex A 2 , the vertex A 3 , and the vertex A 4 are distributed in a counterclockwise order;
calculate coordinates of a center of the first contour according to the coordinates of the four vertices;
calculate an angle α between the first contour and the X-axis; and
correct the image with the coordinate of the center of the first contour as a rotation center and the angle α as a rotation angle.
11. The electronic device of claim 10 , wherein the coordinate of the center of the first contour is
M
(
x
1
-
(
x
1
-
x
3
)
2
,
y
3
-
(
y
3
-
y
1
)
2
)
.
12. The electronic device of claim 10 , wherein a formula of calculating the angle α between the first contour and the X-axis is:
α
=
arc
tan
(
y
2
-
y
1
x
2
-
x
1
)
.
13. The electronic device of claim 9 , wherein the first output indicates whether the first image area corresponds to a first defective region of the product, the second output indicates whether the second image area corresponds to a second defective region of the product, and the at least one processor is further caused to:
determine that the product is defective when the first image area corresponds to the first defective region or the second image area corresponds to the second defective region; and
determine that the product is flawless when neither the first image area corresponds to the first defective region nor the second image area corresponds to the second defective region.
14. The electronic device of claim 9 , wherein the at least one processor is further caused to:
perform vectorization processing on the first image area to obtain a first feature vector, and perform vectorization processing on the second image area to obtain a second feature vector;
input the first feature vector into the autoencoder to obtain a first reconstructed image, and input the second feature vector into the autoencoder to obtain a second reconstructed image;
calculate a first reconstruction error between the first image area and the first reconstructed image, and calculate a second reconstruction error between the second image area and the second reconstructed image; and
determine whether the product is defective according to the first reconstruction error and the second reconstruction error.
15. A non-transitory storage medium having stored thereon computer-readable instructions that, when the computer-readable instructions are executed by a processor to implement the following method:
obtaining an image of a product, and setting a region of interest of the image;
detecting a first contour of a first target object in the region of interest;
correcting the image according to the first contour to obtain a corrected image;
obtaining a position difference between the first contour and a second target object in the region of interest;
detecting a second contour of the second target object in the corrected image according to the position difference;
segmenting a first image area corresponding to the first contour and a second image area corresponding to the second contour; and
inputting the first image area into an autoencoder to obtain a first output, inputting the second image area into the autoencoder to obtain a second output, determining whether the product is defective according to the first output and the second output, and outputting a detection result of the product.
16. The non-transitory storage medium of claim 15 , wherein correcting the image according to the first contour comprises:
establishing a coordinate system, an origin of the coordinate system being a lower left corner of the image when the image is placed in a forward direction, an X-axis of the coordinate system being a horizontal direction of the image, and a Y-axis of the coordinate system being a vertical direction of the image;
determining coordinates of four vertices of the first contour in the coordinate system, the coordinates of the four vertices denoted as A 1 (x1,y1), A 2 (x2,y2), A 3 (x3,y3), and A 4 (x4,y4), wherein A 1 is a vertex closest to the X-axis, and the vertex A 1 , the vertex A 2 , the vertex A 3 , and the vertex A 4 are distributed in a counterclockwise order;
calculating coordinates of a center of the first contour according to the coordinates of the four vertices;
calculating an angle α between the first contour and the X-axis; and
correcting the image with the coordinate of the center of the first contour as a rotation center and the angle α as a rotation angle.
17. The non-transitory storage medium of claim 16 , wherein the coordinate of the center of the first contour is
M
(
x
1
-
(
x
1
-
x
3
)
2
,
y
3
-
(
y
3
-
y
1
)
2
)
.
18. The non-transitory storage medium of claim 16 , wherein a formula of calculating the angle α between the first contour and the X-axis is:
α
=
arc
tan
(
y
2
-
y
1
x
2
-
x
1
)
.
19. The non-transitory storage medium of claim 15 , wherein the first output indicates whether the first image area corresponds to a first defective region of the product, the second output indicates whether the second image area corresponds to a second defective region of the product, and determining whether the product is defective according to the first output and the second output comprises:
determining that the product is defective when the first image area corresponds to the first defective region or the second image area corresponds to the second defective region; and
determining that the product is flawless when neither the first image area corresponds to the first defective region nor the second image area corresponds to the second defective region.
20. The non-transitory storage medium of claim 15 , wherein inputting the first image area into an autoencoder to obtain a first output, inputting the second image area into the autoencoder to obtain a second output, determining whether the product is defective according to the first output and the second output comprises:
performing vectorization processing on the first image area to obtain a first feature vector, and performing vectorization processing on the second image area to obtain a second feature vector;
inputting the first feature vector into the autoencoder to obtain a first reconstructed image, and inputting the second feature vector into the autoencoder to obtain a second reconstructed image;
calculating a first reconstruction error between the first image area and the first reconstructed image, and calculating a second reconstruction error between the second image area and the second reconstructed image; and
determining whether the product is defective according to the first reconstruction error and the second reconstruction error.Cited by (0)
No later patents cite this yet.
References (0)
No backward citations on record.