Methods and apparatuses for object detection, and devices

Abstract

A method for object detection includes: obtaining a plurality of to-be-determined targets in a to-be-detected image; determining confidences of the plurality of to-be-determined targets separately belonging to at least one category, determining categories of the plurality of to-be-determined targets according to the confidences, and determining position offset values corresponding to the respective categories of the plurality of to-be-determined targets; using the position offset values corresponding to the respective categories of the plurality of to-be-determined targets as position offset values of the plurality of to-be-determined targets; and determining position information and a category of at least one to-be-determined target in the to-be-detected image according to the categories of the plurality of to-be-determined targets, the position offset values of the plurality of to-be-determined targets, and the confidences of the plurality of to-be-determined targets belonging to the categories thereof.

What is claimed is: 
     
         1 . A method for object detection, comprising:
 obtaining a plurality of to-be-determined targets in a to-be-detected image;   determining, for at least one category, confidences of a plurality of to-be-determined targets respectively;   determining categories of the plurality of to-be-determined targets according to the confidences;   respectively determining position offset values corresponding to the categories of the plurality of to-be-determined targets;   respectively using the position offset values corresponding to the categories of the plurality of to-be-determined targets as position offset values of the plurality of to-be-determined targets; and   determining a category and position information of at least one to-be-determined target in the to-be-detected image according to the categories of the plurality of to-be-determined targets, the position offset values of the plurality of to-be-determined targets, and confidences of the categories of the plurality of to-be-determined targets,   wherein the operation of obtaining a plurality of to-be-determined targets in a to-be-detected image comprises:   obtaining the plurality of to-be-determined targets formed based on at least one predetermined region size in the to-be-detected image,   wherein the operation of obtaining the plurality of to-be-determined targets formed based on at least one predetermined region size in the to-be-detected image comprises:   obtaining a feature map of the to-be-detected image;   pooling the feature map based on reference box configuration information to obtain a plurality of new feature maps; and   using the plurality of new feature maps as the plurality of to-be-determined targets.   
     
     
         2 . The method according to  claim 1 , wherein the obtaining the plurality of to-be-determined targets formed based on at least one predetermined region size in the to-be-detected image comprises:
 obtaining a feature map of the to-be-detected image;   forming a reference box of a feature point in the feature map according to reference box configuration information;   using the reference box of the feature point in the feature map as one to-be-determined target; and   obtaining, respectively corresponding to a plurality of feature points in the feature map, the plurality of to-be-determined targets.   
     
     
         3 . The method according to  claim 2 , wherein the obtaining a feature map of the to-be-detected image comprises:
 inputting the to-be-detected image into a backbone network in a convolutional neural network;   inputting a feature map output by the backbone network into a filter layer in the convolutional neural network;   filtering the feature map output by the backbone network by the filter layer according to a preset sliding window, and   using the filtered feature map output by the backbone network as the feature map of the to-be-detected image.   
     
     
         4 . The method according to  claim 1 , wherein the predetermined region size comprises: nine predetermined region sizes formed based on three different lengths and three different aspect ratios; or nine predetermined region sizes formed based on three different widths and three different aspect ratios; or nine predetermined region sizes formed based on three different lengths and widths. 
     
     
         5 . The method according to  claim 1 , wherein the category comprises: two object categories and one background category. 
     
     
         6 . The method according to  claim 1 , wherein the determining, for at least one category, confidences of a plurality of to-be-determined targets respectively, and determining categories of the plurality of to-be-determined targets according to the confidences comprises:
 for each of the plurality of to-be-determined target, calculating, for the at least one category, a confidence of the to-be-determined target respectively, and using a category corresponding to a highest confidence as a category of the to-be-determined target.   
     
     
         7 . The method according to  claim 1 , wherein the determining position offset values corresponding to the respective categories of the plurality of to-be-determined targets comprises:
 for each of the plurality of to-be-determined target, calculating, for a category of the to-be-determined target, a top offset value, a bottom offset value, a left offset value, and a right offset value of the to-be-determined target.   
     
     
         8 . The method according to  claim 1 , wherein the position information of at least one to-be-determined target comprises: position information of a bounding box of the at least one to-be-determined target. 
     
     
         9 . The method according to  claim 8 , wherein the determining a category and position information of at least one to-be-determined target in the to-be-detected image according to the categories of the plurality of to-be-determined targets, the position offset values of the plurality of to-be-determined targets, and confidences of the categories of the plurality of to-be-determined targets comprises:
 selecting, from the plurality of to-be-determined targets, at least one to-be-determined target with confidences meeting a predetermined confidence requirement;   forming the position information of the bounding box of the at least one to-be-determined target in the to-be-detected image according to position offset value of the selected at least one to-be-determined target; and   respectively using a category of the selected at least one to-be-determined target as a category of the bounding box of the at least one to-be-determined target in the to-be-detected image.   
     
     
         10 . The method according to  claim 1 , wherein the determining, for at least one category, confidences of a plurality of to-be-determined targets respectively, determining categories of the plurality of to-be-determined targets according to the confidences, respectively determining position offset values corresponding to the categories of the plurality of to-be-determined targets comprises:
 using a convolutional neural network to determine, for at least one category, confidences of the plurality of to-be-determined targets respectively, determine categories of the plurality of to-be-determined targets according to the confidences, and respectively determine position offset values corresponding to the categories of the plurality of to-be-determined targets; and   the method further comprises: training the convolutional neural network, wherein the operation of training the convolutional neural network comprises:   obtaining, from an image sample set, an image sample annotated with information of at least one standard position and category of the at least one standard position;   obtaining a plurality of to-be-determined targets in the image sample;   determining, for at least one category, confidences of the plurality of to-be-determined targets separately by one convolutional layer in the convolutional neural network;   determining categories of the plurality of to-be-determined targets according to the confidences;   respectively determining, by another convolutional layer in the convolutional neural network, position offset values corresponding to the categories of the plurality of to-be-determined targets;   respectively using the position offset values corresponding to the categories of the plurality of to-be-determined targets as position offset values of the plurality of to-be-determined targets;   calculating standard position offset values of the plurality of to-be-determined targets with respect to the corresponding standard position;   calculating a deviation between a position offset value of at least one of the plurality of to-be-determined targets with respect to a category of the corresponding standard position and the corresponding standard position offset value by one loss layer in the convolutional neural network, and calculating a deviation between a confidence of at least one of the plurality of to-be-determined targets with respect to the category of the corresponding standard position and the category of the standard position by one loss layer in the convolutional neural network;   back-propagating the deviations in the convolutional neural network; and   completing the training of the convolutional neural network until a number of image samples obtained from the image sample set reaches a predetermined number and the deviations are within a predetermined range.   
     
     
         11 . The method according to  claim 10 , wherein the obtaining a plurality of to-be-determined targets in the image sample comprises:
 obtaining a plurality of to-be-determined targets formed based on at least one predetermined region size in the image sample.   
     
     
         12 . The method according to  claim 10 , wherein the standard position comprises: a standard bounding box. 
     
     
         13 . The method according to  claim 12 , wherein the calculating standard position offset values of the plurality of to-be-determined targets with respect to the corresponding standard position comprises:
 determining standard bounding boxes having largest overlap areas respectively corresponding to the plurality of to-be-determined targets; and   respectively calculating standard position offset values of the plurality of to-be-determined targets with respect to the standard bounding boxes having the largest overlap areas.   
     
     
         14 . The method according to  claim 12 , wherein the at least one of the plurality of to-be-determined targets comprises at least one of: at least one positive to-be-determined target selected from a plurality of positive to-be-determined targets in the plurality of to-be-determined targets, or at least one negative to-be-determined target selected from a plurality of negative to-be-determined targets in the plurality of to-be-determined targets, wherein the selected positive to-be-determined target and the selected negative to-be-determined target meet a predetermined ratio; and
 for one to-be-determined target, if a ratio of an overlap area between the to-be-determined target and a standard bounding box of the to-be-determined target having a largest overlap area to an area covered by both the to-be-determined target and the standard bounding box is greater than a first ratio threshold, the to-be-determined target is a positive to-be-determined target; and if the ratio is smaller than a second ratio threshold, the to-be-determined target is a negative to-be-determined target.   
     
     
         15 . An electronic apparatus, comprising:
 a processor; and   a memory for storing instructions executable by the processor;   wherein execution of the instructions by the processor causes the processor to perform:   obtaining a plurality of to-be-determined targets in a to-be-detected image;   determining, for at least one category, confidences of a plurality of to-be-determined targets respectively;   determining categories of the plurality of to-be-determined targets according to the confidences;   respectively determining position offset values corresponding to the categories of the plurality of to-be-determined targets;   respectively using the position offset values corresponding to the categories of the plurality of to-be-determined targets as position offset values of the plurality of to-be-determined targets; and   determining a category and position information of at least one to-be-determined target in the to-be-detected image according to the categories of the plurality of to-be-determined targets, the position offset values of the plurality of to-be-determined targets, and confidences of the categories of the plurality of to-be-determined targets,   wherein the operation of obtaining a plurality of to-be-determined targets in a to-be-detected image comprises: obtaining the plurality of to-be-determined targets formed based on at least one predetermined region size in the to-be-detected image.   wherein the operation of obtaining the plurality of to-be-determined targets formed based on at least one predetermined region size in the to-be-detected image comprises:   obtaining a feature map of the to-be-detected image;   pooling the feature map based on reference box configuration information to obtain a plurality of new feature maps; and   using the plurality of new feature maps as the plurality of to-be-determined targets.   
     
     
         16 . The apparatus according to  claim 15 , wherein the obtaining the plurality of to-be-determined targets formed based on at least one predetermined region size in the to-be-detected image comprises:
 obtaining a feature map of the to-be-detected image;   forming a reference box of a feature point in the feature map according to reference box configuration information;   using the reference box of the feature point in the feature map as one to-be-determined target; and   obtaining respectively corresponding to a plurality of feature points in the feature map, the plurality of to-be-determined targets.   
     
     
         17 . The apparatus according to  claim 16 , wherein the operation of obtaining a feature map of the to-be-detected image comprises:
 inputting the to-be-detected image into a backbone network in a convolutional neural network;   inputting a feature map output by the backbone network into a filter layer in the convolutional neural network;   filtering the feature map output by the backbone network by the filter layer according to a preset sliding window, and   using the filtered feature map output by the backbone network as the feature map of the to-be-detected image.   
     
     
         18 . The apparatus according to  claim 15 , wherein the predetermined region size comprises: nine predetermined region sizes formed based on three different lengths and three different aspect ratios; or nine predetermined region sizes formed based on three different widths and three different aspect ratios; or nine predetermined region sizes formed based on three different lengths and widths. 
     
     
         19 . The apparatus according to  claim 15 , wherein the category comprises: two object categories and one background category. 
     
     
         20 . A non-transitory computer-readable storage medium having a computer program stored thereon, wherein execution of the instructions by the processor causes the processor to perform:
 obtaining a plurality of to-be-determined targets in a to-be-detected image;   determining, for at least one category, confidences of a plurality of to-be-determined targets respectively;   determining categories of the plurality of to-be-determined targets according to the confidences;   respectively determining position offset values corresponding to the categories of the plurality of to-be-determined targets;   respectively using the position offset values corresponding to the categories of the plurality of to-be-determined targets as position offset values of the plurality of to-be-determined targets; and   determining a category and position information of at least one to-be-determined target in the to-be-detected image according to the categories of the plurality of to-be-determined targets, the position offset values of the plurality of to-be-determined targets, and confidences of the categories of the plurality of to-be-determined targets,   wherein the operation of obtaining a plurality of to-be-determined targets in a to-be-detected image comprises:   obtaining the plurality of to-be-determined targets formed based on at least one predetermined region size in the to-be-detected image,   wherein the operation of obtaining the plurality of to-be-determined targets formed based on at least one predetermined region size in the to-be-detected image comprises:   obtaining a feature map of the to-be-detected image;   pooling the feature map based on reference box configuration information to obtain a plurality of new feature maps; and   
       using the plurality of new feature maps as the plurality of to-be-determined targets.