Convolutional neural network based system and method for detecting a target object

Abstract

A method for detecting a target object using a convolutional neural network includes: receiving an image frame; producing a first feature map of the image frame; producing a residual feature map from the first feature map by: applying a convolution of a first scale, and thereafter a depth-wise separable convolution, to the first feature map, thereby producing a second feature map; and adding the first feature map and the second feature map to produce an added feature map; producing at least one extracted feature map of the image frame, from the residual feature map by applying additional convolution of a second scale different from the first scale, and thereafter additional depth-wise separable convolution, to the residual feature map; and determining a box using the at least one extracted feature map. The determined box frames the target object.

1 . A method for detecting a target object using a convolutional neural network, comprising:
 receiving an image frame;   producing a first feature map of the image frame;   producing a residual feature map from the first feature map by:
 applying a convolution of a first scale, and thereafter a depth-wise separable convolution, to the first feature map, thereby producing a second feature map; and 
 adding the first feature map and the second feature map to produce an added feature map; 
   producing at least one extracted feature map of the image frame, from the residual feature map by applying additional convolution of a second scale different from the first scale, and thereafter additional depth-wise separable convolution, to the residual feature map; and   determining a box using the at least one extracted feature map, wherein the determined box is configured to frame the target object.   
     
     
         2 . The method of  claim 1 , wherein producing the first feature map of the image frame comprises: applying a convolution of the second scale, and thereafter at least a depth-wise separable convolution, to the image frame. 
     
     
         3 . The method of  claim 1 , further comprising applying a depth-wise separable convolution with a scale of 3*3 and a stride of 2, and a subsequent 1*1 convolution with a linear activation layer, to the added feature map, to produce the residual feature map. 
     
     
         4 . The method of  claim 1 , wherein applying additional convolution and thereafter additional depth-wise separable convolution to the residual feature map comprises:
 feeding the residual feature map as an input into a first convolution branch and a second convolution branch;   producing a first branch feature map by the first convolution branch using the residual feature map; and   producing a second branch feature map by the second convolution branch using the residual feature map.   
     
     
         5 . The method of  claim 4 , wherein the first convolution branch comprises:
 a convolution layer with a scale of 3*3 and a stride of 2;   at least one convolution block comprising a convolution layer with a scale of 1*1 and a depth-wise separable convolution layer with a scale of 3*3 and a stride of 1;   a depth-wise separable convolution layer with a scale of 5*5;   a depth-wise separable convolution layer with a scale of 3*3; and   a convolution layer with a scale of 1*1 and a stride of 1.   
     
     
         6 . The method of  claim 5 , wherein the first convolution branch comprises two convolution blocks. 
     
     
         7 . The method of  claim 4 , wherein the second convolution branch comprises:
 a convolution layer with a scale of 3*3 and a stride of 1;   at least one convolution block comprising a convolution layer with a scale of 1*1 and a depth-wise separable convolution layer with a scale of 3*3 and a stride of 1;   a depth-wise separable convolution layer with a scale of 5*5;   a depth-wise separable convolution layer with a scale of 3*3; and   a convolution layer with a scale of 1*1 and a stride of 1.   
     
     
         8 . The method of  claim 7 , wherein the second convolution branch comprises two convolution blocks. 
     
     
         9 . The method of  claim 1 , wherein determining the box using the extracted feature map comprises:
 determining candidate boxes using the extracted feature map, wherein each of the candidate boxes is corresponding to a confidence value higher than a detection threshold; and   determining a final box from the candidate boxes through Non-Maximum Suppression.   
     
     
         10 . The method of  claim 9 , wherein the extracted feature map comprises a first channel of confidence value, a second channel of horizontal coordinates of central location prediction, a third channel of vertical coordinates of central location prediction, a fourth channel of width prediction, and a fifth channel of height prediction; and wherein determining the candidate boxes comprises:
 determining a confidence value of the first channel of the extracted feature map is higher than the detection threshold;   determining the candidate box as being positioned at a central location with a corresponding horizontal coordinate of the second channel and a corresponding vertical coordinate of the third channel; and   determining the candidate box as having a width of the corresponding width of the fourth channel and a height of the corresponding height of the fifth channel.   
     
     
         11 . A convolutional neural network based system configured to detect a target object comprising:
 a pre-processing unit configured to receive an image frame, and produce pre-processed image data using the received image frame;   a detection unit configured to receive the pre-processed image data, and produce at least one extracted feature map of the image frame; and   a post-processing unit configured to receive the at least one extracted feature map, and determine a box using the at least one extracted feature map, wherein the determined box is configured to frame the target object; wherein   the detection unit comprises:
 an initial extraction block configured to produce a first feature map of the image frame; 
 a residual block configured to produce a residual feature map from the first feature map by: applying a convolution of a first scale, and thereafter a depth-wise separable convolution, to the first feature map, to produce a second feature map, and adding the first feature map and the second feature map to produce an added feature map; and 
 an extraction output block configured to apply additional convolution of a second scale different from the first scale, and thereafter additional depth-wise separable convolution, to the residual feature map, thereby producing the extracted feature map of the image frame. 
   
     
     
         12 . The system of  claim 11 , wherein the initial extraction block is configured to apply a convolution of the second scale, and thereafter at least a depth-wise separable convolution, to the image frame, to produce the first feature map of the image frame. 
     
     
         13 . The system of  claim 11 , wherein the residual block is further configured to apply a depth-wise separable convolution with a scale of 3*3 and a stride of 2, and a subsequent 1*1 convolution with a linear activation layer, to the added feature map, to produce the residual feature map. 
     
     
         14 . The system of  claim 11 , wherein
 the extraction output block comprises a first convolution branch and a second convolution branch; and wherein   the extraction output block is configured to apply the additional convolution and thereafter the additional depth-wise separable convolution to the residual feature map, after the residual block feeding the residual feature map as an input into the first convolution branch and the second convolution branch;   wherein the first convolution branch is configured to produce a first branch feature map using the residual feature map; and   wherein the second convolution branch is configured to produce a second branch feature map using the residual feature map.   
     
     
         15 . The system of  claim 14 , wherein the first convolution branch comprises:
 a convolution layer with a scale of 3*3 and a stride of 2;   at least one convolution block comprising a convolution layer with a scale of 1*1 and a depth-wise separable convolution layer with a scale of 3*3 and a stride of 1;   a depth-wise separable convolution layer with a scale of 5*5;   a depth-wise separable convolution layer with a scale of 3*3; and   a convolution layer with a scale of 1*1 and a stride of 1.   
     
     
         16 . The system of  claim 15 , wherein the first convolution branch comprises two convolution blocks. 
     
     
         17 . The system of  claim 14 , wherein the second convolution branch comprises:
 a convolution layer with a scale of 3*3 and a stride of 1;   at least one convolution block comprising a convolution layer with a scale of 1*1 and a depth-wise separable convolution layer with a scale of 3*3 and a stride of 1;   a depth-wise separable convolution layer with a scale of 5*5;   a depth-wise separable convolution layer with a scale of 3*3; and   a convolution layer with a scale of 1*1 and a stride of 1.   
     
     
         18 . The system of  claim 17 , wherein the second convolution branch comprises two convolution blocks. 
     
     
         19 . The system of  claim 11 , wherein
 the post-processing unit comprises a box extraction unit and a box output unit, and wherein   the box extraction unit is configured to determine candidate boxes using the extracted feature map, wherein each of the candidate boxes is corresponding to a confidence value higher than a detection threshold; and   the box output unit is configured to determine a final box from the candidate boxes through Non-Maximum Suppression.   
     
     
         20 . The system of  claim 19 , wherein the extracted feature map comprises a first channel of confidence value, a second channel of horizontal coordinates of central location prediction, a third channel of vertical coordinates of central location prediction, a fourth channel of width prediction, and a fifth channel of height prediction; and wherein the box extraction unit is configured to:
 determine a confidence value of the first channel of the extracted feature map is higher than the detection threshold;   determine the candidate box as being positioned at a central location with a corresponding horizontal coordinate of the second channel and a corresponding vertical coordinate of the third channel; and   determine the candidate box as having a width of the corresponding width of the fourth channel and a height of the corresponding height of the fifth channel.