On the fly adaptive convolutional neural network for variable computational budget

Abstract

Techniques related to object detection using an adaptive convolutional neural network (CNN) are discussed. Such techniques include applying one of multiple configurations of the CNN to input image data in response to an available computational resources for processing the input image data.

What is claimed is: 
     
       1. A system for performing object recognition comprising:
 a memory to store first and second input image data; and 
 a processor coupled to the memory, the processor to:
 apply, in response to a first computational resource level available for processing the first input image data, only a first subset of available convolutional filters at a convolutional layer of a convolutional neural network (CNN) to first feature maps corresponding to the first input image data; 
 apply, in response to a change to a second computational resource level greater than the first computational resource level available for processing the second input image data, the first subset and one or more additional convolutional filters of the available convolutional filters at the convolutional layer of the CNN to second feature maps corresponding to the second input image data; and 
 transmit first and second object recognition indicators for the first and second images, respectively. 
 
 
     
     
       2. The system of  claim 1 , wherein the processor to apply the first subset of available convolutional filters to the first feature maps and to apply the first subset of available convolutional filters to the second feature maps comprises the processor to apply the same pretrained filter weights. 
     
     
       3. The system of  claim 1 , wherein the processor to apply the first subset of convolutional filters generates a number of third feature maps equal to the number of convolutional filters in the first subset of convolutional filters and the processor to apply the first subset and the one or more additional convolutional filters generates a number of fourth feature maps equal to the number of third feature maps plus the number of additional convolutional filters. 
     
     
       4. The system of  claim 3 , wherein the third feature maps and the fourth feature maps are all of the same resolution. 
     
     
       5. The system of  claim 1 , wherein the processor is further to:
 apply, in response to the first computational resource level, only a second subset of second available convolutional filters at a second convolutional layer of the CNN to the first input image data; and 
 apply, in response to the second computational resource level, the second subset and one or more additional second convolutional filters of the second available convolutional filters at the second convolutional layer of the CNN to the second input image data. 
 
     
     
       6. The system of  claim 5 , wherein a ratio of a number of convolutional filters in the first subset to a number of convolutional filters in the first subset plus the one or more additional convolutional filters is the same as a second ratio of a number of convolutional filters in the second subset to a number of convolutional filters in the second subset plus the one or more additional second convolutional filters. 
     
     
       7. The system of  claim 1 , wherein the processor is further to:
 apply, in response to the first computational resource level, only a second subset of available fully connected weights at a second layer of the CNN to third feature maps corresponding to the first input image data; and 
 apply, in response to the second computational resource level, the second subset and one or more additional fully connected weights of the second available fully connected weights at the second layer of the CNN to fourth feature maps corresponding to the second input image data. 
 
     
     
       8. The system of  claim 1 , wherein the processor is further to:
 apply, in response to a change to a third computational resource level less than the first computational resource level available for processing third input image data, only a second subset of the available convolutional filters at the convolutional layer of the CNN to third feature maps corresponding to the third input image, wherein the second subset has fewer convolutional filters than the first subset and each convolutional filter of the second subset is in the first subset. 
 
     
     
       9. The system of  claim 1 , wherein the processor is further to:
 select a number of available configurations for the CNN; 
 determine a number of convolutional filters for application at each convolutional layer for each of the available configurations; and 
 train, each of the available configurations in conjunction with one another, wherein common convolutional filters of the available configurations share filter weights in the training, to generate finalized weights for the CNN. 
 
     
     
       10. The system of  claim 9 , wherein the processor to train each of the available configurations comprises, at each of a plurality of training iterations, the processor to:
 perform a forward propagation and a backward propagation for a single randomly or iteratively chosen configuration or a number of forward and backward propagations equal to the number of available configurations to determine batch gradient descents; and 
 update convolutional filter weights using the batch gradient descents. 
 
     
     
       11. The system of  claim 10 , wherein the processor to train each of the available configurations, at each of a plurality of training iterations, comprises a layer-wise dropout of convolutional filters at each convolutional layer to train the full CNN and, subsequently, reduced CNN configurations. 
     
     
       12. A computer-implemented method for performing object recognition, comprising:
 applying, in response to a first computational resource level available for processing first input image data, only a first subset of available convolutional filters at a convolutional layer of a convolutional neural network (CNN) to first feature maps corresponding to the first input image data; 
 applying, in response to a change to a second computational resource level greater than the first computational resource level available for processing second input image data, the first subset and one or more additional convolutional filters of the available convolutional filters at the convolutional layer of the CNN to second feature maps corresponding to the second input image data; and 
 transmitting first and second object recognition indicators for the first and second images, respectively. 
 
     
     
       13. The method of  claim 12 , further comprising:
 applying, in response to the first computational resource level, only a second subset of second available convolutional filters at a second convolutional layer of the CNN to the first input image data; and 
 applying, in response to the second computational resource level, the second subset and one or more additional second convolutional filters of the second available convolutional filters at the second convolutional layer of the CNN to the second input image data. 
 
     
     
       14. The method of  claim 13 , wherein a ratio of a number of convolutional filters in the first subset to a number of convolutional filters in the first subset plus the one or more additional convolutional filters is the same as a second ratio of a number of convolutional filters in the second subset to a number of convolutional filters in the second subset plus the one or more additional second convolutional filters. 
     
     
       15. The method of  claim 12 , further comprising:
 applying, in response to the first computational resource level, only a second subset of available fully connected weights at a second layer of the CNN to third feature maps corresponding to the first input image data; and 
 applying, in response to the second computational resource level, the second subset and one or more additional fully connected weights of the second available fully connected weights at the second layer of the CNN to fourth feature maps corresponding to the second input image data. 
 
     
     
       16. The method of  claim 12 , further comprising:
 applying, in response to a change to a third computational resource level less than the first computational resource level available for processing third input image data, only a second subset of available convolutional filters at the convolutional layer of the CNN to third feature maps corresponding to the third input image, wherein the second subset has fewer convolutional filters than the first subset and each convolutional filter of the second subset is in the first subset. 
 
     
     
       17. The method of  claim 16 , wherein a number of convolutional filters in the first subset is not less than twice a number of convolutional filters in the second subset and a number of convolutional filters in the first subset and the one or more additional convolutional filters is not less than twice the number of convolutional filters in the first subset. 
     
     
       18. At least one non-transitory machine readable medium comprising a plurality of instructions that, in response to being executed on a computing device, cause the computing device to perform object detection by:
 applying, in response to a first computational resource level available for processing first input image data, only a first subset of available convolutional filters at a convolutional layer of a convolutional neural network (CNN) to first feature maps corresponding to the first input image data; 
 applying, in response to a change to a second computational resource level greater than the first computational resource level available for processing second input image data, the first subset and one or more additional convolutional filters of the available convolutional filters at the convolutional layer of the CNN to second feature maps corresponding to the second input image data; and 
 transmitting first and second object recognition indicators for the first and second images, respectively. 
 
     
     
       19. The non-transitory machine readable medium of  claim 18 , the machine readable medium further comprising instructions that, in response to being executed on the device, cause the device to perform object detection by:
 applying, in response to the first computational resource level, only a second subset of second available convolutional filters at a second convolutional layer of the CNN to the first input image data; and 
 applying, in response to the second computational resource level, the second subset and one or more additional second convolutional filters of the second available convolutional filters at the second convolutional layer of the CNN to the second input image data. 
 
     
     
       20. The non-transitory machine readable medium of  claim 19 , wherein a ratio of a number of convolutional filters in the first subset to a number of convolutional filters in the first subset plus the one or more additional convolutional filters is the same as a second ratio of a number of convolutional filters in the second subset to a number of convolutional filters in the second subset plus the one or more additional second convolutional filters. 
     
     
       21. The non-transitory machine readable medium of  claim 18 , the machine readable medium further comprising instructions that, in response to being executed on the device, cause the device to perform object detection by:
 applying, in response to the first computational resource level, only a second subset of available fully connected weights at a second layer of the CNN to third feature maps corresponding to the first input image data; and 
 applying, in response to the second computational resource level, the second subset and one or more additional fully connected weights of the second available fully connected weights at the second layer of the CNN to fourth feature maps corresponding to the second input image data. 
 
     
     
       22. The non-transitory machine readable medium of  claim 18 , the machine readable medium further comprising instructions that, in response to being executed on the device, cause the device to perform object detection by:
 applying, in response to a change to a third computational resource level less than the first computational resource level available for processing third input image data, only a second subset of available convolutional filters at the convolutional layer of the CNN to third feature maps corresponding to the third input image, wherein the second subset has fewer convolutional filters than the first subset and each convolutional filter of the second subset is in the first subset. 
 
     
     
       23. The non-transitory machine readable medium of  claim 22 , wherein a number of convolutional filters in the first subset is not less than twice a number of convolutional filters in the second subset and a number of convolutional filters in the first subset and the one or more additional convolutional filters is not less than twice the number of convolutional filters in the first subset.