Generating object proposals using deep-learning models

Abstract

In one embodiment, a plurality of patches of an image are processed using a first-pass of a first deep-learning model to generate object-level information for each of the patches. Each patch includes one or more pixels of the image. Using a second-pass of the first deep-learning model, a respective object proposal is generated for each of the plurality of patches of the image. The second-pass takes as input the first-pass output, and the generated respective object proposals comprise pixel-level information for each of the patches. Using a second deep-learning model, a respective score is computed for each object proposal. The second deep-learning model takes as input the first-pass output, and the object score includes a likelihood that the respective patch of the object proposal contains an entire object.

What is claimed is: 
     
       1. A method comprising, by one or more computing devices:
 accessing an image, wherein the image comprises a plurality of patches, wherein each patch comprises one or more pixels; 
 generating, using a first deep-learning model, a first-pass output comprising object-level information for each patch of the image in a first-pass, wherein the first-pass of the first deep-learning model takes as input the plurality of patches of the image; 
 generating, using the first deep-learning model, a respective object proposal comprising pixel-level information for each patch of the image in a second-pass, wherein the second-pass of the first deep-learning model takes as input the first-pass output; and 
 computing, using a second deep-learning model, a respective score for each object proposal generated using the first deep-learning model, wherein the second deep-learning model takes as input the first-pass output, and wherein the score comprises a likelihood that the patch of the respective object proposal contains an entire object. 
 
     
     
       2. The method of  claim 1 , wherein the first-pass comprises a number N of layers. 
     
     
       3. The method of  claim 2 , wherein:
 a first layer of the first-pass takes as input the plurality of patches; and 
 each remaining layer of the N layers of the first-pass takes as input a respective output of a previous layer of the N layers of the first-pass. 
 
     
     
       4. The method of  claim 3 , wherein the second-pass comprises the number N of layers. 
     
     
       5. The method of  claim 4 , wherein:
 a first layer of the second-pass takes as input the first-pass output; and 
 each remaining layer of the N layers of the second-pass takes as input a respective output of a previous layer of the N layers of the second-pass and output from a corresponding layer of the N layers of the first-pass. 
 
     
     
       6. The method of  claim 1 , wherein the respective object proposal of each patch comprises a prediction of a location of an object in the respective patch. 
     
     
       7. The method of  claim 1 , wherein the respective score of each object proposal comprises a likelihood that the respective patch of each object proposal contains an entire object. 
     
     
       8. The method of  claim 7 , wherein the respective score of each object proposal further comprises a likelihood that the entire object is centered in the respective patch. 
     
     
       9. The method of  claim 1 , wherein the image is associated with a privacy setting indicating a permission for generation of object proposals of the image. 
     
     
       10. The method of  claim 1 , wherein a respective object proposal of a patch is represented as a shape overlaying an object in the image. 
     
     
       11. The method of  claim 1 , wherein:
 the first-pass comprises forward-pass layers; and 
 the second-pass comprises backward-pass layers. 
 
     
     
       12. The method of  claim 1 , further comprising:
 generating a sliding window having a fixed size; and 
 shifting the sliding window over the image to generate the plurality of patches. 
 
     
     
       13. The method of  claim 12 , wherein the generated plurality of patches comprises a first set of overlapping patches. 
     
     
       14. The method of  claim 13 , further comprising:
 resizing the image; and 
 shifting the sliding window over the resized image to create a new set of overlapping patches. 
 
     
     
       15. The method of  claim 14 , further comprising:
 iteratively resizing the image and shifting the sliding window over each resized image until at least one patch contains an entire object. 
 
     
     
       16. The method of  claim 1 , further comprising:
 generating an identification of an object in the image based on at least one respective object proposal of a patch of the plurality of patches. 
 
     
     
       17. The method of  claim 1 , further comprising:
 ranking the plurality of object proposals based on their respective scores; and 
 determining a subset of object proposals of the plurality of object proposals based on the ranking. 
 
     
     
       18. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:
 access an image, wherein the image comprises a plurality of patches, wherein each patch comprises one or more pixels; 
 generate, using a first deep-learning model, a first-pass output comprising object-level information for each patch of the image in a first-pass, wherein the first-pass of the first deep-learning model takes as input the plurality of patches of the image; 
 generate, using the first deep-learning model, a respective object proposal comprising pixel-level information for each patch of the image in a second-pass, wherein the second-pass of the first deep-learning model takes as input the first-pass output; and 
 compute, using a second deep-learning model, a respective score for each object proposal generated using the first deep-learning model, wherein the second deep-learning model takes as input the first-pass output, and wherein the score comprises a likelihood that the patch of the respective object proposal contains an entire object. 
 
     
     
       19. A system comprising one or more processors and a memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to:
 access an image, wherein the image comprises a plurality of patches, wherein each patch comprises one or more pixels; 
 generate, using a first deep-learning model, a first-pass output comprising object-level information for each patch of the image in a first-pass, wherein the first-pass of the first deep-learning model takes as input the plurality of patches of the image; 
 generate, using the first deep-learning model, a respective object proposal comprising pixel-level information for each patch of the image in a second-pass, wherein the second-pass of the first deep-learning model takes as input the first-pass output; and 
 compute, using a second deep-learning model, a respective score for each object proposal generated using the first deep-learning model, wherein the second deep-learning model takes as input the first-pass output, and wherein the score comprises a likelihood that the patch of the respective object proposal contains an entire object.