Universal correspondence network

Abstract

A computer-implemented method for training a convolutional neural network (CNN) is presented. The method includes extracting coordinates of corresponding points in the first and second locations, identifying positive points in the first and second locations, identifying negative points in the first and second locations, training features that correspond to positive points of the first and second locations to move closer to each other, and training features that correspond to negative points in the first and second locations to move away from each other.

What is claimed is: 
     
       1. A computer-implemented method for training a convolutional neural network (CNN), the method comprising:
 extracting coordinates of corresponding points in first and second locations; 
 identifying positive points in the first and second locations; 
 identifying negative points in the first and second locations; 
 training features that correspond to positive points of the first and second locations to move closer to each other; and 
 training features that correspond to negative points in the first and second locations to move away from each other; 
 wherein a contrastive loss layer is trained with hard negative mining and by reusing activations in overlapping regions. 
 
     
     
       2. The method of  claim 1 , wherein the CNN has a fully convolutional spatial transformer for normalizing patches to handle rotation and scaling. 
     
     
       3. The method of  claim 2 , wherein the convolutional spatial transformer applies spatial transformations to lower layer activations. 
     
     
       4. The method of  claim 1 , wherein a contrastive loss layer encodes distances between the features of the first and second locations. 
     
     
       5. The method of  claim 1 , wherein hard negative pairs are mined that violate constraints. 
     
     
       6. A system for training a convolutional neural network (CNN), the system comprising:
 a memory; and 
 a processor in communication with the memory, wherein the processor is configured to:
 extract coordinates of corresponding points in the first and second locations; 
 identify positive points in the first and second locations; 
 identify negative points in the first and second locations; 
 train features that correspond to positive points of the first and second locations to move closer to each other; and 
 train features that correspond to negative points in the first and second locations to move away from each other; 
 wherein a contrastive loss layer is trained with hard negative mining and by reusing activations in overlapping re ions. 
 
 
     
     
       7. The system of  claim 6 , wherein the CNN has a fully convolutional spatial transformer for normalizing patches to handle rotation and scaling. 
     
     
       8. The system of  claim 7 , wherein the convolutional spatial transformer applies spatial transformations to lower layer activations. 
     
     
       9. The system of  claim 6 , wherein a contrastive loss layer encodes distances between the features of the first and second locations. 
     
     
       10. The system of  claim 6 , wherein hard negative pairs are mined that violate constraints. 
     
     
       11. A non-transitory computer-readable storage medium comprising a computer-readable program for training a convolutional neural network (CNN), wherein the computer-readable program when executed on a computer causes the computer to perform the steps of:
 extracting coordinates of corresponding points in the first and second locations; 
 identifying positive points in the first and second locations; 
 identifying negative points in the first and second locations; 
 training features that correspond to positive points of the first and second locations to move closer to each other; and 
 training features that correspond to negative points in the first and second locations to move away from each other; 
 wherein a contrastive loss layer is trained with hard negative mining and by reusing activations in overlapping regions. 
 
     
     
       12. The non-transitory computer-readable storage medium of  claim 11 , wherein the CNN has a fully convolutional spatial transformer for normalizing patches to handle rotation and scaling. 
     
     
       13. The non-transitory computer-readable storage medium of  claim 11 , wherein the convolutional spatial transformer applies spatial transformations to lower layer activations. 
     
     
       14. The non-transitory computer-readable storage medium of  claim 11 , wherein a contrastive loss layer encodes distances between the features of the first and second locations. 
     
     
       15. The non-transitory computer-readable storage medium of  claim 11 , wherein hard negative pairs are mined that violate constraints.