Process for updating templates used in facial recognition

Abstract

Templates used for a facial recognition process for authentication of a user to use a device may be updated by the device as features of the user change over time. Features of the user may gradually change over time due to changes such as facial hair changes, haircuts, gaining/losing weight, and/or aging. Updating the templates used for the facial recognition process may allow the user to continue being authenticated as features of the user change without the need for additional enrollments of the user.

What is claimed is: 
     
       1. A method, comprising:
 obtaining a first image of a face of a user using a camera located on a device in response to an authentication attempt initiated by the user, the device comprising a computer processor and a memory; 
 encoding the first image to generate at least one first feature vector, wherein the first feature vector represents one or more facial features of the user in the first image; 
 comparing the first feature vector to one or more reference templates stored in a first space in the memory of the device to obtain a first matching score; 
 authorizing the user to perform at least one operation on the device that requires authentication in response to the first matching score being above an unlock threshold for the device; 
 storing the first feature vector as a temporary template in a second space in the memory of the device in response to the first matching score being above a first threshold; 
 obtaining a plurality of additional images of the face of the user using the camera, wherein the additional images are obtained in response to additional authentication attempts initiated by the user; 
 subsequent to each additional authentication attempt initiated by the user:
 encoding at least one additional image for the additional authentication attempt to generate at least one additional feature vector, wherein the additional feature vector represents one or more facial features of the user in the at least one additional image; 
 comparing the additional feature vector to the temporary template stored in the second space to obtain an additional matching score; and 
 assessing the additional authentication attempt as being successful in response to the additional matching score for the additional authentication attempt being above a second threshold; 
 
 after a selected number of additional authentication attempts, assessing a confidence score for the temporary template, the confidence score being based on a number of the additional authentication attempts assessed as being successful for the selected number of additional authentication attempts; and 
 adding the temporary template to the one or more reference templates in response to the assessed confidence score being above a selected confidence score. 
 
     
     
       2. The method of  claim 1 , wherein the first threshold is above the second threshold. 
     
     
       3. The method of  claim 1 , wherein the first image and the additional images are infrared images of the face of the user. 
     
     
       4. The method of  claim 1 , wherein assessing the confidence score for the temporary template comprises counting the number of additional matching scores that are above the second threshold for a certain number of successful unlock attempts of the device using the one or more reference templates for facial recognition authentication of the user. 
     
     
       5. The method of  claim 1 , wherein the selected confidence score comprises a lowest confidence score for a reference template in the one or more reference templates. 
     
     
       6. The method of  claim 1 , further comprising:
 obtaining a plurality of reference images of the face of the user using the camera; 
 selecting reference images for encoding based on the selected reference images meeting selected criteria for the reference images; 
 encoding a plurality of facial features of the user from the selected reference images to generate the plurality of reference feature vectors; and 
 storing the plurality of reference feature vectors as the one or more reference templates in the memory of the device. 
 
     
     
       7. The method of  claim 6 , wherein the selected criteria for the first images comprises one or more of the following criteria: the face of the user being in a field of view of the camera, a pose of the user being proper, a distance to the face of the user being within a certain distance, the face of the user having occlusion below a minimum value, and eyes of the user not being closed. 
     
     
       8. The method of  claim 1 , further comprising updating the unlock threshold when the temporary template is added to the one or more reference templates. 
     
     
       9. The method of  claim 1 , wherein assessing the confidence score for the temporary template comprises counting the number of the additional authentication attempts assessed as being successful out of the selected number of additional authentication attempts. 
     
     
       10. A method, comprising:
 obtaining a first image of a face of a user using a camera located on a device, the device comprising a computer processor and a memory; 
 encoding the first image to generate at least one first feature vector, wherein the first feature vector represents one or more facial features of the user in the first image; 
 comparing the first feature vector to one or more reference templates stored in a first space in the memory of the device to obtain a first matching score; 
 comparing the first matching score to an unlock threshold for the device; 
 in response to the first matching score being below the unlock threshold:
 prompting the user to complete at least one additional authentication protocol; 
 in response to the user being successfully authenticated using the at least one additional authentication protocol:
 storing the first feature vector as a temporary template in a second space in the memory of the device in response to the first matching score being above a first threshold; 
 obtaining a second image of the face of the user using the camera; 
 encoding the second image to generate at least one second feature vector, wherein the second feature vector represents one or more facial features of the user in the second image; 
 comparing the second feature vector to the temporary template stored in the second space to obtain a second matching score; 
 assessing a confidence score for the temporary template by comparing the second matching score to a second threshold, the second threshold being above the first threshold; and 
 adding the temporary template to the reference templates in response to the assessed confidence score being above a selected confidence score. 
 
 
 
     
     
       11. The method of  claim 10 , wherein the second threshold comprises a threshold selected to ensure a minimum number of successful comparisons between the second features scores and the second threshold are reached before allowing the temporary template to be added to the one or more reference templates. 
     
     
       12. The method of  claim 10 , wherein assessing the confidence score for the temporary template comprises counting the number of second matching scores that are above the second threshold for a certain number of failed attempts of authenticating the user by comparing the first matching score to the unlock threshold for the device. 
     
     
       13. The method of  claim 10 , wherein the at least one additional authentication protocol comprises entering a passcode for the device. 
     
     
       14. The method of  claim 10 , wherein the first threshold is below the unlock threshold. 
     
     
       15. The method of  claim 10 , wherein adding the temporary template to the one or more reference templates comprises:
 assessing the at least one first feature vector and a plurality of reference feature vectors for the reference templates in a feature space to determine a largest outlier in the reference feature vectors; and 
 replacing the largest outlier in the reference feature vectors with the at least one first feature vector. 
 
     
     
       16. The method of  claim 10 , further comprising using the temporary template to authenticate the user to use the device for a selected period of time. 
     
     
       17. A device, comprising:
 a camera; 
 at least one illuminator providing infrared illumination; 
 circuitry coupled to the camera and the illuminator, wherein the circuitry is configured to:
 obtain a first image of the face of the user using the camera; 
 encode the first image to generate at least one first feature vector, wherein the first feature vector represents one or more facial features of the user in the first image; 
 compare the first feature vector to one or more reference templates stored in a first space in the memory of the device to obtain a first matching score; 
 authorize the user to perform at least one operation on the device that requires authentication in response to the first matching score being above an unlock threshold for the device; 
 store the first feature vector as a temporary template in a second space in the memory of the device in response to the first matching score being above a first threshold; 
 obtain a plurality of additional images of the face of the user using the camera, wherein the additional images are obtained in response to additional authentication attempts initiated by the user; 
 subsequent to each additional authentication attempt initiated by the user:
 encode at least one additional image for the additional authentication attempt to generate at least one additional feature vector, wherein the additional feature vector represents one or more facial features of the user in the at least one additional image; 
 compare the additional feature vector to the temporary template stored in the second space to obtain an additional matching score; and 
 assess the additional authentication attempt as being successful 
 
 in response to the additional matching score for the additional 
 authentication attempt being above a second threshold; 
 after a selected number of additional authentication attempts, assess a confidence score for the temporary template, the confidence score being based on a number of the additional authentication attempts assessed as being successful for the selected number of additional authentication attempts; and 
 add the temporary template to the reference templates in response to the assessed confidence score being above a selected confidence score. 
 
 
     
     
       18. The device of  claim 17 , wherein the camera comprises an infrared sensor. 
     
     
       19. The device of  claim 17 , wherein the at least one illuminator comprises a flood infrared illuminator and a speckle pattern infrared illuminator. 
     
     
       20. The device of  claim 17 , wherein the circuitry is configured to update the unlock threshold when the temporary template is added to the one or more reference templates.