Systems and Methods of Person Recognition in Video Streams

Abstract

The various implementations described herein include systems and methods for recognizing persons in video streams. In one aspect, a method includes: (1) obtaining a live video stream; (2) detecting person(s) in the stream; and (3) determining, from analysis of the live video stream, personally identifiable information of the detected person(s); (4) determining, based on the personally identifiable information, that the first person is not known to the computing system; (5) in accordance with the determination that the first person is not known: (a) storing the personally identifiable information; and (b) requesting a user to classify the first person; and (6) in accordance with (i) a determination that a predetermined amount of time has elapsed since the request was transmitted and a response was not received, or (ii) a determination that a response was received classifying the first person as a stranger, deleting the stored personally identifiable information.

1 - 20 . (canceled) 
     
     
         21 . A method comprising:
 obtaining one or more images from one or more of a plurality of devices associated with a smart home environment, each image of the one or more images having an image portion corresponding to a person;   determining, for each of the one or more images, information that identifies one or more attributes of the person, the information determined from analysis of the one or more images or metadata associated with the one or more images;   grouping, based on the information that identifies the one or more attributes of the person, images of the one or more images into a plurality of groups;   causing a group of images of the plurality of groups to be presented, the group of images associated with the person;   receiving a request to remove at least one image from the group of images associated with the person;   removing, in response to the request, the at least one image from the group of images associated with the person; and   generating, in response to removing the at least one image, updated information for the group of images associated with the person by disassociating the information from the group of images associated with the person.   
     
     
         22 . The method as described in  claim 21 , further comprising, prior to determining information that identifies one or more attributes of the person, cropping each image of the one or more images to the image portion corresponding to the person. 
     
     
         23 . The method as described in  claim 22 , wherein determining the information that identifies one or more attributes of the person is determined based on the image portion corresponding to the person. 
     
     
         24 . The method as described in  claim 21 , wherein the one or more attributes of the person include:
 a gait;   a pose;   an attire;   one or more biometrics, including a height, a facial structure, an eye color, a skin tone, or a hair color; or   an activity.   
     
     
         25 . The method as described in  claim 21 , wherein the metadata associated with the one or more images includes:
 a time of day;   an occupancy of the smart home environment;   a camera source; or   a camera source location.   
     
     
         26 . The method as described in  claim 21 , further comprising, prior to causing a group of images of a plurality of groups to be presented, receiving an indication from a user to view the group of images associated with the person, and wherein the causing the group of images of a plurality of groups to be presented is responsive to receiving the indication. 
     
     
         27 . The method as described in  claim 21 , further comprising, in addition to causing the group of images of the plurality of groups to be presented, causing information associated with the group of images to be presented. 
     
     
         28 . The method as described in  claim 27 , wherein the information associated with the group of images includes one or more of:
 an associated person identifier;   an associated image identifier;   quality information;   pose information;   timing information;   confidence information;   location information;   physical feature information; or   behavioral information.   
     
     
         29 . The method as described in  claim 21 , wherein the plurality of groups each represent a different person. 
     
     
         30 . The method as described in  claim 29 , further comprising, for each respective group of the plurality of groups, selecting an image from the respective group as a profile image. 
     
     
         31 . The method as described in  claim 30 , wherein the profile image for a respective group is selected based on a pose of a respective person in an image of the respective group. 
     
     
         32 . The method as described in  claim 30 , wherein the profile image for a respective group is selected based on a quality factor of an image of the respective group. 
     
     
         33 . The method as described in  claim 29 , further comprising, based on the metadata associated with the one or more images, labeling a respective group. 
     
     
         34 . The method as described in  claim 29 , further comprising, after causing a group of images of the plurality of groups to be presented and for each respective group of the plurality of groups, prompting a user to label a respective group. 
     
     
         35 . The method as described in  claim 34 , further comprising, based at least in part on the labeling of a respective group, classifying a person of the respective group as any of:
 a known person;   a trusted person;   an authorized person;   a stranger; or   an unknown person.   
     
     
         36 . The method as described in  claim 34 , further comprising, based at least in part on the classification or the metadata associated with the one or more images of the respective group, removing the one or more images of the respective group. 
     
     
         37 . The method as described in  claim 36 , wherein removing the one or more images of the respective group is based on the respective group representing a person who is classified as an unknown person and the metadata associated with the one or more images of the respective group indicates the one or more images as being captured one or more years ago. 
     
     
         38 . The method as described in  claim 21 , wherein the plurality of devices associated with the smart home environment include any one of a camera configured to capture images, a communication system configured to receive or transmit one or more images or metadata associated with the one or more images, or storage media configured to store images. 
     
     
         39 . A system comprising:
 one or more processors; and   memory coupled to the one or more processors, the memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:
 obtaining one or more images from one or more of a plurality of devices associated with a smart home environment, each image of the one or more images having an image portion corresponding to a person; 
 determining, for each of the one or more images, information that identifies one or more attributes of the person, the information determined from analysis of the one or more images or metadata associated with the one or more images; 
 grouping, based on the information that identifies the one or more attributes of the person, images of the one or more images into a plurality of groups; 
 causing a group of images of the plurality of groups to be presented, the group of images associated with the person; 
 receiving a request to remove at least one image from the group of images associated with the person; 
 removing, in response to the request, the at least one image from the group of images associated with the person; and 
 generating, in response to removing the at least one image, updated information for the group of images associated with the person by disassociating the information from the group of images associated with the person. 
   
     
     
         40 . A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing system, cause the system to:
 obtain one or more images from one or more of a plurality of devices associated with a smart home environment, each image of the one or more images having an image portion corresponding to a person;   determine, for each of the one or more images, information that identifies one or more attributes of the person, the information determined from analysis of the one or more images or metadata associated with the one or more images;   group, based on information that identifies the one or more attributes of the person, images of the one or more images into a plurality of groups;   cause a group of images of the plurality of groups to be presented, the group of images associated with the person;   receive a request to remove at least one image from the group of images associated with the person;   remove, in response to the request, the at least one image from the group of images associated with the person; and   generate, in response to removing the at least one image, updated information for the group of images associated with the person by disassociating the information from the group of images associated with the person.