Object modeling and replacement in a video stream

Abstract

Systems, devices, and methods are presented for segmenting an image of a video stream with a client device by receiving one or more images depicting an object of interest and determining pixels within the one or more images corresponding to the object of interest. The systems, devices, and methods identify a position of a portion of the object of interest and determine a direction for the portion of the object of interest. Based on the direction of the portion of the object of interest, a histogram threshold is dynamically modified for identifying pixels as corresponding to the portion of the object of interest. The portion of the object of interest is replaced with a graphical interface element aligned with the direction of the portion of the object of interest.

What is claimed is: 
     
       1. A method, comprising:
 accessing, by one or more processors, a video stream; 
 generating a display of the video stream that depicts an object of interest in at least one frame of the video stream; 
 determining a direction associated with the object of interest for a set of previous frames of the video stream; 
 processing the direction of the object of interest for the set of previous frames to identify a target direction of the object of interest; 
 modifying the video stream to cause a graphical element to be positioned and depicted within the at least one frame of the video stream that depicts the object of interest based on the target direction of the object of interest; and 
 repositioning the graphical element within the video stream based on detected movement of the object of interest. 
 
     
     
       2. The method of  claim 1 , wherein the object of interest is identified by:
 determining pixels within the video stream corresponding to a user's body part; 
 identifying a position of the body part; and 
 determining, as the direction, a direction of the body part based on the identified position. 
 
     
     
       3. The method of  claim 1 , wherein processing the direction comprises combining the direction for the set of previous frames to identify an aggregate direction of the object of interest. 
     
     
       4. The method of  claim 1 , further comprising replacing a portion of the object of interest with the graphical element aligned with the direction of the object of interest. 
     
     
       5. The method of  claim 1 , further comprising adding the graphical element to a portion of the object of interest. 
     
     
       6. The method of  claim 1 , wherein the graphical element comprises a weapon in a video game, and wherein the weapon is pointed in the direction associated with the object of interest. 
     
     
       7. The method of  claim 1 , further comprising detecting the movement of the object of interest across a plurality of frames of the video stream. 
     
     
       8. The method of  claim 1  further comprising based on the direction of the object of interest, dynamically modifying a histogram threshold for identifying pixels as corresponding to the object of interest. 
     
     
       9. A system, comprising:
 one or more processors; and 
 a processor-readable storage device storing processor-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: 
 accessing a video stream; 
 generating a display of the video stream that depicts an object of interest in at least one frame of the video stream; 
 determining a direction associated with the object of interest for a set of previous frames of the video stream; 
 processing the direction of the object of interest for the set of previous frames to identify a target direction of the object of interest; 
 modifying the video stream to cause a graphical element to be positioned and depicted within the at least one frame of the video stream that depicts the object of interest based on the target direction of the object of interest; and 
 repositioning the graphical element within the video stream based on detected movement of the object of interest. 
 
     
     
       10. The system of  claim 9 , wherein the object of interest is identified by:
 determining pixels within the video stream corresponding to a user's body part; 
 identifying a position of the body part; and 
 determining, as the direction, a direction of the body part based on the identified position. 
 
     
     
       11. The system of  claim 10 , wherein processing the direction comprises combining the direction for the set of previous frames to identify an aggregate direction of the object of interest. 
     
     
       12. The system of  claim 9 , wherein the operations further comprise replacing a portion of the object of interest with the graphical element aligned with the direction of the object of interest. 
     
     
       13. The system of  claim 9 , wherein the operations further comprise adding the graphical element to a portion of the object of interest. 
     
     
       14. The system of  claim 9 , wherein the graphical element comprises a weapon in a video game, and wherein the weapon is pointed in the direction associated with the object of interest. 
     
     
       15. The system of  claim 9 , wherein the operations further comprise detecting the movement of the object of interest across a plurality of frames of the video stream. 
     
     
       16. The system of  claim 9 , wherein the operations further comprise based on the direction of the object of interest, dynamically modifying a histogram threshold for identifying pixels as corresponding to the object of interest. 
     
     
       17. A non-transitory processor-readable storage device storing processor-executable instructions that, when executed by a processor of a machine, cause the machine to perform operations comprising:
 accessing a video stream; 
 generating a display of the video stream that depicts an object of interest in at least one frame of the video stream; 
 determining a direction associated with the object of interest for a set of previous frames of the video stream; 
 processing the direction of the object of interest for the set of previous frames to identify a target direction of the object of interest; 
 modifying the video stream that is generated for display to cause a graphical element to be positioned and depicted within the at least one frame of the video stream that depicts the object of interest based on the target direction of the object of interest; and 
 repositioning the graphical element within the video stream based on detected movement of the object of interest. 
 
     
     
       18. The non-transitory processor-readable storage device of  claim 17 , the operations comprising replacing a portion of the object of interest with the graphical element aligned with the direction of the object of interest . 
     
     
       19. The non-transitory processor-readable storage device of  claim 17 , wherein the graphical element comprises a weapon in a video game, and wherein the weapon is pointed in the direction associated with the object of interest. 
     
     
       20. The non-transitory processor-readable storage device of  claim 19 , wherein the object of interest is identified by:
 determining pixels within the video stream corresponding to a user's body part; 
 identifying a position of the body part; and 
 determining, as the direction, a direction of the body part based on the identified position.