Method and device for detecting and tracking non-rigid objects in movement, in real time, in a video stream, enabling a user to interact with a computer system

Abstract

The invention relates in particular to the detection of interactions with a software application according to a movement of an object situated in the field of an image sensor. After having received a first and a second image and having identified a first region of interest in the first image, a second region of interest, corresponding to the first region of interest, is identified in the second image. The first and second regions of interest are compared and a mask of interest characterizing a variation of at least one feature of corresponding points in the first and second regions of interest is determined. A movement of the object is then determined from said mask of interest. The movement is analyzed and, in response, a predetermined action is triggered or not triggered.

1 . A computer implemented method of detecting movement of at least one object situated in a field of an image sensor, the image sensor providing a stream of images to the computer, the method comprising:
 receiving at least one first image from the image sensor;   identifying at least one first region of interest in the first image, wherein the at least one first region of interest corresponds to a part of the at least one first image;   receiving at least one second image from the image sensor;   identifying at least one second region of interest in the at least one second image, wherein the at least one second region of interest corresponds to the at least one first region of interest of the at least one first image;   comparing the at least one first and second regions of interest and determining a mask of interest characterizing a variation of at least one feature of corresponding points in the at least one first and second regions of interest;   determining a movement of the at least one object from the mask of interest, wherein the at least one object is at least partially represented in at least one of the at least one first and second regions of interest;   analyzing the movement; and   determining whether to trigger an action.   
     
     
         2 . The method according to  claim 1 , wherein determining the movement comprises determining and matching at least one pair of points of interest in the at least one first and second images, wherein at least one point of the at least one pair of points of interest belong to the mask of interest. 
     
     
         3 . The method according to  claim 2 , wherein determining the movement comprises determining and matching a plurality of pairs of points of interest in the at least one first and second images, wherein at least one point of each of the pairs of points of interest belong to the mask of interest, wherein the movement is estimated on the basis of a transformation of a first set of points of interest into a second set of points of interest, wherein the points of interest of the first and second sets belong to the plurality of pairs of points of interest, wherein the points of interest of the first set of points of interest additionally belong to at least one first image, and wherein the points of interest of the second set of points of interest additionally belong to at least one second image. 
     
     
         4 . The method according to  claim 3 , wherein the transformation implements a weighting function based on a distance between two points of interest from the same pairs of points of interest of the plurality of pairs of points of interest. 
     
     
         5 . The method according to  claim 3 , further comprising validating at least one point of interest of the at least one first image, belonging to the at least one pair of points of interest, according to the determined movement, wherein the at least one validated point of interest is used to track the object in at least one third image following the at least one second image and the at least one validated point of interest is used for modifying a mask of interest created on the basis of the at least one second and third images. 
     
     
         6 . The method according to  claim 1 , wherein comparing the at least one first and second regions of interest comprises performing subtraction, point by point, of values of corresponding points of the at least one first and second regions of interest and comparing a result of the subtraction to a threshold. 
     
     
         7 . The method according to  claim 1 , further comprising detecting at least one feature in the at least one first image, wherein the at least one first region of interest is at least partially identified in response to the detecting. 
     
     
         8 . The method according to  claim 7 , wherein the at least one feature includes at least one of a shape and a color. 
     
     
         9 . The method according to  claim 1 , further comprising estimating at least one modified second region of interest in the at least one second image, wherein the at least one modified second region of interest of the at least one second image is estimated according to the at least one first region of interest of the at least one first image and of the at least one second region of interest of the at least one second image. 
     
     
         10 . The method according to  claim 9 , wherein the estimating comprises performing an object tracking algorithm of KL T type. 
     
     
         11 . The method according to  claim 1 , wherein the movement comprises at least one of a translation, a rotation, a scale factor. 
     
     
         12 . The method according to  claim 11 , wherein the movement comprises a scale factor and wherein whether the action is triggered is determined based at least in part on the scale factor. 
     
     
         13 . The method according to  claim 1 , wherein movements of at least two objects situated in the field of the image sensor are determined, and wherein whether the action is triggered is determined based at least in part on a combination of the movements associated with the at least two objects. 
     
     
         14 . (canceled) 
     
     
         15 . (canceled) 
     
     
         16 . A non-transitory computer readable medium having instructions, which, when executed cause the computer to perform the method of  claim 1 . 
     
     
         17 . A device configured to perform the method of  claim 1 .