Automatic haptic generation based on color features and motion analysis

Abstract

Examples of devices, systems, and methods to automatically generate haptics based on visual color features and motion analysis are disclosed. In one example, a video having a plurality of frames is received and masked frames for the video are generated by applying a color mask to the plurality of frames. An event between two of the masked frames is detected and an optical flow estimate is generated for these masked frames. At least one haptic effect corresponding to the event is generated based on the optical flow. The generated haptic effect(s) may be output to a haptic file or a haptic output device, or both.

1 . A method comprising:
 receiving a first video frame from a video, the video having a plurality of video frames;   detecting an event shown in the first video frame, the detecting based on a second video frame of the plurality of video frames from the video, the second video frame preceding the first video frame in the video;   determining whether the event is a false detection;   if the event is a false detection, not generating a haptic effect; and   if the event is not a false detection:
 generating a haptic effect corresponding to the event; and 
 producing the haptic effect via a haptic output device. 
   
     
     
         2 . The method of  claim 1 , further comprising:
 applying a color mask to each of the first and second video frames;   determining a blob in the second video frame; and   determining a change in a size of the blob in the first video frame; and   wherein detecting the event is based on the change in the size of the blob.   
     
     
         3 . The method of  claim 1 , wherein determining whether the event is a false detection comprises:
 determining one or more motion vectors between the first and second frames; and   determining the event based on the one or more motion vectors.   
     
     
         4 . The method of  claim 3 , further comprising:
 applying a color mask to each of the first and second video frames to generate respective corresponding first and second masked video frames; and   determining the one or more motion vectors based on the first and second masked video frames.   
     
     
         5 . The method of  claim 3 , wherein the second video frame is a reference frame, and detecting the event comprises estimating an optical flow between the second video frame and the first video frame. 
     
     
         6 . The method of  claim 5 , wherein the first video frame comprises a predictive or bi-predictive video frame, and wherein determining the event is based on motion vector displacement information associated with the first video frame. 
     
     
         7 . The method of  claim 3 , wherein determining whether the event is a false detection comprises, in response to determining that the one or more motion vectors collectively point in a substantially same direction, determining that the event is a false detection. 
     
     
         8 . The method of  claim 1 , wherein detecting the event occurs substantially in real-time. 
     
     
         9 . A system comprising:
 a non-transitory computer-readable medium; and   a processor configured to execute processor-executable instructions stored in the non-transitory computer-readable medium, the processor-executable instructions configured to cause the processor to:
 receive a first video frame from a video, the video having a plurality of video frames; 
 detect an event shown in the first video frame, the detecting based on a second video frame of the plurality of video frames from the video, the second video frame preceding the first video frame in the video 
 determine whether the event is a false detection; 
 if the event is a false detection, not generate a haptic effect; and 
 if the event is not a false detection:
 generate a haptic effect corresponding to the event; and 
 produce the haptic effect via a haptic output device. 
 
   
     
     
         10 . The system of  claim 9 , wherein the processor-executable instructions are further configured to cause the processor to:
 apply a color mask to each of the first and second video frames;   determine a blob in the second video frame; and   determine a change in a size of the blob in the first video frame; and   wherein the processor-executable instructions are further configured to cause the processor to detect the event based on the change in the size of the blob.   
     
     
         11 . The system of  claim 9 , wherein the processor-executable instructions configured to cause the processor to determine whether the event is a false detection comprise processor-executable instructions configured to cause the processor to:
 determine one or more motion vectors between the first and second frames; and   determine the event based on the one or more motion vectors.   
     
     
         12 . The system of  claim 11 , wherein the processor-executable instructions are further configured to cause the processor to:
 apply a color mask to each of the first and second video frames to generate respective corresponding first and second masked video frames; and   determine the one or more motion vectors based on the first and second masked video frames.   
     
     
         13 . The system of  claim 11 , wherein the second video frame is a reference frame, and wherein the processor-executable instructions configured to cause the processor to detect the event comprise processor-executable instructions configured to cause the processor to estimate an optical flow between the second video frame and the first video frame. 
     
     
         14 . The system of  claim 13 , wherein the first video frame comprises a predictive or bi-predictive video frame, and wherein the processor-executable instructions are further configured to cause the processor to determine the event based on motion vector displacement information associated with the first video frame. 
     
     
         15 . The system of  claim 11 , wherein the processor-executable instructions configured to cause the processor to determine whether the event is a false detection comprises processor-executable instructions configured to cause the processor to, in response to a determination that the one or more motion vectors collectively point in a substantially same direction, determine that the event is a false detection. 
     
     
         16 . The system of  claim 9 , wherein the processor-executable instructions are further configured to cause the processor to detect the event substantially in real-time. 
     
     
         17 . A non-transitory computer-readable medium comprising processor-executable instructions configured to cause a processor to:
 receive a first video frame from a video, the video having a plurality of video frames;   detect an event shown in the first video frame, the detecting based on a second video frame of the plurality of video frames from the video, the second video frame preceding the first video frame in the video   determine whether the event is a false detection;   if the event is a false detection, not generate a haptic effect; and   if the event is not a false detection:
 generate a haptic effect corresponding to the event; and 
 produce the haptic effect via a haptic output device. 
   
     
     
         18 . The non-transitory computer-readable medium of  claim 17 , wherein the processor-executable instructions are further configured to cause the processor to:
 apply a color mask to each of the first and second video frames;   determine a blob in the second video frame; and   determine a change in a size of the blob in the first video frame; and   wherein the processor-executable instructions are further configured to cause the processor to detect the event based on the change in the size of the blob.   
     
     
         19 . The non-transitory computer-readable medium of  claim 17 , wherein the processor-executable instructions configured to cause the processor to determine whether the event is a false detection comprise processor-executable instructions configured to cause the processor to:
 determine one or more motion vectors between the first and second frames; and   determine the event based on the one or more motion vectors.   
     
     
         20 . The non-transitory computer-readable medium of  claim 19 , wherein the processor-executable instructions configured to cause the processor to determine whether the event is a false detection comprises processor-executable instructions configured to cause the processor to, in response to a determination that the one or more motion vectors collectively point in a substantially same direction, determine that the event is a false detection. 
     
     
         21 . The non-transitory computer-readable medium of  claim 17 , wherein the processor-executable instructions are further configured to cause the processor to detect the event substantially in real-time.