Method and apparatus for extracting static pattern based on output of event-based sensor

Abstract

A method of extracting a static pattern from an output of an event-based sensor. The method may include receiving an event signal from the event-based sensor in response to dynamic input, and extracting a static pattern associated with the dynamic input based on an identifier and time included in the event signal. The static pattern may be extracted from a map generated based on the identifier and time.

What is claimed is: 
     
       1. A pattern extraction method comprising:
 receiving an event signal indicating an event from an event-based sensor; 
 extracting a static pattern, which represents an object, based on an identification and time included in the event signal; and 
 outputting the extracted static pattern, 
 wherein the identification comprises location information of each of a plurality of pixels of the event-based sensor that sensed the event and the time comprises a time point in which the event is detected by a respective pixel from among said each pixel, and 
 wherein the extracting comprises generating a map comprising a plurality of elements corresponding to the location information of the plurality of pixels of the event-based sensor in which the time point is a value of a respective element from among the plurality of elements, 
 wherein the map is a three dimensional (3D) map and the extracting comprises extracting a ridge from the 3D map generated based on the identification and the time. 
 
     
     
       2. The pattern extraction method of  claim 1 , further comprising:
 detecting the event, which is a motion event, by the event-based sensor, which is a motion sensor, based on dynamic input, 
 wherein the dynamic input comprises an event to change at least one of color and intensity of light incident on at least one pixel from among the plurality of pixels in the event-based sensor. 
 
     
     
       3. The pattern extraction method of  claim 2 , wherein the static pattern is an appearance of the object in a motionless state associated with the dynamic input. 
     
     
       4. The pattern extraction method of  claim 1 , wherein the event indicates a motion of the object, and wherein the static pattern corresponds to an appearance of the object in a state in which the motion stops. 
     
     
       5. The pattern extraction method of  claim 1 , wherein the location information indicates the each pixel that senses a motion of the object among the plurality of pixels of the event-based sensor. 
     
     
       6. The pattern extraction method of  claim 1 , wherein the time point is a timestamp in which the event is detected based on sensing dynamic input by the each pixel. 
     
     
       7. The pattern extraction method of  claim 1 , wherein the extracting comprises:
 storing the identification and the time for each of a plurality of received events; and 
 extracting the static pattern based on the identification and the time. 
 
     
     
       8. The pattern extraction method of  claim 7 , wherein the storing comprises:
 storing, for each element of the event-based sensor, the identification comprising address of the element and the time indicating last change of the respective element; and 
 in response to an event changing a state of the respective element, overwriting the time for the respective element in the address for the respective element. 
 
     
     
       9. The pattern extraction method of  claim 1 , wherein the map is a three dimensional (3D) map and the extracting comprises extracting a ridge from the 3D map generated based on the identification and the time. 
     
     
       10. The pattern extraction method of claim  9  1, wherein the identification corresponds to coordinate values on the 3D map, and the time corresponds to an altitude on the 3D map. 
     
     
       11. The pattern extraction method of claim  9  1, wherein the extracting of the ridge comprises extracting at least two elements of the plurality of elements which store the time within a threshold time interval among the plurality of elements included in the 3D map. 
     
     
       12. The pattern extraction method of  claim 11 , wherein the extracting of the ridge further comprises:
 generating a new threshold time interval by adjusting a length of the threshold time interval based on a number of the elements that are extracted; and 
 extracting the at least two elements within the new threshold time interval. 
 
     
     
       13. The pattern extraction method of claim  9  1, wherein the extracting of the ridge comprises tracking the ridge so that a thickness of an edge in the ridge is equal to or less than a threshold thickness. 
     
     
       14. The pattern extraction method of  claim 1 , wherein the extracting comprises:
 setting a region of the map from which the static pattern is to be extracted; and 
 extracting a the ridge from the region based on a threshold time interval. 
 
     
     
       15. The pattern extraction method of  claim 14 , wherein the setting comprises setting the region based on a part of the object corresponding to dynamic input which generates the event of the event-based sensor. 
     
     
       16. The pattern extraction method of  claim 14 , wherein the extracting the ridge further comprises at least one of:
 adjusting a length of the threshold time interval based on a number of valid event signals received in the region; and 
 adjusting the length of the threshold time interval in response to a new event signal being received in the set region. 
 
     
     
       17. The pattern extraction method of  claim 1 , wherein the extracting further comprises:
 receiving depth information corresponding to dynamic input sensed by the event-based sensor; 
 storing the depth information in a corresponding element from among the plurality of elements; and 
 extracting the static pattern based on the location, the time point, and the depth information. 
 
     
     
       18. A computer program stored in a non-transitory computer-readable recording medium to implement the method of  claim 1  through a combination with hardware. 
     
     
       19. The pattern extraction method of  claim 1 , further comprising:
 receiving at least one additional signal indicating at least one additional even event from the event-based sensor; and 
 updating at least one point in the map based on the identification and the time, 
 wherein the extracting of the static pattern is asynchronous from the generating and the updating of the map, and 
 whereingwherein the location is an (x, y) location for each sensing element of the event-based sensor and the time point is the value for each of the (x, y) location. 
 
     
     
       20. The pattern extraction method of  claim 1 , wherein the event-based sensor outputs the information and the time only for sensing elements of the sensor that changed value from off to on or vice versa forming the event. 
     
     
       21. A pattern extraction method comprising:
 receiving an event signal indicating an event from an event-based sensor; 
 extracting a pattern, which represents a motionless object, based on an identification and time included in the event signal; and 
 outputting the pattern, 
 wherein the extracting comprises:
 generating a plurality of maps corresponding to a plurality of depth regions based on the identification and the time; and 
 extracting a ridge from at least one of the plurality of maps. 
 
 
     
     
       22. The pattern extraction method of  claim 21 , wherein the generating comprises:
 receiving depth information corresponding to dynamic input sensed by the event-based sensor; 
 selecting a map from the plurality of maps based on the depth information; and 
 updating the map based on the identification and the time. 
 
     
     
       23. A pattern extraction apparatus comprising:
 an event-based sensor configured to output an event signal indicating an event, the event signal comprising time and an identification of the event; and 
 a processor configured to extract a static pattern, which represents an object, based on the event signal and to output the extracted static pattern outside the pattern extraction apparatus, 
 wherein the identification comprises location information of each of a plurality of pixels of the event-based sensor that sensed the event and the time comprises a time point in which the event is detected by a respective pixel from among said each pixel, and 
 wherein the processor is configured to extract the static pattern by generating a map comprising a plurality of elements corresponding to the location information of the plurality of pixels of the event-based sensor in which the time point is a value of a respective element from among the plurality of elements, and 
 wherein the processor is configured to extract a ridge from the map, which is a 3D map. 
 
     
     
       24. The pattern extraction apparatus of  claim 23 , wherein the event-based sensor outputs the event signal, which is a motion event, in response to sensing dynamic input, which comprises a change of light incident on at least one pixel from among the plurality of pixels in the event-based sensor. 
     
     
       25. The pattern extraction apparatus of  claim 24 , wherein the event-based sensor outputs the event signal in response to sensing the dynamic input, which comprises a motion of the object, and wherein the static pattern is an appearance of the object in a state in which the motion stops. 
     
     
       26. The pattern extraction apparatus of  claim 23 , wherein the location indicates each pixel of the event-based sensor that senses dynamic input among the plurality of pixels of the event-based sensor, and
 wherein the time point is a timestamp in which the event is detected based on sensing dynamic input by the each pixel. 
 
     
     
       27. The pattern extraction apparatus of  claim 23 , wherein the processor is configured to store the identification and the time in a memory, and to extract the static pattern based on the identification and the time. 
     
     
       28. The pattern extraction apparatus of  claim 27 , wherein the processor is configured to overwrite the time in an element from among the plurality of elements corresponding to the identification based on the event detected by the event-based sensor. 
     
     
       29. The pattern extraction apparatus of  claim 23 , wherein the processor is configured to extract a ridge from the map, which is a 3D map, and
 wherein the location is coordinate values on the map, and the time is an altitude on the map. 
 
     
     
       30. The pattern extraction apparatus of  claim 29 , wherein the processor is configured to extract at least two elements from among the plurality of elements storing time within a threshold time interval among the plurality of elements included in the map. 
     
     
       31. The pattern extraction apparatus of  claim 30 , wherein the processor is configured to generate a new threshold time interval by adjusting a length of the threshold time interval based on a number of the at least two elements, and to extract the at least two elements storing time within the new threshold time interval. 
     
     
       32. The pattern extraction apparatus of  claim 29 , wherein the processor is configured to track the ridge so that a thickness of an edge in the ridge is equal to or less than a threshold thickness. 
     
     
       33. The pattern extraction apparatus of  claim 23 , wherein the processor is configured to set a region on the map from which the static pattern is to be extracted, and to extract a the ridge from the region based on a threshold time interval. 
     
     
       34. The pattern extraction apparatus of  claim 33 , wherein the processor is configured to set the region based on a part of the object corresponding to the event. 
     
     
       35. The pattern extraction apparatus of  claim 33 , wherein the processor is configured to adjust a length of the threshold time interval based on a number of valid event signals received in the region. 
     
     
       36. The pattern extraction apparatus of  claim 33 , wherein the processor is configured to adjust a length of the threshold time interval in response to a new event signal being received in the region. 
     
     
       37. The pattern extraction apparatus of  claim 23 , wherein the processor is configured to generate a plurality of maps corresponding to a plurality of depth regions based on the identification and the time, and to extract a the ridge from at least one of the plurality of maps. 
     
     
       38. The pattern extraction apparatus of  claim 37 , further comprising a depth sensor configured to output a depth information corresponding to the event sensed by the event-based sensor,
 wherein the processor is configured to select a map from the plurality of maps based on the depth information, and to update the map based on the identification and the time. 
 
     
     
       39. The pattern extraction apparatus of  claim 37 , further comprising a depth sensor configured to output a depth information corresponding to the event sensed by the event-based sensor,
 wherein the processor is configured to store the time and the depth information in an element identified by the identification from among the plurality of elements, and to extract the static pattern based on the identification, the time, and the depth information. 
 
     
     
       40. The pattern extraction apparatus of  claim 37 , wherein the processor is configured to generate a composite map based on the plurality of maps and extract the static pattern based on the composite map. 
     
     
       41. A display apparatus comprising
 a motion sensor comprising a plurality of sensing pixels and configured to output a signal representing a motion of an object, wherein the motion of the object is represented by time and corresponding location information indicating at least one pixel sensing the motion of the object among the plurality of sensing pixels; 
 a processor configured to generate at least one map based on the location information and the time, extract elements storing time within a threshold time interval among a plurality of elements included in the map, generate a new threshold time interval by adjusting a length of the threshold time interval based on a number of the extracted elements, and extracting new elements within the new threshold time interval, and detect the object based on the elements and the new elements, extracted by the processor, 
 wherein the processor is configured to extract a ridge from the map, which is a 3D map. 
 
     
     
       42. The display apparatus of  claim 41 , further comprising a display which displays the object. 
     
     
       43. The display apparatus of  claim 41 , wherein the processor controls the display apparatus based on the object. 
     
     
       44. The display apparatus of  claim 41 , wherein the motion of the object is a non-contact gesture of a user. 
     
     
       45. A pattern extraction apparatus comprising:
 an event-based sensor configured to generate time information and identification information in response to an event, and output an event signal including the time information and the identification information; and   a memory configured to store the time information and the identification information,   wherein the event-based sensor is configured to determine whether a change in a log intensity of light detected by at least one pixel among a plurality of pixels of the event-based sensor, after the event, exceeds a threshold, and to generate an ON event signal or an OFF event signal based on whether the change in the log intensity, after the event, exceeds the threshold,   wherein the identification information comprises location information of each of the plurality of pixels of the event-based sensor that sensed the event and the time information comprises a time point in which the event is detected by a respective pixel from among said each pixel, and   wherein, based on the event signal, a pattern, representing an object, is extracted, by generating a 3D map comprising a plurality of elements corresponding to the location information of the plurality of pixels of the event-based sensor in which the time point is a value of a respective element from among the plurality of elements, and extracting a ridge from the 3D map.   
     
     
       46. The pattern extracting apparatus of claim 45, wherein, the event-based sensor is further configured to:
 generate the ON event signal based on the change in the log intensity after the event exceeds a first threshold, and   generate the OFF event signal based on the change in the log intensity after the event exceeds a second threshold that is lower than the first threshold.   
     
     
       47. The pattern extracting apparatus of claim 45, wherein the identification information further comprises an address of the plurality of pixels included in the event-based sensor. 
     
     
       48. The pattern extracting apparatus of claim 45, wherein the map is a timestamp map that stores a latest time at which the time information corresponding to each of the plurality of elements is received, as a timestamp. 
     
     
       49. The pattern extracting apparatus of claim 45, wherein at least one pixel, sensing the event, among the plurality of pixels of the event-based sensor is an active pixel. 
     
     
       50. The pattern extracting apparatus of claim 49, wherein the active pixel is configured to generate an activation signal in response to the event, and
 wherein the event-based sensor is further configured to generate an event signal comprising the time information and the identification information of the active pixel, based on the activation signal.   
     
     
       51. The pattern extracting apparatus of claim 50, wherein the event-based sensor is further configured to generate the event signal asynchronously. 
     
     
       52. The pattern extracting apparatus of claim 50, wherein the event-based sensor is further configured to generate a pass flag to identify a plurality of event signals being spatiotemporally associated with each other, and to filter one of the plurality of event signals corresponding to background noise activity. 
     
     
       53. A pattern extracting apparatus comprising:
 an event-based sensor configured to generate time information and identification information based on a movement of an object; and   a processor configured to extract a static pattern corresponding to an appearance of the object in a state in which the object stops based on the time information and the identification information,   wherein the identification information comprises location information of each of a plurality of pixels of the event-based sensor that sensed the movement of the object and the time comprises a time point in which the movement of the object is detected by a respective pixel from among said each pixel,   wherein the processor is configured to extract the static pattern by generating a map comprising a plurality of elements corresponding to the location information of the plurality of pixels of the event-based sensor in which the time point is a value of a respective element from among the plurality of elements, and   wherein the processer is further configured to track the static pattern to adjust a thickness of an edge of the static pattern to be equal or less than a threshold thickness.   
     
     
       54. The pattern extracting apparatus of claim 53, wherein the identification information further comprises an address indicating at least one pixel, from among the plurality of pixels, sensing the movement of the object. 
     
     
       55. The pattern extracting apparatus of claim 54, wherein the map is a timestamp map storing a timestamp corresponding to each of the plurality of elements. 
     
     
       56. The pattern extracting apparatus of claim 55, wherein the processor is further configured to update the timestamp map by overwriting the timestamp corresponding to at least one of the plurality of elements with a new timestamp received from the event-based sensor. 
     
     
       57. The pattern extracting apparatus of claim 54, wherein the processor is further configured to extract the static pattern based on the timestamp last stored in each of the plurality of elements. 
     
     
       58. The pattern extracting apparatus of claim 55, wherein the processor is further configured to extract the static pattern based on the timestamp within a threshold time interval. 
     
     
       59. The pattern extracting apparatus of claim 58, wherein the processor is further configured to adjust a length of the threshold time interval to set a number of the plurality of elements storing the timestamp within the threshold time interval being greater than a predetermined minimum number and lesser than a predetermined maximum number. 
     
     
       60. The pattern extracting apparatus of claim 58, wherein the processor is further configured to, based on the timestamp map comprising a first region corresponding to a first object and a second region corresponding to a second object different from the first object, apply a first threshold time interval to the first region and a second threshold time interval to the second region, the second threshold different being from the first threshold time interval.