Image sensor for vision based human computer interaction

Abstract

According to the invention, an image sensor is disclosed. The image sensor may include a plurality of pixels. Each pixel of a first portion of the plurality of pixels may include a near-infrared filter configured to block red, green, and blue light; and pass near-infrared light. Each pixel of a second portion of the plurality of pixels may be configured to receive at least one of red, green, or blue light; and receive near-infrared light.

What is claimed is: 
     
         1 . An image sensor comprising:
 a plurality of pixels, wherein:
 each pixel of a first portion of the plurality of pixels comprises a near-infrared filter configured to:
 block red, green, and blue light; and 
 pass at least a portion of near-infrared light; and 
 
 each pixel of a second portion of the plurality of pixels is configured to:
 receive at least one of red, green, or blue light; and 
 receive near-infrared light. 
 
   
     
     
         2 . The image sensor of  claim 1 , wherein:
 the first portion of the plurality of pixels comprises substantially 15 out of every 16 pixels in the plurality of pixels; and   the second portion of the plurality of pixels comprises substantially one out of every 16 pixels in the plurality of pixels.   
     
     
         3 . The image sensor of  claim 1 , wherein the image sensor further comprises:
 at least one processor configured to process information from the plurality of pixels in at least two modes, wherein:
 in a first mode, the at least one processor is configured to return data characterizing at least one of red, blue, and green light received by the plurality of pixels; and 
 in a second mode, the at least one processor is configured to return data characterizing near-infrared light received by the plurality of pixels. 
   
     
     
         4 . The image sensor of  claim 3 , wherein being configured to return data characterizing at least one of red, blue, and green light received by the plurality of pixels comprises being configured to:
 receive data from only the second portion of the plurality of pixels; and   process the data from only the second portion of the plurality of pixels.   
     
     
         5 . The image sensor of  claim 3 , wherein being configured to return data characterizing at least one of red, blue, and green light received by the plurality of pixels comprises being configured to:
 receive data from both the first portion of the plurality of pixels and the second portion of the plurality of pixels;   remove data received from the first portion of the plurality of pixels; and   process the data remaining from the second portion of the plurality of pixels.   
     
     
         6 . The image sensor of  claim 3 , wherein being configured to return data characterizing near-infrared light received by the plurality of pixels comprises being configured to:
 receive a first set of data from both the first portion of the plurality of pixels and the second portion of the plurality of pixels produced during activation of a near-infrared illuminator;   receive a second set of data from both the first portion of the plurality of pixels and the second portion of the plurality of pixels produced when the near-infrared illuminator is not activated; and   process the first set of data and the second set of data.   
     
     
         7 . The image sensor of  claim 6 , wherein processing the first set of data and the second set of data comprises:
 comparing the first set of data to the second set of data.   
     
     
         8 . The image sensor of  claim 6 , wherein:
 a time the first set of data is produced is substantially close in time to a time the second set of data is produced.   
     
     
         9 . The image sensor of  claim 3 , wherein in either one of the first mode or the second mode, the at least one processor is further configured to:
 receive data from only a sub-portion of the first portion of the plurality of pixels and only a sub-portion of the second portion of pixels.   
     
     
         10 . The image sensor of  claim 9 , wherein:
 the sub-portion of the first portion of the plurality of pixels and the sub-portion of the second portion of the plurality of pixels represent four non-overlapping equally sized regions of pixels of the plurality of pixels.   
     
     
         11 . The image sensor of  claim 9 , wherein:
 the four non-overlapping equally sized regions of pixels of the plurality of pixels are defined by a two pairs of x/y coordinates.   
     
     
         12 . The image sensor of  claim 11 , wherein a size of each of the four non-overlapping equally sized regions is based at least in part on characteristics of a user observed by the image sensor. 
     
     
         13 . The image sensor of  claim 11 , wherein a size of each of the four non-overlapping equally sized regions is based at least in part on at least one of a size or a location of a user's eye observed by the image sensor. 
     
     
         14 . The image sensor of  claim 3 , wherein in either one of the first mode or the second mode, the at least one processor is further configured to at least:
 adjust a gain level of the data received from the first portion of the plurality of pixels or a gain level of the data received from the second portion of the plurality of pixels.   
     
     
         15 . The image sensor of  claim 14 , wherein adjusting the gain level of the data received from the second portion of the plurality of pixels comprises:
 reducing the gain level of the data received from the second portion of the plurality of pixels.   
     
     
         16 . The image sensor of  claim 3 , wherein in the second mode, the at least one processor is further configured to:
 interpolate the data from the first portion of the plurality of pixels to determine data representing an estimate of the near-infrared light received at the second portion of the plurality of pixels.   
     
     
         17 . The image sensor of  claim 1 , wherein the image sensor is disposed in an eye tracking device in a virtual reality headset, an augmented reality headset, or a display headset. 
     
     
         18 . The image sensor of  claim 1 , wherein the image sensor further comprises:
 a processing layer disposed beneath the plurality of pixels and communicatively coupled therewith; and   a memory layer disposed beneath the processing layer and communicatively coupled therewith.   
     
     
         19 . A method for capturing images, wherein the method comprises:
 receiving light at a plurality of pixels, wherein receiving light at the plurality of pixels comprises:
 blocking red, green, and blue light at each pixel of a first portion of the plurality of pixels with a near-infrared filter; and 
 receiving near-infrared light at each pixel of the first portion of the plurality of pixels; 
 receiving at least one of red, green, or blue light at each pixel of a second portion of the plurality of pixels; and 
 receive near-infrared light at each pixel of the second portion of the plurality of pixels. 
   
     
     
         20 . A non-transitory machine readable medium having instructions stored thereon for capturing images, wherein the instructions are executable by one or more processors to at least:
 process signals produced by a first portion of a plurality of pixels receiving light, wherein receiving light at the first portion of the plurality of pixels comprises:
 blocking red, green, and blue light at each pixel of the first portion of the plurality of pixels with a near-infrared filter; and 
 receiving near-infrared light at each pixel of the first portion of the plurality of pixels; 
   process signals produced by a second portion of the plurality of pixels receiving light, wherein receiving light at the second portion of the plurality of pixels comprises:
 receiving at least one of red, green, or blue light at each pixel of the second portion of the plurality of pixels; and 
 receive near-infrared light at each pixel of the second portion of the plurality of pixels.