Varied depth determination using stereo vision and phase detection auto focus (pdaf)

Abstract

Disclosed are systems, methods, and non-transitory computer-readable media for varied depth determination using stereo vision and phase detection auto focus (PDAF). Computer stereo vision (stereo vision) is used to extract three-dimensional information from digital images. To utilize stereo vison, two optical sensors are displaced horizontally from one another and used to capture images depicting two differing views of a real-world environment from two different vantage points. The relative depth of the objects captured in the images is determined using triangulation by comparing the relative positions of the objects in the two images. For example, the relative positions of matching objects (e.g., features) identified in the captured images are used along with the known orientation of the optical sensors (e.g., distance between the optical sensors, vantage points the optical sensors) to estimate the depth of the objects.

What is claimed is: 
     
         1 . A method comprising:
 accessing a pair of corresponding images that depict an object captured by a pair of optical sensors that include a first optical sensor and a second optical sensor;   determining a stereo vision depth value of the object based on the pair of corresponding images;   determining an expected location of the object within each image among the pair of corresponding images based on known vantage points of the first optical sensor and the second optical sensor;   detecting an actual location of the object within each image among the pair of corresponding images;   comparing the actual location of the object within each image to the expected location of the object within each respective image;   determining that the actual location of the object does not match the expected location in at least one of the images based on the comparing;   determining a Phase Detection Autofocus (PDAF) depth value of the object based on at least one image from among the pair of corresponding images in response to determining that the actual location does not match the expected location; and   providing the PDAF depth value as an output value based on the determining that the actual location does not match the expected location.   
     
     
         2 . The method of  claim 1 , wherein the first optical sensor and the second optical sensor are displaced horizontally and at a known distance from one another at the known vantage points. 
     
     
         3 . The method of  claim 1 , wherein determining the PDAF depth value of the object includes:
 determining a first PDAF value based on the first image captured by the first optical sensor;   determining a second PDAF value based on the second image captured by the second optical sensor; and   generating an average PDAF value based on an aggregation of the first PDAF value and the second PDAF value.   
     
     
         4 . The method of  claim 1 , further comprising:
 comparing the PDAF depth value with a threshold value, wherein the threshold value comprises a predefined value that indicates a maximum distance.   
     
     
         5 . The method of  claim 1 , wherein providing the PDAF depth value as the output value includes:
 causing display of a presentation of the PDAF depth value.   
     
     
         6 . The method of  claim 1 , wherein the pair of optical sensors are components of an augmented reality (AR) device, and wherein the output value is used to generate virtual content based on the depth of the object. 
     
     
         7 . The method of  claim 6 , wherein the virtual content is presented to appear overlaid on or adjacent to the object in a display of the AR device. 
     
     
         8 . A system comprising:
 one or more computer processors; and   one or more computer-readable mediums storing instructions that, when executed by the one or more computer processors, cause the system to perform operations comprising:   accessing a pair of corresponding images that depict an object captured by a pair of optical sensors that include a first optical sensor and a second optical sensor;   determining a stereo vision depth value of the object based on the pair of corresponding images;   determining an expected location of the object within each image among the pair of corresponding images based on known vantage points of the first optical sensor and the second optical sensor;   detecting an actual location of the object within each image among the pair of corresponding images;   comparing the actual location of the object within each image to the expected location of the object within each respective image;   determining that the actual location of the object does not match the expected location in at least one of the images based on the comparing;   determining a Phase Detection Autofocus (PDAF) depth value of the object based on at least one image from among the pair of corresponding images in response to determining that the actual location does not match the expected location; and   providing the PDAF depth value as an output value based on the determining that the actual location does not match the expected location.   
     
     
         9 . The system of  claim 8 , wherein the first optical sensor and the second optical sensor are displaced horizontally and at a known distance from one another at the known vantage points. 
     
     
         10 . The system of  claim 8 , wherein determining the PDAF depth value of the object includes:
 determining a first PDAF value based on the first image captured by the first optical sensor;   determining a second PDAF value based on the second image captured by the second optical sensor; and   generating an average PDAF value based on an aggregation of the first PDAF value and the second PDAF value.   
     
     
         11 . The system of  claim 8 , further comprising:
 comparing the PDAF depth value with a threshold value, wherein the threshold value comprises a predefined value that indicates a maximum distance.   
     
     
         12 . The system of  claim 8 , wherein providing the PDAF depth value as the output value includes:
 causing display of a presentation of the PDAF depth value.   
     
     
         13 . The system of  claim 8 , wherein the pair of optical sensors are components of an augmented reality (AR) device, and wherein the output value is used to generate virtual content based on the depth of the object. 
     
     
         14 . The system of  claim 13 , wherein the virtual content is presented to appear overlaid on or adjacent to the object in a display of the AR device. 
     
     
         15 . A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
 accessing a pair of corresponding images that depict an object captured by a pair of optical sensors that include a first optical sensor and a second optical sensor;   determining a stereo vision depth value of the object based on the pair of corresponding images;   determining an expected location of the object within each image among the pair of corresponding images based on known vantage points of the first optical sensor and the second optical sensor;   detecting an actual location of the object within each image among the pair of corresponding images;   comparing the actual location of the object within each image to the expected location of the object within each respective image;   determining that the actual location of the object does not match the expected location in at least one of the images based on the comparing;   determining a Phase Detection Autofocus (PDAF) depth value of the object based on at least one image from among the pair of corresponding images in response to determining that the actual location does not match the expected location; and   providing the PDAF depth value as an output value based on the determining that the actual location does not match the expected location.   
     
     
         16 . The non-transitory machine-readable storage medium of  claim 15 , wherein the first optical sensor and the second optical sensor are displaced horizontally and at a known distance from one another at the known vantage points. 
     
     
         17 . The non-transitory machine-readable storage medium of  claim 15 , wherein determining the PDAF depth value of the object includes:
 determining a first PDAF value based on the first image captured by the first optical sensor;   determining a second PDAF value based on the second image captured by the second optical sensor; and   generating an average PDAF value based on an aggregation of the first PDAF value and the second PDAF value.   
     
     
         18 . The non-transitory machine-readable storage medium of  claim 15 , further comprising:
 comparing the PDAF depth value with a threshold value, wherein the threshold value comprises a predefined value that indicates a maximum distance.   
     
     
         19 . The non-transitory machine-readable storage medium of  claim 15 , wherein providing the PDAF depth value as the output value includes:
 causing display of a presentation of the PDAF depth value.   
     
     
         20 . The non-transitory machine-readable storage medium of  claim 15 , wherein the pair of optical sensors are components of an augmented reality (AR) device, and wherein the output value is used to generate virtual content based on the depth of the object.