Camera autofocus using time-of-flight assistance

Abstract

Various embodiments disclosed herein include techniques for determining autofocus for a camera on a mobile device. In some instances, depth imaging is used to assist in determining a focus position for the camera through an autofocus process. For example, a determination of depth may be used to determine a focus position for the camera. In another example, the determination of depth may be used to assist another autofocus process.

What is claimed is: 
     
       1. A method, comprising:
 illuminating a subject in a field of view of a camera system with illumination from a time-of-flight imaging system; 
 determining a depth of the subject based on time-of-flight values assessed by the time-of-flight imaging system; 
 in response to the depth of the subject being above a depth threshold, determining a focus position of the camera system based on the determined depth of the subject; and 
 in response to the depth of the subject being below the depth threshold, operating a contrast based autofocus process to determine the focus position of the camera system. 
 
     
     
       2. The method of  claim 1 , wherein the depth is determined for the subject in a region of interest in a scene being captured by the camera system. 
     
     
       3. The method of  claim 1 , wherein the depth of the subject is determined in response to an illuminance value of the subject being determined to be below a predetermined illuminance threshold. 
     
     
       4. The method of  claim 1 , further comprising operating a phase detection autofocus process to determine the focus position of the camera system in response to an illuminance value of the subject being determined to be above a predetermined illuminance threshold. 
     
     
       5. The method of  claim 1 , wherein the focus position of the camera system is determined based on the determined depth of the subject when the depth of the subject is above the depth threshold and a depth-to-position model is converged. 
     
     
       6. The method of  claim 1 , wherein the focus position of the camera system is determined by operating a contrast based autofocus process to determine the focus position of the camera system when the depth of the subject is above the depth threshold and a depth-to-position model is not converged, the contrast based autofocus process beginning determining the focus position based on the determined depth of the subject. 
     
     
       7. The method of  claim 1 , further comprising determining a focus position of an additional camera system based on the focus position of the camera system. 
     
     
       8. The method of  claim 1 , wherein determining the focus position of the camera system based on the determined depth includes translating the determined depth to the focus position using a depth-to-position model implemented in the camera system. 
     
     
       9. A non-transitory computer-readable medium having instructions stored thereon that are executable by a computing device to perform operations, comprising:
 assessing whether a phase detection autofocus process is able to determine a focus position associated with a subject being captured by a camera system; 
 in response to an assessment that the phase detection autofocus process fails to determine the focus position:
 illuminating a subject in a field of view of the camera system with illumination from a time-of-flight imaging system; 
 determining a depth of the subject based on time-of-flight values assessed by the time-of-flight imaging system; 
 determining whether a depth-to-position model is converged; 
 in response to the depth-to-position model being converged, determining the focus position of the camera system based on the determined depth of the subject; and 
 in response to the depth-to-position model not being converged, operating a contrast based autofocus process to determine the focus position of the camera system. 
 
 
     
     
       10. The non-transitory computer-readable medium of  claim 9 , further comprising determining a focus position of an additional camera system based on the focus position of the camera system. 
     
     
       11. The non-transitory computer-readable medium of  claim 9 , wherein the depth-to-position model translates the determined depth of the subject to a position of least one lens in the camera system. 
     
     
       12. The non-transitory computer-readable medium of  claim 9 , wherein determining whether the depth-to-position model is converged includes:
 determining a difference between a first focal length of least one lens in the camera system determined using the depth of the subject and a second focal length of the least one lens in the camera system determined using a phase detection autofocus process; and 
 assessing whether the difference is above or below a predetermined difference threshold, wherein the depth-to-position model is converged when the difference is below the predetermined difference threshold. 
 
     
     
       13. The non-transitory computer-readable medium of  claim 9 , further comprising:
 assessing whether the determined depth of the subject is above or below a predetermined depth threshold; and 
 determining the focus position by beginning a scan of the contrast based autofocus process using the determined depth of the subject when the determined depth is above the predetermined depth threshold. 
 
     
     
       14. The non-transitory computer-readable medium of  claim 13 , wherein the contrast based autofocus process begins a scan using the predetermined depth threshold when the determined depth is below the predetermined depth threshold. 
     
     
       15. A device, comprising:
 a computer processor; 
 a memory; 
 a camera system having at least one lens and an image sensor; 
 a time-of-flight imaging system; and 
 circuitry coupled to the camera system and the time-of-flight imaging system, wherein the circuitry is configured to:
 determine, using a phase detection autofocus process, a first focus position for the at least one lens while capturing one or more images of a subject; 
 illuminate the subject in a field of view of the camera system with illumination from the time-of-flight imaging system; 
 determine a depth of the subject based on time-of-flight values assessed by the time-of-flight imaging system; 
 estimate, using a depth-to-position model, a second focus position for the at least one lens based on the determined depth of the subject; 
 assess a difference between the second focus position and the first focus position; and 
 in response to the difference between the second focus position and the first focus position being above a predetermined difference threshold, update the depth-to-position model based on the difference between the second focus position and the first focus position. 
 
 
     
     
       16. The device of  claim 15 , wherein updating the depth-to-position model based on the difference between the second focus position and the first focus position includes repeatedly updating the depth-to-position model based on the difference between the second focus position and the first focus position until the difference between the second focus position and the first focus position is below the predetermined difference threshold. 
     
     
       17. The device of  claim 15 , wherein updating the depth-to-position model based on the difference between the second focus position and the first focus position includes adjusting offset correction parameters of the depth-to-position model using the difference between the second focus position and the first focus position. 
     
     
       18. The device of  claim 15 , wherein the circuitry is configured to reduce a frame rate of determining the depth of the subject based on time-of-flight values when the difference between the second focus position and the first focus position moves below the predetermined difference threshold. 
     
     
       19. The device of  claim 15 , wherein the depth-to-position model implements a thin lens equation to translate the determined depth of the subject to the second focus position for the at least one lens. 
     
     
       20. The device of  claim 15 , wherein the circuitry is configured to update the depth-to-position model in response to changes in one or more operating parameters of the camera system in combination with the difference between the second focus position and the first focus position being above the predetermined difference threshold.