Method and apparatus for determining image depth information, electronic device, and media

Abstract

Provided are a method and apparatus for determining image depth information, an electronic device, and a medium. The method includes: acquiring first depth information of pixels in a target image output by a first prediction layer; generating the point cloud model of the target image according to the first depth information, and determining initial depth information of the pixels in the target image in a second prediction layer according to the point cloud model; and performing propagation optimization according to the initial depth information, and determining second depth information of the pixels in the target image output by the second prediction layer, where the first prediction layer is configured before the second prediction layer.

What is claimed is: 
     
       1. A method for determining image depth information, comprising:
 acquiring first depth information of pixels in a target image output by a first prediction layer; 
 generating a point cloud model of the target image according to the first depth information, and determining initial depth information of the pixels in the target image in a second prediction layer according to the point cloud model; and 
 performing propagation optimization according to the initial depth information, and determining second depth information of the pixels in the target image output by the second prediction layer, 
 wherein the first prediction layer is configured before the second prediction layer, 
 wherein the performing propagation optimization according to the initial depth information, and determining the second depth information of the pixels in the target image output by the second prediction layer comprises: 
 determining a match image matched with the target image from an image sequence to which the target image belongs by using a binocular vision algorithm, wherein the image sequence is obtained by collecting a same object; 
 determining a correspondence relationship between the pixels in the target image and pixels in the match image according to extrinsics information and intrinsics information of a camera that collects the target image and extrinsics information and intrinsics information of a camera that collects the match image; and 
 performing optimization propagation on the initial depth information of the pixels in the target image according to similarity between corresponding pixels, and determining the second depth information of the pixels in the target image output by the second prediction layer. 
 
     
     
       2. The method according to  claim 1 , wherein generating the point cloud model of the target image according to the first depth information comprises:
 acquiring image coordinate information of the pixels in the target image and intrinsics information and extrinsics information of a camera that collects the target image; and 
 determining world coordinate information of the pixels in the target image according to the image coordinate information, the intrinsics information, the extrinsics information, and the first depth information, and generating the point cloud model of the target image according to the world coordinate information. 
 
     
     
       3. The method according to  claim 1 , wherein the determining the initial depth information of the pixels in the target image in the second prediction layer according to the point cloud model comprises:
 determining a projection pixel of each point in the point cloud model in the target image; and 
 determining the initial depth information of the pixels in the target image in the second prediction layer according to depth information of projection pixels. 
 
     
     
       4. The method according to  claim 3 , wherein the determining the initial depth information of the pixels in the target image in the second prediction layer according to the depth information of the projection pixels comprises:
 triangulating the projection pixels in the target image, and determining at least one triangular region formed by the projection pixels in the target image; and 
 performing linear interpolation on the pixels in the at least one triangular region according to the depth information of the projection pixels, and determining the initial depth information of the pixels in the target image in the second prediction layer according to interpolation results. 
 
     
     
       5. The method according to  claim 1 , wherein the performing optimization propagation on the initial depth information of the pixels in the target image according to the similarity between corresponding pixels and determining the second depth information of the pixels in the target image output by the second prediction layer comprises:
 determining a match pixel, which corresponds to a current pixel in the target image, in the match image according to the correspondence relationship; 
 determining current initial depth information of the current pixel and adjacent initial depth information of an adjacent pixel of the current pixel in the target image; 
 determining first similarity between the current pixel and the match pixel when the current pixel has the current initial depth information and second similarity between the current pixel and the match pixel when the current pixel has the adjacent initial depth information; and 
 determining second depth information of the current pixel output by the second prediction layer according to the first similarity and the second similarity. 
 
     
     
       6. The method according to  claim 5 , wherein determining the second depth information of the current pixel output by the second prediction layer according to the first similarity and the second similarity comprises:
 in a case where the first similarity is greater than the second similarity, using the current initial depth information as optimization depth information of the current pixel, and in a case where the first similarity is less than the second similarity, using the adjacent initial depth information as optimization depth information of the current pixel; and 
 determining the second depth information of the current pixel output by the second prediction layer according to the optimization depth information. 
 
     
     
       7. The method according to  claim 6 , wherein the determining the second depth information of the current pixel output by the second prediction layer according to the optimization depth information comprises:
 determining a depth information interval according to the optimization depth information, and determining at least one piece of adjustment depth information according to the depth information interval; 
 determining third similarity between the current pixel and the match pixel when the current pixel has each piece of the adjustment depth information; and 
 determining the second depth information of the current pixel output by the second prediction layer according to the third similarity and the each piece of the adjustment depth information. 
 
     
     
       8. An electronic device, comprising:
 at least one processor; and 
 a memory communicatively connected to the at least one processor, 
 wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to execute a method for determining image depth information, wherein the method comprises: 
 acquiring first depth information of pixels in a target image output by a first prediction layer; 
 generating a point cloud model of the target image according to the first depth information, and determining initial depth information of the pixels in the target image in a second prediction layer according to the point cloud model; and 
 performing propagation optimization according to the initial depth information, and determining second depth information of the pixels in the target image output by the second prediction layer, 
 wherein the first prediction layer is configured before the second prediction layers; 
 wherein the performing propagation optimization according to the initial depth information, and determining the second depth information of the pixels in the target image output by the second prediction layer comprises: 
 determining a match image matched with the target image from an image sequence to which the target image belongs by using a binocular vision algorithm, wherein the image sequence is obtained by collecting a same object; 
 determining a correspondence relationship between the pixels in the target image and pixels in the match image according to extrinsics information and intrinsics information of a camera that collects the target image and extrinsics information and intrinsics information of a camera that collects the match image; and 
 performing optimization propagation on the initial depth information of the pixels in the target image according to similarity between corresponding pixels, and determining the second depth information of the pixels in the target image output by the second prediction layer. 
 
     
     
       9. The electronic device according to  claim 8 , wherein generating the point cloud model of the target image according to the first depth information comprises:
 acquiring image coordinate information of the pixels in the target image and intrinsics information and extrinsics information of a camera that collects the target image; and 
 determining world coordinate information of the pixels in the target image according to the image coordinate information, the intrinsics information, the extrinsics information, and the first depth information, and generating the point cloud model of the target image according to the world coordinate information. 
 
     
     
       10. The electronic device according to  claim 8 , wherein the determining the initial depth information of the pixels in the target image in the second prediction layer according to the point cloud model comprises:
 determining a projection pixel of each point in the point cloud model in the target image; and 
 determining the initial depth information of the pixels in the target image in the second prediction layer according to depth information of projection pixels. 
 
     
     
       11. The electronic device according to  claim 10 , wherein the determining the initial depth information of the pixels in the target image in the second prediction layer according to the depth information of the projection pixels comprises:
 triangulating the projection pixels in the target image, and determining at least one triangular region formed by the projection pixels in the target image; and 
 performing linear interpolation on the pixels in the at least one triangular region according to the depth information of the projection pixels, and determining the initial depth information of the pixels in the target image in the second prediction layer according to interpolation results. 
 
     
     
       12. The electronic device according to  claim 8 , wherein the performing optimization propagation on the initial depth information of the pixels in the target image according to the similarity between corresponding pixels and determining the second depth information of the pixels in the target image output by the second prediction layer comprises:
 determining a match pixel, which corresponds to a current pixel in the target image, in the match image according to the correspondence relationship; 
 determining current initial depth information of the current pixel and adjacent initial depth information of an adjacent pixel of the current pixel in the target image; 
 determining first similarity between the current pixel and the match pixel when the current pixel has the current initial depth information and second similarity between the current pixel and the match pixel when the current pixel has the adjacent initial depth information; and 
 determining second depth information of the current pixel output by the second prediction layer according to the first similarity and the second similarity. 
 
     
     
       13. The electronic device according to  claim 12 , wherein determining the second depth information of the current pixel output by the second prediction layer according to the first similarity and the second similarity comprises:
 in a case where the first similarity is greater than the second similarity, using the current initial depth information as optimization depth information of the current pixel, and in a case where the first similarity is less than the second similarity, using the adjacent initial depth information as optimization depth information of the current pixel; and 
 determining the second depth information of the current pixel output by the second prediction layer according to the optimization depth information. 
 
     
     
       14. The electronic device according to  claim 13 , wherein the determining the second depth information of the current pixel output by the second prediction layer according to the optimization depth information comprises:
 determining a depth information interval according to the optimization depth information, and determining at least one piece of adjustment depth information according to the depth information interval; 
 determining third similarity between the current pixel and the match pixel when the current pixel has each piece of the adjustment depth information; and 
 determining the second depth information of the current pixel output by the second prediction layer according to the third similarity and the each piece of the adjustment depth information. 
 
     
     
       15. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute a method for determining image depth information, wherein the method comprises:
 acquiring first depth information of pixels in a target image output by a first prediction layer; 
 generating a point cloud model of the target image according to the first depth information, and determining initial depth information of the pixels in the target image in a second prediction layer according to the point cloud model; and 
 performing propagation optimization according to the initial depth information, and determining second depth information of the pixels in the target image output by the second prediction layer, 
 wherein the first prediction layer is configured before the second prediction layers; 
 wherein the performing propagation optimization according to the initial depth information, and determining the second depth information of the pixels in the target image output by the second prediction layer comprises: 
 determining a match image matched with the target image from an image sequence to which the target image belongs by using a binocular vision algorithm, wherein the image sequence is obtained by collecting a same object; 
 determining a correspondence relationship between the pixels in the target image and pixels in the match image according to extrinsics information and intrinsics information of a camera that collects the target image and extrinsics information and intrinsics information of a camera that collects the match image; and 
 performing optimization propagation on the initial depth information of the pixels in the target image according to similarity between corresponding pixels, and determining the second depth information of the pixels in the target image output by the second prediction layer. 
 
     
     
       16. The non-transitory computer-readable storage medium according to  claim 15 , wherein generating the point cloud model of the target image according to the first depth information comprises:
 acquiring image coordinate information of the pixels in the target image and intrinsics information and extrinsics information of a camera that collects the target image; and 
 determining world coordinate information of the pixels in the target image according to the image coordinate information, the intrinsics information, the extrinsics information, and the first depth information, and generating the point cloud model of the target image according to the world coordinate information. 
 
     
     
       17. The non-transitory computer-readable storage medium according to  claim 15 , wherein the determining the initial depth information of the pixels in the target image in the second prediction layer according to the point cloud model comprises:
 determining a projection pixel of each point in the point cloud model in the target image; and 
 determining the initial depth information of the pixels in the target image in the second prediction layer according to depth information of projection pixels. 
 
     
     
       18. The non-transitory computer-readable storage medium according to  claim 17 , wherein the determining the initial depth information of the pixels in the target image in the second prediction layer according to the depth information of the projection pixels comprises:
 triangulating the projection pixels in the target image, and determining at least one triangular region formed by the projection pixels in the target image; and 
 performing linear interpolation on the pixels in the at least one triangular region according to the depth information of the projection pixels, and determining the initial depth information of the pixels in the target image in the second prediction layer according to interpolation results.