Point cloud geometry compression

Abstract

A method for learning-based point cloud geometry compression includes: given a source point cloud, regressing an aligned mesh that is driven by a set of parameters from a deformable template mesh, quantizing the set of parameters into a parameter bitstream, generating an aligned point cloud from the quantized parameters by mesh manipulation and mesh-to-point-cloud conversion, extracting features from both the source point cloud and the aligned point cloud based on sparse tensors including coordinates and features, the coordinates being encoded into a coordinate bitstream, warping the features of the aligned point cloud onto the coordinates of the source point cloud, obtaining residual features through feature subtraction, processing the residual features using an entropy model into a residual feature bitstream, and obtaining a reconstructed point cloud by processing the parameter bitstream, the coordinate bitstream and the residual feature bitstream.

The invention claimed is: 
     
       1. A method for learning-based point cloud geometry compression, comprising:
 given a source point cloud, regressing an aligned mesh that is driven by a set of parameters from a deformable template mesh; 
 quantizing the set of parameters into a parameter bitstream; 
 generating an aligned point cloud from the quantized parameters by mesh manipulation and mesh-to-point-cloud conversion; 
 extracting features from the source point cloud to provide features of the source point cloud and extracting features from the aligned point cloud to provide features of the aligned point cloud based on sparse tensors comprising coordinates and features, the coordinates being encoded into a coordinate bitstream; 
 warping the features of the aligned point cloud onto the coordinates of the source point cloud to provide warped features of the aligned point cloud; 
 obtaining residual features through feature subtraction; 
 processing the residual features using an entropy model into a residual feature bitstream; and 
 obtaining a reconstructed point cloud by processing the parameter bitstream, the coordinate bitstream and the residual feature bitstream. 
 
     
     
       2. The method of  claim 1 , wherein generating the aligned point cloud comprises:
 recovering the aligned mesh from the quantized parameters in the mesh manipulation; and 
 processing the aligned mesh by mesh-to-point-cloud conversion. 
 
     
     
       3. The method of  claim 1 , wherein extracting the features from both the source point cloud and the aligned point cloud comprises using stacked downsampling blocks. 
     
     
       4. The method of  claim 3 , wherein each downsampling block comprises a strided convolution unit, a Voxception-ResNet (VRN) unit, and another convolution layer, arranged in a cascading manner. 
     
     
       5. The method of  claim 3 , wherein using the stacked downsampling blocks outputs multiscale sparse tensors. 
     
     
       6. The method of  claim 1 , wherein the feature extraction and the feature warping can be applied in a plug-and-play fashion with one or more methods. 
     
     
       7. The method of  claim 6 , wherein the one or more methods for the feature extraction and the feature warping comprises a method based on a deep point cloud compression using sparse convolution. 
     
     
       8. The method of  claim 1 , wherein obtaining the residual features comprises subtracting the warped features of the aligned point cloud from the features of the source point cloud to obtain the residual features. 
     
     
       9. The method of  claim 1 , wherein the residual features have entropy, and processing the residual features comprises compressing the residual features by vector quantization on original signal space and estimating the entropy of the residual features. 
     
     
       10. The method of  claim 1 , wherein obtaining the reconstructed point cloud comprises:
 decoding the parameter bitstream to use the set of parameters to manipulate the deformable template mesh; 
 producing an aligned point cloud by mesh-to-point-cloud conversion from a reconstructed aligned mesh; 
 predicting features from the aligned point cloud; 
 warping the predicted features onto a decoded set of coordinates corresponding to skeleton points of a source point cloud; 
 decoding the residual feature bitstream to the residual features; and 
 adding the residual features to the obtained warped features to recover the features of the source point cloud. 
 
     
     
       11. The method of  claim 10 , wherein obtaining the reconstructed point cloud further comprises conducting feature propagation on the recovered features of the source point cloud to upscale points of the recovered features of the source point cloud close to the source point cloud. 
     
     
       12. The method of  claim 11 , wherein conducting the feature propagation comprises employing a transposed convolution layer with a two-stride in each upsampling block to upscale an input coordinate set while retaining its sparsity pattern. 
     
     
       13. A system for learning-based point cloud geometry compression, comprising:
 one or more processors; and 
 memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing or facilitating performing of the method of  claim 1 . 
 
     
     
       14. A non-transitory computer readable medium having instructions stored thereon which, when executed by one or more processors, cause the one or more processors to execute the method of  claim 1 .