Point cloud compression of deep learning based on multi-scale feature and attention mechanism

doi:10.7523/j.ucas.2023.077

Abstract

Abstract: 3D point clouds have extensive applications in the auto-drive, 3D real scene, and other fields. But complex scene requires massive point clouds to represent which brings great challenges to storage space, data processing and transmission bandwidth. A multi-scale attention point cloud geometry compression (MSA-GPCC) is proposed to compress point cloud data based on multi-scale features, attention mechanism, and variational auto-encoders (VAE). Experiments and analysis are carried out based on MPEG data sets. The results show that MSA-GPCC performs better than those of the traditional G-PCC and deep-learning-based D-PCC algorithms, D₁ BD-PSNR is improved by 7.72 and 4.91 dB respectively, and D₂ BD-PSNR is improved by 5.56 and 3.09 dB respectively.

Key words: point cloud compression, deep learning, attention mechanism, variational auto-encoders, multi-scale feature

CLC Number:

TP391.41

HUANG Yulin, LIANG Lei, LI Weijun, XI Xiaohuan. Point cloud compression of deep learning based on multi-scale feature and attention mechanism[J]. Journal of University of Chinese Academy of Sciences, 2024, 41(5): 687-694.

References

[1] 杨必胜,梁福逊,黄荣刚.三维激光扫描点云数据处理研究进展、挑战与趋势[J].测绘学报, 2017, 46(10):1509-1516. DOI:10.11947/j.AGCS.2017.20170351.
[2] Schnabel R, Klein R. Octree-based point-cloud compression[C]//IEEE VGTC Conference on Point-Based Graphics. July 29, 2006, Goslar, Germany. IEEE VGTC, 2006:111-120. DOI:10.2312/SPBG/SPBG06/111-120.
[3] Kammerl J, Blodow N, Rusu R B, et al. Real-time compression of point cloud streams[C]//2012 IEEE International Conference on Robotics and Automation. May 14-18, 2012, Saint Paul, MN, USA. IEEE, 2012:778-785. DOI:10.1109/ICRA.2012.6224647.
[4] 律帅,达飞鹏,黄源.基于数据类型转换的点云快速有损压缩算法[J].图学学报, 2016, 37(2):199-205. DOI:10.11996/JG.j.2095-302X.2016020199.
[5] 冯燕,何明一,魏江.基于神经网络的多光谱遥感图像无损压缩[J].遥感技术与应用, 2004, 19(1):42-46. DOI:10.3969/j.issn.1004-0323.2004.01.010.
[6] Yan W, Shao Y, Liu S, et al. "Deep autoencoder-based lossy geometry compression for point clouds" [EB/OL]. 2019:arXiv:1905.03691.(2019-04-18)[2023-03-06]. https://arxiv.org/abs/1905.03691.
[7] Huang L L, Wang S L, Wong K, et al. OctSqueeze:octree-structured entropy model for LiDAR compression[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 13-19, 2020, Seattle, WA, USA. IEEE, 2020:1310-1320. DOI:10.1109/CVPR42600.2020.00139.
[8] Biswas S, Liu J, Wong K, et al. "MuSCLE:multi sweep compression of LiDAR using deep entropy models" [EB/OL]. 2020:arXiv:2011.07590.(2020-11-15)[2023-03-06]. https://arxiv.org/abs/2011.07590.
[9] Quach M, Valenzise G, Dufaux F. Learning convolutional transforms for lossy point cloud geometry compression[C]//2019 IEEE International Conference on Image Processing (ICIP). September 22-25, 2019, Taipei, China. IEEE, 2019:4320-4324. DOI:10.1109/ICIP.2019.8803413.
[10] Guarda A F R, Rodrigues N M M, Pereira F. Point cloud coding:adopting a deep learning-based approach[C]//2019 Picture Coding Symposium (PCS). November 12-15, 2019, Ningbo, China. IEEE, 2020:1-5. DOI:10.1109/PCS48520.2019.8954537.
[11] 徐嘉诚,方志军,黄勃,等.端到端优化的3D点云几何信息有损压缩模型[J].武汉大学学报(理学版), 2022, 68(3):297-303. DOI:10.14188/j.1671-8836.2021.0083.
[12] Wang J Q, Ding D D, Li Z, et al. Multiscale point cloud geometry compression[C]//2021 Data Compression Conference (DCC). March 23-26, 2021, Snowbird, UT, USA. IEEE, 2021:73-82. DOI:10.1109/DCC50243.2021.00015.
[13] Tu C X, Takeuchi E, Carballo A, et al. Point cloud compression for 3D LiDAR sensor using recurrent neural network with residual blocks[C]//2019 International Conference on Robotics and Automation (ICRA). May 20-24, 2019, Montreal, QC, Canada. IEEE, 2019:3274-3280. DOI:10.1109/ICRA.2019.8794264.
[14] Mentzer F, Agustsson E, Tschannen M, et al. Practical full resolution learned lossless image compression[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 15-20, 2019, Long Beach, CA, USA. IEEE, 2020:10621-10630. DOI:10.1109/CVPR.2019.01088.
[15] Wu Z R, Song S R, Khosla A, et al. 3D ShapeNets:a deep representation for volumetric shapes[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 7-12, 2015, Boston, MA USA. IEEE, 2015:1912-1920. DOI:10.1109/CVPR.2015.7298801.
[16] Loop C, Cai Q, Escolano S O, et al. Jpeg pleno database:Microsoft voxelized upper bodies-a voxelized point cloud dataset[J]. ISO/IEC JTC1/SC29 Joint WG11/WG1(MPEG/JPEG) input document m38673/M72012, 2021.
[17] Krivokuca M, Chou P A, Savill P. 8i voxelized surface light field (8iVSLF) dataset[J]. ISO/IEC JTC1/SC29/WG11 MPEG, input document m42914, 2018.
[18] Graziosi D, Nakagami O, Kuma S, et al. An overview of ongoing point cloud compression standardization activities:video-based (V-PCC) and geometry-based (G-PCC)[J]. APSIPA Transactions on Signal and Information Processing, 2020, 9(1):1-7. DOI:10.1017/atsip.2020.12.
[19] Schwarz S, Martin-Cocher G, Flynn D, et al. Common test conditions for point cloud compression[J]. Document ISO/IEC JTC1/SC29/WG11 w17766, Ljubljana, Slovenia, 2018.