Accuracy analysis of 3D object detection based on stereo point cloud

doi:10.7523/j.ucas.2020.0042

Abstract

Abstract: 3D object detection is a crucial task in autonomous driving. Recently, the accuracy of LiDAR-based 3D object detection algorithms have improved dramatically. However, if we replace the LiDAR data with the depth map that generates from stereo cameras, the detection performance drops a lot. In a recent research, it is found that by transforming the stereo-based depth map to point cloud representation, referred to as pseudo-LiDAR, the performance of 3D object detection can be improved significantly. However, there is still a big gap of accuracy between LiDAR-based and stereo-based algorithms. One of the main reasons is that there are error points in the transformed pseudo-LiDAR data. To study this, we use the LiDAR points cloud to correct the pseudo-LiDAR data, i.e., to detect and exclude the error points. Then we use the optimized pseudo-LiDAR to perform 3D object detection. Experimental results show that the detection accuracy can be improved by as much as 21.02%.

Key words: 3D object detection, LiDAR, stereo cameras, point cloud, error

CLC Number:

TN47

LIU Wangchao, LOU Xin. Accuracy analysis of 3D object detection based on stereo point cloud[J]. Journal of University of Chinese Academy of Sciences, 2022, 39(5): 677-683.

References

[1] Lei Z Y, Wang Y, Xu Y F, et al. From coarse to fine:a monocular depth estimation model based on left-right consistency[C]//2019 IEEE 19th International Conference on Communication Technology (ICCT). October 16-19, 2019, Xi'an, China. IEEE, 2019:1621-1625.DOI:10.1109/ICCT46805.2019.8947220.
[2] Mayer N, Ilg E, Häuusser P, et al. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 27-30, 2016, Las Vegas, NV,USA. IEEE, 2016:4040-4048.DOI:10.1109/CVPR.2016.438.
[3] Chen X Z, Kundu K, Zhu Y K, et al. 3D object proposals using stereo imagery for accurate object class detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(5):1259-1272.DOI:10.1109/TPAMI.2017.2706685.
[4] Xu B, Chen Z Z. Multi-level fusion based 3D object detection from monocular images[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 18-23, 2018, Salt Lake City, UT,USA. IEEE, 2018:2345-2353.DOI:10.1109/CVPR.2018.00249.
[5] Liang M, Yang B, Wang S L, et al. Deep continuous fusion for multi-sensor 3D object detection[C]//European Conference on Computer Vision (ECCV). Munich, Germany:2018:663-678.
[6] Wang Y, Chao W, Garg D, et al. Pseudo-LiDAR from visual depth estimation:bridging the gap in 3D object detection for autonomous driving[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 15-20, 2019, Long Beach, CA, USA. IEEE, 2019:8437-8445.DOI:10.1109/CVPR.2019.00864.
[7] Chen X, Ma H, Wan J, et al. Multi-view 3D object detection network for autonomous driving[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26,2017, Honolulu, HI, USA. IEEE, 2017:6526-6534.DOI:10.1109/CVPR.2017.691.
[8] Zhou Y, Tuzel O. VoxelNet:end-to-end learning for point cloud based 3D object detection[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). June 18-23, 2018, Salt Lake City, UT, USA. IEEE, 2018:4490-4499.DOI:10.1109/CVPR.2018.00472.
[9] Charles R Q, Su H, Mo K C, et al. PointNet:deep learning on point sets for 3D classification and segmentation[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26, 2017, Honolulu, HI, USA. IEEE, 2017:77-85.DOI:10.1109/CVPR.2017.16.
[10] Guo X Y, Yang K, Yang W K, et al. Group-wise correlation stereo network[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 15-20, 2019, Long Beach, CA, USA. IEEE, 2019:3268-3277.DOI:10.1109/CVPR.2019.00339.
[11] Wu Z Y, Wu X Y, Zhang X P, et al. Semantic stereo matching with pyramid cost volumes[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). October 27-November 2, 2019, Seoul, Korea(South):IEEE, 2019:7483-7492.DOI:10.1109/ICCV.2019.00758.
[12] Chang J R, Chen Y S. Pyramid stereo matching network[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 18-23, 2018, Salt Lake City, UT, USA. IEEE, 2018:5410-5418.DOI:10.1109/CVPR.2018.00567.
[13] Geiger A, Lenz P, Urtasun R. Are we ready for autonomous driving? The KITTI vision benchmark suite[C]//2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 16-21, 2012, Providence, RI, USA. IEEE, 2012:3354-3361.DOI:10.1109/CVPR.2012.6248074.
[14] Ku J, Mozifian M, Lee J, et al. Joint 3D proposal generation and object detection from view aggregation[C]//2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). October 1-5, 2018, Madrid, Spain. IEEE, 2018:1-8.DOI:10.1109/IROS.2018.8594049.
[15] Qi C R, Liu W, Wu C X, et al. Frustum PointNets for 3D object detection from RGB-D data[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 18-23, 2018, Salt Lake City, UT, USA. IEEE, 2018:918-927. DOI:10.1109/CVPR.2018.00102.