双目相机虽然能通过算法生成密集的深度数据，但其在精度上与激光雷达生成的深度数据有着较大的差距，特别是在纹理不明显的区域。针对这种现象，尝试使用激光雷达的精确点云来排除由双目相机产生的pseudo-LiDAR数据中与之差别较大的部分点，然后将优化后的pseudo-LiDAR用以进行三维物体检测。实验结果显示，将pseudo-LiDAR数据中的不准确点（坏点）排除，有助于提高检测准确率，最多可提高21.02%。因此，如何不依赖激光雷达数据来排除pseudo-LiDAR点云中的坏点是进一步提高双目相机系统检测准确率的关键。
3D object detection is a crucial task in autonomous driving. Recently, the accuracy of LiDAR-based 3D object detection algorithms have improved dramatically. However, if we replace the LiDAR data with the depth map that generates from stereo cameras, the detection performance drops a lot. In a recent research, it is found that by transforming the stereo-based depth map to point cloud representation, referred to as pseudo-LiDAR, the performance of 3D object detection can be improved significantly. However, there is still a big gap of accuracy between LiDAR-based and stereo-based algorithms. One of the main reasons is that there are error points in the transformed pseudo-LiDAR data. To study this, we use the LiDAR points cloud to correct the pseudo-LiDAR data, i.e., to detect and exclude the error points. Then we use the optimized pseudo-LiDAR to perform 3D object detection. Experimental results show that the detection accuracy can be improved by as much as 21.02%.
[1] Lei Z Y, Wang Y, Xu Y F, et al. From coarse to fine:a monocular depth estimation model based on left-right consistency[C]//2019 IEEE 19th International Conference on Communication Technology (ICCT). October 16-19, 2019, Xi'an, China. IEEE, 2019:1621-1625.DOI:10.1109/ICCT46805.2019.8947220.
[2] Mayer N, Ilg E, Häuusser P, et al. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 27-30, 2016, Las Vegas, NV,USA. IEEE, 2016:4040-4048.DOI:10.1109/CVPR.2016.438.
[3] Chen X Z, Kundu K, Zhu Y K, et al. 3D object proposals using stereo imagery for accurate object class detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(5):1259-1272.DOI:10.1109/TPAMI.2017.2706685.
[4] Xu B, Chen Z Z. Multi-level fusion based 3D object detection from monocular images[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 18-23, 2018, Salt Lake City, UT,USA. IEEE, 2018:2345-2353.DOI:10.1109/CVPR.2018.00249.
[5] Liang M, Yang B, Wang S L, et al. Deep continuous fusion for multi-sensor 3D object detection[C]//European Conference on Computer Vision (ECCV). Munich, Germany:2018:663-678.
[6] Wang Y, Chao W, Garg D, et al. Pseudo-LiDAR from visual depth estimation:bridging the gap in 3D object detection for autonomous driving[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 15-20, 2019, Long Beach, CA, USA. IEEE, 2019:8437-8445.DOI:10.1109/CVPR.2019.00864.
[7] Chen X, Ma H, Wan J, et al. Multi-view 3D object detection network for autonomous driving[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26,2017, Honolulu, HI, USA. IEEE, 2017:6526-6534.DOI:10.1109/CVPR.2017.691.
[8] Zhou Y, Tuzel O. VoxelNet:end-to-end learning for point cloud based 3D object detection[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). June 18-23, 2018, Salt Lake City, UT, USA. IEEE, 2018:4490-4499.DOI:10.1109/CVPR.2018.00472.
[9] Charles R Q, Su H, Mo K C, et al. PointNet:deep learning on point sets for 3D classification and segmentation[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26, 2017, Honolulu, HI, USA. IEEE, 2017:77-85.DOI:10.1109/CVPR.2017.16.
[10] Guo X Y, Yang K, Yang W K, et al. Group-wise correlation stereo network[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 15-20, 2019, Long Beach, CA, USA. IEEE, 2019:3268-3277.DOI:10.1109/CVPR.2019.00339.
[11] Wu Z Y, Wu X Y, Zhang X P, et al. Semantic stereo matching with pyramid cost volumes[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). October 27-November 2, 2019, Seoul, Korea(South):IEEE, 2019:7483-7492.DOI:10.1109/ICCV.2019.00758.
[12] Chang J R, Chen Y S. Pyramid stereo matching network[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 18-23, 2018, Salt Lake City, UT, USA. IEEE, 2018:5410-5418.DOI:10.1109/CVPR.2018.00567.
[13] Geiger A, Lenz P, Urtasun R. Are we ready for autonomous driving? The KITTI vision benchmark suite[C]//2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 16-21, 2012, Providence, RI, USA. IEEE, 2012:3354-3361.DOI:10.1109/CVPR.2012.6248074.
[14] Ku J, Mozifian M, Lee J, et al. Joint 3D proposal generation and object detection from view aggregation[C]//2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). October 1-5, 2018, Madrid, Spain. IEEE, 2018:1-8.DOI:10.1109/IROS.2018.8594049.
[15] Qi C R, Liu W, Wu C X, et al. Frustum PointNets for 3D object detection from RGB-D data[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 18-23, 2018, Salt Lake City, UT, USA. IEEE, 2018:918-927. DOI:10.1109/CVPR.2018.00102.
