Image dense matching algorithm combining superpixel segmentation and guided filtering

doi:10.7523/j.ucas.2023.081

Abstract

Abstract: In order to solve the problem that the existing local stereo matching method has low matching accuracy in the discontinuous region of parallax, a dense matching method combining superpixel segmentation and guided filtering is proposed in this paper. Firstly, a feature matching method was used to determine the disparity range, and the zero-mean normalized cross correlation was combined with image gray and gradient information to construct the cost calculation function.
Secondly, the label map after superpixel segmentation is used to constrain the adaptive changes of the guided filtering window shape, and the cost is aggregated. Finally, the aggregation cost is used as the data item to construct the global energy function, and the disparity map is solved by graph cut algorithm, and multi-step disparity optimization is performed on the disparity map. Experimental results show that the average mismatching rate of the proposed method is 4.84% on the standard test image set provided by Middlebury website, which is significantly better than the traditional guided filtering dense matching method and semi-global matching method.

Key words: Machine vision, dense matching, superpixel segmentation, guided filter, graph cut algorithm

CLC Number:

TP394.1

ZHANG Zheng, ZHANG Wenyi, XU Shu. Image dense matching algorithm combining superpixel segmentation and guided filtering[J]. Journal of University of Chinese Academy of Sciences, DOI: 10.7523/j.ucas.2023.081.

References

[1] Remondino F, Spera M G, Nocerino E, et al.State of the art in high density image matching[J]. The Photogrammetric Record, 2014, 29(146): 144-166. DOI: 10.1111/phor.12063.
[2] Scharstein D, Szeliski R.A taxonomy and evaluation of dense two-frame stereo correspondence algorithms[J]. International Journal of Computer Vision, 2002, 47(1): 7-42. DOI: 10. 1023/A: 1014573219977.
[3] Zhan Y L, Gu Y Z, Huang K, et al.Accurate image-guided stereo matching with efficient matching cost and disparity refinement[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2016, 26(9): 1632-1645. DOI: 10.1109/TCSVT.2015.2473375.
[4] Stentoumis C, Grammatikopoulos L, Kalisperakis I, et al.On accurate dense stereo-matching using a local adaptive multi-cost approach[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2014, 91: 29-49. DOI: 10.1016/j.isprsjprs.2014.02.006.
[5] Kordelas G A, Alexiadis D S, Daras P, et al.Enhanced disparity estimation in stereo images[J]. Image and Vision Computing, 2015, 35: 31-49. DOI: 10.1016/j.imavis.2014.12.001.
[6] Marroquin J, Mitter S, Poggio T.Probabilistic solution of ill-posed problems in computational vision[J]. Journal of the American Statistical Association, 1987, 82(397):76-89. DOI: 10.1080/01621459.1987.10478393.
[7] Yang Q X.A non-local cost aggregation method for stereo matching[C]//2012 IEEE Conference on Computer Vision and Pattern Recognition. June 16-21, 2012, Providence, RI, USA. IEEE, 2012:1402-1409. DOI: 10.1109/CVPR.2012.6247827.
[8] Boykov Y, Veksler O, Zabih R.Fast approximate energy minimization via graph cuts[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(11):1222-1239. DOI: 10.1109/34.969114.
[9] Hong L, Chen G. Segment-based stereo matching using graph cuts[C]//Proceedings of the2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. June 27 - July 2, 2004, Washington, DC, USA. IEEE, 2004: I. DOI: 10.1109/CVPR.2004.1315016.
[10] Fezza S A, Ouddane S.Fast stereo matching via graph cuts[C]//International Workshop on Systems, Signal Processing and their Applications, WOSSPA. May 9-11, 2011, Tipaza, Algeria. IEEE, 2011: 115-118. DOI: 10.1109/WOSSPA.2011.5931427.
[11] Hirschmuller H, Scharstein D.Evaluation of stereo matching costs on images with radiometric differences[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(9): 1582-1599. DOI: 10.1109/TPAMI.2008.221.
[12] Birchfield S, Tomasi C.A pixel dissimilarity measure that is insensitive to image sampling[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1998, 20(4):401-406. DOI: 10.1109/34.677269.
[13] 邱建滨, 郑茜颖, 俞金玲. 基于融合代价和自适应惩罚项系数的立体匹配[J]. 激光与光电子学进展, 2022, 59(16): 366-371. 1615008. DOI: 10.3788/LOP202259.1615008.
[14] 闫利, 王芮, 刘华, 等. 基于改进代价计算和自适应引导滤波的立体匹配[J]. 光学学报, 2018, 38(11): 257-267. DOI: 10.3788/AOS201838.1115007.
[15] Kanade T, Okutomi M.A stereo matching algorithm with an adaptive window: Theory and experiment[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1994, 16(9):920-932. DOI: 10.1109/34.310690.
[16] Hosni A, Rhemann C, Bleyer M, et al.Fast cost-volume filtering for visual correspondence and beyond[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(2):504-511. DOI: 10.1109/TPAMI.2012.156.
[17] Kendall A, Martirosyan H, Dasgupta S, et al.End-to-end learning of geometry and context for deep stereo regression[C]//2017 IEEE International Conference on Computer Vision. October 22-29, 2017, Venice, Italy. IEEE, 2017: 66-75. DOI: 10.1109/ICCV.2017.17.
[18] Zhang F H,Prisacariu V,Yang R G,et al.GA-net: Guided aggregation net for end-to-end stereo matching[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 15-20, 2019, Long Beach, CA, USA. IEEE, 2020: 185-194. DOI: 10.1109/CVPR.2019.00027.
[19] Chang J R, Chen Y S.Pyramid stereo matching network[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018, Salt Lake City, UT, USA. IEEE, 2018: 5410-5418. DOI: 10.1109/CVPR.2018.00567.
[20] Li J Y, Hu Q W, Ai M Y.RIFT: Multi-modal image matching based on radiation-variation insensitive feature transform[J]. IEEE Transactions on Image Processing, 2020, 29: 3296-3310. DOI: 10.1109/TIP.2019.2959244.
[21] Achanta R, Shaji A, Smith K, et al.SLIC superpixels compared to state-of-the-art superpixel methods[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(11): 2274-2282. DOI: 10.1109/TPAMI.2012.120.
[22] 张祖勋, 张剑清. 数字摄影测量学[M]. 2版. 武汉:武汉大学出版社, 2012: 196-199.
[23] Scharstein D, Szeliski R, Hirschmüller H. Middlebury stereo vision page[EB/OL]. (2017-11-15)[2023-9-7]. http://vision.middlebury.edu/stereo/.
[24] Yang Q Q, Ji P, Li D X, et al.Fast stereo matching using adaptive guided filtering[J]. Image and Vision Computing, 2014, 32(3):202-211. DOI: 10.1016/j.imavis.2014.01.001.
[25] Hirschmuller H.Stereo vision in structured environments by consistent semi-global matching[C]//2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. June 17-22, 2006, New York, NY, USA. IEEE, 2006: 2386-2393. DOI: 10.1109/CVPR.2006.294.
[26] Tombari F, Mattoccia S, Di Stefano L.Segmentation-based adaptive support for accurate stereo correspondence[C]//Pacific-Rim Symposium on Image and Video Technology. Berlin, Heidelberg: Springer, 2007: 427-438. DOI: 10.1007/978-3-540-77129-6_38.
[27] Kolmogorov V, Zabih R.Computing visual correspondence with occlusions using graph cuts[C]//Proceedings Eighth IEEE International Conference on Computer Vision. ICCV. July 7-14, 2001, Vancouver, BC, Canada. IEEE, 2002: 508-515. DOI: 10.1109/ICCV.2001.937668.
[28] 祝世平, 闫利那, 李政. 基于改进Census变换和动态规划的立体匹配算法[J]. 光学学报, 2016, 36(4): 216-224. DOI: 10.3788/AOS201636.0415001.