Skeleton detection based on local short-connection unidirectional fusion networks

doi:10.7523/j.ucas.2021.0048

Abstract

Abstract: In recent years, skeleton detection based on side-output network has shown significant effectiveness. However, the existing methods are still unable to tackle the problem of image distortion in side-output structure caused by high-multiplier up/down-sampling, and the fixed receptive field limits the feature expression of the networks. To solve these problems, this paper proposes a local short-connection unidirectional fusion network based on side-output connection, which includes a feature extraction network and a side-output connection network. The feature extraction network is a deep convolutional neural network, which is used for multi-layer feature extraction. The side-output connection network consists of a local short-connection unit and a unidirectional fusion network. The local short-connection gradually constructs the continuous large receptive field features by integrating the adjacent features of receptive field, while the unidirectional fusion of multi-scale features from deep to shallow can achieve the characterization of the object from rough to fine. Experimental results on four commonly used skeleton detection datasets demonstrate the effectiveness of the proposed method.

Key words: skeleton detection, local short-connection unidirectional fusion network, side-output network

CLC Number:

TP391

QIAO Yang, XIAO Shixiang, LIU Yue, JIAO Jianbin. Skeleton detection based on local short-connection unidirectional fusion networks[J]. Journal of University of Chinese Academy of Sciences, 2023, 40(2): 250-257.

References

[1] Wei S H, Ramakrishna V, Kanade T, et al. Convolutional pose machines[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 27-30, 2016, Las Vegas, NV, USA. IEEE, 2016:4724-4732.
[2] Teo C L, Fermüller C, Aloimonos Y. Detection and segmentation of 2D curved reflection symmetric structures[C]//2015 IEEE International Conference on Computer Vision (ICCV). December 7-13, 2015, Santiago, Chile. IEEE, 2015:1644-1652.
[3] Zhang Z, Shen W, Yao C, et al. Symmetry-based text line detection in natural scenes[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 7-12, 2015, Boston, MA, USA. IEEE, 2015:2558-2567.
[4] Lee T, Fidler S, Dickinson S. Learning to combine mid-level cues for object proposal generation[C]//2015 IEEE International Conference on Computer Vision (ICCV). December 7-13, 2015, Santiago, Chile. IEEE, 2015:1680-1688.
[5] Lam L, Lee S W, Suen C Y. Thinning methodologies-a comprehensive survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1992, 14(9):869-885.
[6] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6):84-90.
[7] Xie S N, Tu Z W. Holistically-nested edge detection[C]//2015 IEEE International Conference on Computer Vision (ICCV). December 7-13, 2015, Santiago, Chile. IEEE, 2015:1395-1403.
[8] Tsogkas S, Kokkinos I. Learning-based symmetry detection in natural images[M]//Computer Vision-ECCV 2012. Berlin, Heidelberg:Springer, 2012:41-54.
[9] Ke W, Chen J, Jiao J B, et al. SRN:side-output residual network for object symmetry detection in the wild[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26, 2017, Honolulu, HI, USA. IEEE, 2017:302-310.
[10] Liu C, Ke W, Qin F, et al. Linear span network for object skeleton detection[M]//Computer Vision-ECCV 2018. Cham:Springer International Publishing, 2018:136-151.
[11] Hou Q B, Cheng M M, Hu X W, et al. Deeply supervised salient object detection with short connections[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(4):815-828.
[12] Wang Y K, Xu Y C, Tsogkas S, et al. DeepFlux for skeletons in the wild[C]//2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 15-20, 2019, Long Beach, CA, USA. IEEE, 2019:5282-5291.
[13] Qiao Y, Tian Y J, Liu Y, et al. Genetic feature fusion for object skeleton detection[J]. Security and Communication Networks, 2021, 2021:1-9.
[14] Zhao K, Shen W, Gao S H, et al. Hi-fi:hierarchical feature integration for skeleton detection[C]//Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence. July 13-19, 2018. Stockholm, Sweden. California:International Joint Conferences on Artificial Intelligence Organization, 2018:1191-1197.
[15] Shen W, Zhao K, Jiang Y, et al. Object skeleton extraction in natural images by fusing scale-associated deep side outputs[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 27-30, 2016, Las Vegas, NV, USA. IEEE, 2016:222-230.
[16] Blake R, Logothetis N K. Visual competition[J]. Nature Reviews Neuroscience, 2002, 3(1):13-21.
[17] Wang Y L, Lv K C, Huang R, et al. Glance and focus:a dynamic approach to reducing spatial redundancy in image classification[EB/OL]. arXiv:2010.05300. (2020-10-11)[2021-06-01]. https://arxiv.org/abs/2010.05300.
[18] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[EB/OL]. arXiv:1409.1556. (2015-04-10)[2021-06-01]. https://arxiv.org/abs/1409.1556.
[19] Shen W, Bai X, Hu Z H, et al. Multiple instance subspace learning via partial random projection tree for local reflection symmetry in natural images[J]. Pattern Recognition, 2016, 52:306-316.
[20] Shen W, Zhao K, Jiang Y, et al. DeepSkeleton:learning multi-task scale-associated deep side outputs for object skeleton extraction in natural images[J]. IEEE Transactions on Image Processing, 2017, 26(11):5298-5311.
[21] Lin T Y, Maire M, Belongie S, et al. Microsoft COCO:common objects in context[M]//Computer Vision-ECCV 2014. Cham:Springer International Publishing, 2014:740-755.
[22] Everingham M, Gool L, Williams C K I, et al. The pascal visual object classes (VOC) challenge[J]. International Journal of Computer Vision, 2010, 88(2):303-338.
[23] Deng J, Dong W, Socher R, et al. ImageNet:a large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 20-25, 2009. Miami, FL, USA. IEEE, 2009:248-255.
[24] Hubara I, Courbariaux M, Soudry D, et al. Quantized neural networks:training neural networks with low precision weights and activations[EB/OL]. arXiv:1609.07061. (2016-09-22)[2021-06-01]. https://arxiv.org/abs/1609.07061.
[25] He K M, Zhang X Y, Ren S Q, et al. Delving deep into rectifiers:surpassing human-level performance on ImageNet classification[C]//2015 IEEE International Conference on Computer Vision (ICCV). December 7-13, 2015, Santiago, Chile. IEEE, 2015:1026-1034.