基于局部短接单向融合网络的骨架检测

doi:10.7523/j.ucas.2021.0048

摘要/Abstract

摘要： 近年来，基于侧输出网络的骨架检测方法获得了显著的性能提升。但是，现有方法仍无法解决侧输出结构中高倍上采样和下采样带来的图像失真问题，固定的感受野大小也限制了其视觉特征表达能力。为解决这些问题，提出一种基于侧输出连接的局部短接单向融合网络。该网络由特征提取网络和侧输出网络组成。特征提取网络为深度卷积神经网络，主要用于多层次视觉特征提取。侧输出网络包含局部短接网络和单项融合网络2个模块，其中局部短接网络通过整合感受野邻近特征逐步构建起连续的大感受野特征，而多尺度特征从深到浅的单向融合则实现了对目标从粗糙到精细的刻画。在4种常用骨架检测数据集上的实验结果验证了所提方法的有效性。

关键词: 骨架检测, 局部短接单向融合网络, 侧输出网络

Abstract: In recent years, skeleton detection based on side-output network has shown significant effectiveness. However, the existing methods are still unable to tackle the problem of image distortion in side-output structure caused by high-multiplier up/down-sampling, and the fixed receptive field limits the feature expression of the networks. To solve these problems, this paper proposes a local short-connection unidirectional fusion network based on side-output connection, which includes a feature extraction network and a side-output connection network. The feature extraction network is a deep convolutional neural network, which is used for multi-layer feature extraction. The side-output connection network consists of a local short-connection unit and a unidirectional fusion network. The local short-connection gradually constructs the continuous large receptive field features by integrating the adjacent features of receptive field, while the unidirectional fusion of multi-scale features from deep to shallow can achieve the characterization of the object from rough to fine. Experimental results on four commonly used skeleton detection datasets demonstrate the effectiveness of the proposed method.

Key words: skeleton detection, local short-connection unidirectional fusion network, side-output network

中图分类号:

TP391

乔杨, 肖士湘, 刘悦, 焦建彬. 基于局部短接单向融合网络的骨架检测[J]. 中国科学院大学学报, 2023, 40(2): 250-257.

QIAO Yang, XIAO Shixiang, LIU Yue, JIAO Jianbin. Skeleton detection based on local short-connection unidirectional fusion networks[J]. Journal of University of Chinese Academy of Sciences, 2023, 40(2): 250-257.

参考文献

[1] Wei S H, Ramakrishna V, Kanade T, et al. Convolutional pose machines[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 27-30, 2016, Las Vegas, NV, USA. IEEE, 2016:4724-4732.
[2] Teo C L, Fermüller C, Aloimonos Y. Detection and segmentation of 2D curved reflection symmetric structures[C]//2015 IEEE International Conference on Computer Vision (ICCV). December 7-13, 2015, Santiago, Chile. IEEE, 2015:1644-1652.
[3] Zhang Z, Shen W, Yao C, et al. Symmetry-based text line detection in natural scenes[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 7-12, 2015, Boston, MA, USA. IEEE, 2015:2558-2567.
[4] Lee T, Fidler S, Dickinson S. Learning to combine mid-level cues for object proposal generation[C]//2015 IEEE International Conference on Computer Vision (ICCV). December 7-13, 2015, Santiago, Chile. IEEE, 2015:1680-1688.
[5] Lam L, Lee S W, Suen C Y. Thinning methodologies-a comprehensive survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1992, 14(9):869-885.
[6] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6):84-90.
[7] Xie S N, Tu Z W. Holistically-nested edge detection[C]//2015 IEEE International Conference on Computer Vision (ICCV). December 7-13, 2015, Santiago, Chile. IEEE, 2015:1395-1403.
[8] Tsogkas S, Kokkinos I. Learning-based symmetry detection in natural images[M]//Computer Vision-ECCV 2012. Berlin, Heidelberg:Springer, 2012:41-54.
[9] Ke W, Chen J, Jiao J B, et al. SRN:side-output residual network for object symmetry detection in the wild[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26, 2017, Honolulu, HI, USA. IEEE, 2017:302-310.
[10] Liu C, Ke W, Qin F, et al. Linear span network for object skeleton detection[M]//Computer Vision-ECCV 2018. Cham:Springer International Publishing, 2018:136-151.
[11] Hou Q B, Cheng M M, Hu X W, et al. Deeply supervised salient object detection with short connections[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(4):815-828.
[12] Wang Y K, Xu Y C, Tsogkas S, et al. DeepFlux for skeletons in the wild[C]//2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 15-20, 2019, Long Beach, CA, USA. IEEE, 2019:5282-5291.
[13] Qiao Y, Tian Y J, Liu Y, et al. Genetic feature fusion for object skeleton detection[J]. Security and Communication Networks, 2021, 2021:1-9.
[14] Zhao K, Shen W, Gao S H, et al. Hi-fi:hierarchical feature integration for skeleton detection[C]//Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence. July 13-19, 2018. Stockholm, Sweden. California:International Joint Conferences on Artificial Intelligence Organization, 2018:1191-1197.
[15] Shen W, Zhao K, Jiang Y, et al. Object skeleton extraction in natural images by fusing scale-associated deep side outputs[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 27-30, 2016, Las Vegas, NV, USA. IEEE, 2016:222-230.
[16] Blake R, Logothetis N K. Visual competition[J]. Nature Reviews Neuroscience, 2002, 3(1):13-21.
[17] Wang Y L, Lv K C, Huang R, et al. Glance and focus:a dynamic approach to reducing spatial redundancy in image classification[EB/OL]. arXiv:2010.05300. (2020-10-11)[2021-06-01]. https://arxiv.org/abs/2010.05300.
[18] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[EB/OL]. arXiv:1409.1556. (2015-04-10)[2021-06-01]. https://arxiv.org/abs/1409.1556.
[19] Shen W, Bai X, Hu Z H, et al. Multiple instance subspace learning via partial random projection tree for local reflection symmetry in natural images[J]. Pattern Recognition, 2016, 52:306-316.
[20] Shen W, Zhao K, Jiang Y, et al. DeepSkeleton:learning multi-task scale-associated deep side outputs for object skeleton extraction in natural images[J]. IEEE Transactions on Image Processing, 2017, 26(11):5298-5311.
[21] Lin T Y, Maire M, Belongie S, et al. Microsoft COCO:common objects in context[M]//Computer Vision-ECCV 2014. Cham:Springer International Publishing, 2014:740-755.
[22] Everingham M, Gool L, Williams C K I, et al. The pascal visual object classes (VOC) challenge[J]. International Journal of Computer Vision, 2010, 88(2):303-338.
[23] Deng J, Dong W, Socher R, et al. ImageNet:a large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 20-25, 2009. Miami, FL, USA. IEEE, 2009:248-255.
[24] Hubara I, Courbariaux M, Soudry D, et al. Quantized neural networks:training neural networks with low precision weights and activations[EB/OL]. arXiv:1609.07061. (2016-09-22)[2021-06-01]. https://arxiv.org/abs/1609.07061.
[25] He K M, Zhang X Y, Ren S Q, et al. Delving deep into rectifiers:surpassing human-level performance on ImageNet classification[C]//2015 IEEE International Conference on Computer Vision (ICCV). December 7-13, 2015, Santiago, Chile. IEEE, 2015:1026-1034.