Remote sensing semantic segmentation method based on high-resolution relational graph convolutional network

doi:10.7523/j.ucas.2023.079

Abstract

Abstract: Semantic segmentation of remote sensing images is an important task in remote sensing image processing and analysis, especially in multi-category semantic segmentation. Current methods mainly revolve around convolutional neural networks, but convolution only focuses on the local information of the image while ignoring the global information. Therefore, inspired by High Resolution Network (HRNet) and Relational Graph Convolutional Network (R-GCN), this paper proposes a High-resolution Relational Graph Convolutional Network (HRGCN) for multi-category semantic segmentation. Firstly, simple linear iterative clustering (SLIC) is done on the original image, and the result is used to segment the feature map output from HRNet to obtain superpixel blocks with high homogeneity and containing multi-resolution information; then graph nodes and edges are constructed based on the superpixel blocks, and R-GCN is used to classify the graph nodes, so as to learn the long-distance dependency between different features and complete the extraction and classification of remote sensing images. The HRGCN model designed in this paper is experimented on Potsdam and Vaihingen datasets, and the results are compared with the existing methods, and the MF1 values and MIoU values are improved to certain degrees, which prove that the method has good advancement.

Key words: remote sensing image, R-GCN, HRNet, superpixel

WANG Yinda, CHEN Jiahui, PENG Ling, LI Zhaobo, Yang Lina. Remote sensing semantic segmentation method based on high-resolution relational graph convolutional network[J]. Journal of University of Chinese Academy of Sciences, DOI: 10.7523/j.ucas.2023.079.

References

[1] 许泽宇,沈占锋,李杨,等. 增强型DeepLab算法和自适应损失函数的高分辨率遥感影像分类[J]. 遥感学报,2022,26(2):406-415.DOI: 10.11834/jrs.20209200.
[2] 陈若男, 彭玲, 刘玉菲, 等. 引入空间距离信息的城郊山区道路提取与应用[J]. 中国科学院大学学报, 2022, 39(5): 658-667. DOI: 10.7523/j.ucas.2021.0004.
[3] 王寅达,彭玲,陈德跃,等. 基于改进 U-Net 模型的农业大棚遥感提取方法[J/OJ]. 中国科学院大学学报. (2023-06-21) [2023-09-15]. DOI: 10.7523/j.ucas. 2023. 060.
[4] Yu J E, Cai Y, Lyu X, et al.Boundary-guided semantic context network for water body extraction from remote sensing images[J]. Remote Sensing, 2023, 15(17): 4325. DOI: 10.3390/rs15174325.
[5] Yan G D, Jing H T, Li H, et al.Enhancing building segmentation in remote sensing images: Advanced multi-scale boundary refinement with MBR-HRNet[J]. Remote Sensing, 2023, 15(15): 3766. DOI: 10.3390/rs15153766.
[6] 李新娜, 王小鹏, 魏统艺. 自适应形态学与多尺度结合的植被区域遥感图像分割方法[J]. 激光与光电子学进展, 2022, 59(24): 240-246. DOI: 10.3788/LOP202259.2428001.
[7] 闵蕾, 高昆, 李维, 等. 光学遥感图像分割技术综述[J]. 航天返回与遥感, 2020, 41(6): 1-13. DOI: 10.3969/j.issn.1009-8518.2020.06.001.
[8] Badrinarayanan V, Kendall A, Cipolla R.SegNet: A deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495. DOI: 10.1109/TPAMI.2016.2644615.
[9] Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2015: 234-241.10.1007/978-3-319-24574-4_28
[10] Chen L C, Papandreou G, Kokkinos I, et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs[EB/OL].2014: arXiv: 1412.7062.(2014-12-22) [2023-09-15]. https://arxiv.org/abs/1412.7062.
[11] Chen L C, Papandreou G, Kokkinos I, et al.DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4): 834-848. DOI: 10.1109/TPAMI.2017.2699184.
[12] Chen L C, Papandreou G, Schroff F, et al. Rethinking atrous convolution for semantic image segmentation[EB/OL].2017: arXiv:1706.05587.(2017-06-17) [2023-09-15]. https://arxiv.org/abs/1706.05587.
[13] Chen L C, Zhu Y K, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[EB/OL].2018: arXiv: 1802.02611.(2018-02-07) [2023-09-15]. https://arxiv.org/abs/1802.02611.
[14] Chen F L, Liu H J, Zeng Z H, et al.BES-net: Boundary enhancing semantic context network for high-resolution image semantic segmentation[J]. Remote Sensing, 2022, 14(7): 1638. DOI: 10.3390/rs14071638.
[15] Wang X L, Girshick R, Gupta A, et al.Non-local neural networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018. Salt Lake City, UT, USA. IEEE, 2018: 7794-7803. DOI: 10.1109/cvpr.2018.00813.
[16] Liang J L, Deng Y F, Zeng D.A deep neural network combined CNN and GCN for remote sensing scene classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13: 4325-4338. DOI: 10.1109/JSTARS.2020.3011333.
[17] Li Y S, Chen R X, Zhang Y J, et al.Multi-label remote sensing image scene classification by combining a convolutional neural network and a graph neural network[J]. Remote Sensing, 2020, 12(23): 4003. DOI: 10.3390/rs12234003.
[18] Peng F F, Lu W, Tan W X, et al.Multi-output network combining GNN and CNN for remote sensing scene classification[J]. Remote Sensing, 2022, 14(6): 1478. DOI: 10.3390/rs14061478.
[19] Diao Q, Dai Y P, Zhang C, et al.Superpixel-based attention graph neural network for semantic segmentation in aerial images[J]. Remote Sensing, 2022, 14(2): 305. DOI: 10.3390/rs14020305.
[20] Veličković P, Cucurull G, Casanova A, et al. Graph attention networks[EB/OL].2017: arXiv:1710.10903.(2017-10-30) [2023-09-15]. https://arxiv.org/abs/1710.10903.
[21] Sun K, Xiao B, Liu D, et al.Deep high-resolution representation learning for human pose estimation[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 15-20, 2019. Long Beach, CA, USA. IEEE, 2019: 5693-5703, DOI: 10.1109/cvpr.2019.00584.
[22] Schlichtkrull M, Kipf T, Bloem P, et al. Modeling relational data with graph convolutional networks. [EB/OL].2017: arXiv: 1703.06103.(2017-03-17) [2023-09-15]. https://arxiv.org/abs/1703.06103.
[23] Achanta R, Shaji A, Smith K, et al.SLIC superpixels compared to state-of-the-art superpixel methods[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(11): 2274-2282. DOI: 10.1109/TPAMI.2012.120.
[24] Wang J D, Sun K, Cheng T H, et al.Deep high-resolution representation learning for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(10): 3349-3364. DOI: 10.1109/TPAMI.2020.2983686.
[25] 徐存东, 李洪飞, 谷丰佑, 等. 基于无人机遥感影像的盐碱地信息的精准提取方法[J]. 中国农村水利水电, 2021(8): 116-122. DOI: 10.3969/j.issn.1007-2284.2021.08.020.
[26] 白俊龙, 王章琼, 闫海涛. K-means聚类引导的无人机遥感图像阈值分类方法[J]. 自然资源遥感, 2021, 33(3): 114-120. DOI: 10.6046/zrzyyg.2020301.
[27] 杨栩, 杨润书, 朱大明, 等. 基于遥感数据的耕地信息提取方法研究[J]. 软件导刊, 2018, 17(9): 166-170, 174, 2. DOI: 10.11907/rjdk.182110.
[28] Zhang X P, Cheng B, Chen J F, et al.High-resolution boundary refined convolutional neural network for automatic agricultural greenhouses extraction from GaoFen-2 satellite imageries[J]. Remote Sensing, 2021, 13(21): 4237. DOI: 10.3390/rs13214237.
[29] Kingma D P, Ba J. Adam: A method for stochastic optimization[EB/OL].2014: arXiv:1412.6980.(2014-12-22) [2023-09-15]. https://arxiv.org/abs/1412.6980.
[30] Lin G S, Milan A, Shen C H, et al.RefineNet: multi-path refinement networks for high-resolution semantic segmentation[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26, 2017, Honolulu, HI, USA. IEEE, 2017: 5168-5177. DOI: 10.1109/CVPR.2017.549.
[31] Jha D, Smedsrud P H, Johansen D, et al.A comprehensive study on colorectal polyp segmentation with ResUNet++, conditional random field and test-time augmentation[J]. IEEE Journal of Biomedical and Health Informatics, 2021, 25(6): 2029-2040. DOI: 10.1109/jbhi.2021.3049304.
[32] Hu J, Shen L, Sun G.Squeeze-and-excitation networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018, Salt Lake City, UT, USA. IEEE, 2018: 7132-7141. DOI: 10.1109/CVPR.2018.00745.
[33] Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks[EB/OJ].2016: arXiv:1609.02907.(2016-09-09) [2023-09-15]. https://arxiv.org/abs/1609.02907.
[34] Takikawa T, Acuna D, Jampani V, et al. Gated-scnn: Gated shape cnns for semantic segmentation[EB/OL].2019: arXiv: 1907.05740.(2019-07-12) [2023-09-15]. https://arxiv.org/abs/1907.05740.