高分辨率关系图卷积网络遥感语义分割方法

doi:10.7523/j.ucas.2023.079

摘要/Abstract

摘要： 遥感影像语义分割是遥感图像处理分析的重要任务，尤其是在多类别语义分割方面。目前方法主要围绕卷积神经网络展开，但卷积仅关注图像局部信息而忽视全局信息。因此，受高分辨率网络(HRNet)和关系图卷积网络(R-GCN)启发，提出一种高分辨率关系图卷积网络(HRGCN)，用于多类别语义分割。首先对原始图像做简单线性迭代聚类(SLIC)，利用该结果分割HRNet输出的特征图，获得同质性高且包含多分辨率信息的超像素块；然后基于超像素块构建图节点和边，使用R-GCN对图节点分类，从而学习到不同地物间长距离依赖关系，并完成遥感影像的提取分类。利用HRGCN模型在Potsdam和Vaihingen数据集上进行实验，将结果与已有方法对比，$\bar{F}_1$值和MIoU值均有不同程度提升，证明该方法具有较好的先进性。

关键词: 遥感影像, R-GCN, HRNet, 超像素, 语义分割, 图像处理

Abstract: Semantic segmentation of remote sensing images is an important task in remote sensing image processing and analysis, especially in multi-category semantic segmentation. Current methods mainly revolve around convolutional neural networks, but convolution only focuses on the local information of the image while ignoring the global information. Therefore, inspired by high resolution network (HRNet) and relational graph convolutional network (R-GCN), this paper proposes a high-resolution relational graph convolutional network (HRGCN) for multi-category semantic segmentation. Firstly, simple linear iterative clustering (SLIC) is done on the original image, and the result is used to segment the feature map output from HRNet to obtain superpixel blocks with high homogeneity and containing multi-resolution information; then graph nodes and edges are constructed based on the superpixel blocks, and R-GCN is used to classify the graph nodes, so as to learn the long-distance dependency between different features and complete the extraction and classification of remote sensing images. The HRGCN model designed in this paper is experimented on Potsdam and Vaihingen datasets, and the results are compared with the existing methods, and the $\bar{F}_1$ values and MIoU values are improved to certain degrees, which proves that the method has good advancement.

Key words: remote sensing image, R-GCN, HRNet, superpixel, semamtic segmentation, image processing

中图分类号:

TP751
P237

王寅达, 陈嘉辉, 彭玲, 李兆博, 杨丽娜. 高分辨率关系图卷积网络遥感语义分割方法[J]. 中国科学院大学学报, 2025, 42(1): 107-115.

WANG Yinda, CHEN Jiahui, PENG Ling, LI Zhaobo, YANG Lina. Remote sensing semantic segmentation method based on high-resolution relational graph convolutional network[J]. Journal of University of Chinese Academy of Sciences, 2025, 42(1): 107-115.

参考文献

[1] 许泽宇，沈占锋，李杨，等. 增强型DeepLab算法和自适应损失函数的高分辨率遥感影像分类[J]. 遥感学报,2022,26(2):406-415.DOI: 10.11834/jrs.20209200.
[2] 陈若男, 彭玲, 刘玉菲, 等. 引入空间距离信息的城郊山区道路提取与应用[J]. 中国科学院大学学报, 2022, 39(5): 658-667. DOI: 10.7523/j.ucas.2021.0004.
[3] 王寅达，彭玲，陈德跃，等. 基于改进 U-Net 模型的农业大棚遥感提取方法[J]. 中国科学院大学学报, 2024,41(3)：375-386. DOI: 10.7523/j.ucas. 2023. 060.
[4] Yu J E, Cai Y, Lyu X, et al. Boundary-guided semantic context network for water body extraction from remote sensing images[J]. Remote Sensing, 2023, 15(17): 4325. DOI: 10.3390/rs15174325.
[5] Yan G D, Jing H T, Li H, et al. Enhancing building segmentation in remote sensing images: advanced multi-scale boundary refinement with MBR-HRNet[J]. Remote Sensing, 2023, 15(15): 3766. DOI: 10.3390/rs15153766.
[6] 李新娜, 王小鹏, 魏统艺. 自适应形态学与多尺度结合的植被区域遥感图像分割方法[J]. 激光与光电子学进展, 2022, 59(24): 240-246. DOI: 10.3788/LOP202259. 2428001.
[7] 闵蕾, 高昆, 李维, 等. 光学遥感图像分割技术综述[J]. 航天返回与遥感, 2020, 41(6): 1-13. DOI: 10.3969/j.issn.1009-8518.2020.06.001.
[8] Badrinarayanan V, Kendall A, Cipolla R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495. DOI: 10.1109/TPAMI.2016.2644615.
[9] Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2015: 234-241.10.1007/978-3-319-24574-4_28.
[10] Chen L C, Papandreou G, Kokkinos I, et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs[EB/OL]. arXiv: 1412.7062. (2014-12-22) [2023-09-15]. https://arxiv.org/abs/1412.7062.
[11] Chen L C, Papandreou G, Kokkinos I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4): 834-848. DOI: 10.1109/TPAMI.2017.2699184.
[12] Chen L C, Papandreou G, Schroff F, et al. Rethinking atrous convolution for semantic image segmentation[EB/OL]. arXiv:1706.05587. (2017-06-17) [2023-09-15]. https://arxiv.org/abs/1706.05587.
[13] Chen L C, Zhu Y K, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[EB/OL]. arXiv: 1802.02611. (2018-02-07) [2023-09-15]. https://arxiv.org/abs/1802.02611.
[14] Chen F L, Liu H J, Zeng Z H, et al. BES-net: boundary enhancing semantic context network for high-resolution image semantic segmentation[J]. Remote Sensing, 2022, 14(7): 1638. DOI: 10.3390/rs14071638.
[15] Wang X L, Girshick R, Gupta A, et al. Non-local neural networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018. Salt Lake City, UT, USA. IEEE, 2018: 7794-7803. DOI: 10.1109/cvpr.2018.00813.
[16] Liang J L, Deng Y F, Zeng D. A deep neural network combined CNN and GCN for remote sensing scene classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13: 4325-4338. DOI: 10.1109/JSTARS.2020.3011333.
[17] Li Y S, Chen R X, Zhang Y J, et al. Multi-label remote sensing image scene classification by combining a convolutional neural network and a graph neural network[J]. Remote Sensing, 2020, 12(23): 4003. DOI: 10.3390/rs12234003.
[18] Peng F F, Lu W, Tan W X, et al. Multi-output network combining GNN and CNN for remote sensing scene classification[J]. Remote Sensing, 2022, 14(6): 1478. DOI: 10.3390/rs14061478.
[19] Diao Q, Dai Y P, Zhang C, et al. Superpixel-based attention graph neural network for semantic segmentation in aerial images[J]. Remote Sensing, 2022, 14(2): 305. DOI: 10.3390/rs14020305.
[20] Veličković P, Cucurull G, Casanova A, et al. Graph attention networks[EB/OL]. arXiv:1710.10903. (2017-10-30) [2023-09-15]. https://arxiv.org/abs/1710.10903.
[21] Sun K, Xiao B, Liu D, et al. Deep high-resolution representation learning for human pose estimation[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 15-20, 2019. Long Beach, CA, USA. IEEE, 2019: 5693-5703, DOI: 10.1109/cvpr.2019.00584.
[22] Schlichtkrull M, Kipf T, Bloem P, et al. Modeling relational data with graph convolutional networks [EB/OL]. arXiv: 1703.06103. (2017-03-17) [2023-09-15]. https://arxiv.org/abs/1703.06103.
[23] Achanta R, Shaji A, Smith K, et al. SLIC superpixels compared to state-of-the-art superpixel methods[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(11): 2274-2282. DOI: 10.1109/TPAMI.2012.120.
[24] Wang J D, Sun K, Cheng T H, et al. Deep high-resolution representation learning for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(10): 3349-3364. DOI: 10.1109/TPAMI.2020.2983686.
[25] 徐存东, 李洪飞, 谷丰佑, 等. 基于无人机遥感影像的盐碱地信息的精准提取方法[J]. 中国农村水利水电, 2021(8): 116-122. DOI: 10.3969/j.issn.1007-2284.2021.08.020.
[26] 白俊龙, 王章琼, 闫海涛. K-means聚类引导的无人机遥感图像阈值分类方法[J]. 自然资源遥感, 2021, 33(3): 114-120. DOI: 10.6046/zrzyyg.2020301.
[27] 杨栩, 杨润书, 朱大明, 等. 基于遥感数据的耕地信息提取方法研究[J]. 软件导刊, 2018, 17(9): 166-170, 174. DOI: 10.11907/rjdk.182110.
[28] Zhang X P, Cheng B, Chen J F, et al. High-resolution boundary refined convolutional neural network for automatic agricultural greenhouses extraction from GaoFen-2 satellite imageries[J]. Remote Sensing, 2021, 13(21): 4237. DOI: 10.3390/rs13214237.
[29] Kingma D P, Ba J. Adam: a method for stochastic optimization[EB/OL]. arXiv:1412.6980. (2014-12-22) [2023-09-15]. https://arxiv.org/abs/1412.6980.
[30] Lin G S, Milan A, Shen C H, et al. RefineNet: multi-path refinement networks for high-resolution semantic segmentation[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26, 2017, Honolulu, HI, USA. IEEE, 2017: 5168-5177. DOI: 10.1109/CVPR.2017.549.
[31] Jha D, Smedsrud P H, Johansen D, et al. A comprehensive study on colorectal polyp segmentation with ResUNet++, conditional random field and test-time augmentation[J]. IEEE Journal of Biomedical and Health Informatics, 2021, 25(6): 2029-2040. DOI: 10.1109/jbhi.2021.3049304.
[32] Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018, Salt Lake City, UT, USA. IEEE, 2018: 7132-7141. DOI: 10.1109/CVPR.2018.00745.
[33] Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks[EB/OL]. arXiv:1609.02907. (2016-09-09) [2023-09-15]. https://arxiv.org/abs/1609.02907.
[34] Takikawa T, Acuna D, Jampani V, et al. Gated-scnn: gated shape cnns for semantic segmentation[EB/OL]. arXiv: 1907.05740. (2019-07-12) [2023-09-15]. https://arxiv.org/abs/1907.05740.