Building extraction method based on MFF-Deeplabv3+ network for high-resolution remote sensing images

doi:10.7523/j.ucas.2023.010

Abstract

Abstract: Automatic extraction of building information from high-resolution remote sensing images is of great significance in the fields of environmental monitoring, earthquake mitigation， and land use, making it a research hotspot in the field of high-resolution remote sensing applications. In order to improve the accuracy of building extraction from high-resolution remote sensing images, a building extraction method based on MFF-Deeplabv3+(multiscale feature fusion-Deeplabv3+) network for high-resolution remote sensing images is proposed in this paper. First, the multi-scale feature enhancement module is designed to enable the network to capture more scale context information; then, the feature fusion module is designed to effectively fuse deep features with shallow features to reduce the loss of detail information; finally, the attention mechanism module is introduced to select accurate features adaptively. In the comparison experiments of the Inria building dataset, MFF-Deeplabv3+ achieved the highest accuracy in PA, MPA, FWIoU, and MIoU metrics with 95.75%, 91.22%, 92.12%, and 85.01%, respectively, while the generalization experiments of the WHU building dataset achieved good results. The results show that this method extracts building information from high-resolution remote sensing images with high accuracy and strong generalization.

Key words: building extraction, deep learning, attention mechanism, multi-scale feature enhancement, high-resolution remote sensing images

CLC Number:

TP751.1

CHEN Jingwei, LI Yu, CHEN Jun, ZHANG Hongqun. Building extraction method based on MFF-Deeplabv3+ network for high-resolution remote sensing images[J]. Journal of University of Chinese Academy of Sciences, 2024, 41(5): 654-664.

References

[1] Sirmacek B, Unsalan C. Urban-area and building detection using SIFT keypoints and graph theory[J]. IEEE Transactions on Geoscience and Remote Sensing, 2009, 47(4):1156-1167. DOI:10.1109/TGRS.2008.2008440.
[2] Lee D S, Shan J, Bethel J S. Class-guided building extraction from ikonos imagery[J]. Photogrammetric Engineering&Remote Sensing, 2003, 69(2):143-150. DOI:10.14358/PERS.69.2.143.
[3] Li E, Xu S B, Meng W L, et al. Building extraction from remotely sensed images by integrating saliency cue[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(3):906-919. DOI:10.1109/JSTARS.2016.2603184.
[4] Bischof H, Schneider W, Pinz A J. Multispectral classification of Landsat-images using neural networks[J]. IEEE Transactions on Geoscience and Remote Sensing, 1992, 30(3):482-490. DOI:10.1109/36.142926.
[5] Zhang F, Du B, Zhang L P. Scene classification via a gradient boosting random convolutional network framework[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(3):1793-1802. DOI:10.1109/TGRS.2015.2488681.
[6] Tarabalka Y, Benediktsson J A, Chanussot J. Spectral-spatial classification of hyperspectral imagery based on partitional clustering techniques[J]. IEEE Transactions on Geoscience and Remote Sensing, 2009, 47(8):2973-2987. DOI:10.1109/TGRS.2009.2016214.
[7] Huang X, Zhang L P. A multidirectional and multiscale morphological index for automatic building extraction from multispectral GeoEye-1 imagery[J]. Photogrammetric Engineering&Remote Sensing, 2011, 77(7):721-732. DOI:10.14358/PERS.77.7.721.
[8] Shi Y L, Li Q Y, Zhu X X. Building segmentation through a gated graph convolutional neural network with deep structured feature embedding[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 159:184-197. DOI:10.1016/j.isprsjprs.2019.11.004.
[9] Collobert R, Weston J. A unified architecture for natural language processing:deep neural networks with multitask learning[C]//Proceedings of the 25th international conference on Machine learning. July 5-9, 2008, Helsinki, Finland. New York:ACM, 2008:160-167. DOI:10.1145/1390156.1390177.
[10] Zhu X X, Tuia D, Mou L C, et al. Deep learning in remote sensing:a comprehensive review and list of resources[J]. IEEE Geoscience and Remote Sensing Magazine, 2017, 5(4):8-36. DOI:10.1109/MGRS.2017.2762307.
[11] Song S R, Lichtenberg S P, Xiao J X. SUN RGB-D:a RGB-D scene understanding benchmark suite[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 7-12, 2015, Boston, MA, USA. IEEE, 2015:567-576. DOI:10.1109/CVPR.2015.7298655.
[12] Ji S P, Wei S Q, Lu M. Fully convolutional networks for multisource building extraction from an open aerial and satellite imagery data set[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(1):574-586. DOI:10.1109/TGRS.2018.2858817.
[13] 杨旭勃,田金文.小数据集中的小型建筑物提取方法研究[J].测绘通报, 2019(10):51-55. DOI:10.13474/j.cnki.11-2246.2019.0317.
[14] Zuo T C, Feng J T, Chen X J. HF-FCN:hierarchically fused fully convolutional network for robust building extraction[M]//Computer Vision-ACCV 2016. Cham:Springer International Publishing, 2017:291-302. DOI:10.1007/978-3-319-54181-5_19.
[15] Ronneberger O, Fischer P, Brox T. U-net:convolutional networks for biomedical image segmentation[M]//Lecture Notes in Computer Science. Cham:Springer International Publishing, 2015:234-241. DOI:10.1007/978-3-319-24574-4_28.
[16] 古煜民,阎福礼.基于不同骨架UNet++网络的建筑物提取[J].中国科学院大学学报, 2022, 39(4):512-523. DOI:10.7523/j.ucas.2020.0040.
[17] Liu J Y, Wang S S, Hou X W, et al. A deep residual learning serial segmentation network for extracting buildings from remote sensing imagery[J]. International Journal of Remote Sensing, 2020, 41(14):5573-5587. DOI:10.1080/01431161.2020.1734251.
[18] Yi Y N, Zhang Z J, Zhang W C, et al. Semantic segmentation of urban buildings from VHR remote sensing imagery using a deep convolutional neural network[J]. Remote Sensing, 2019, 11(15):1774. DOI:10.3390/rs11151774.
[19] 胡明洪,李佳田,姚彦吉,等.结合多路径的高分辨率遥感影像建筑物提取SER-UNet算法[J].测绘学报,2023,52(5):808-817.DOI:10.11947/j.AGCS.2023.20210691.
[20] 徐佳伟,刘伟,单浩宇,等.基于PRCUnet的高分遥感影像建筑物提取[J].地球信息科学学报, 2021, 23(10):1838-1849. DOI:10.12082/dqxxkx.2021.210283.
[21] Chen L C, Papandreou G, Kokkinos I, et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs[EB/OL]. arXiv:1412.7062.(2014-12-22)[2023-02-02]. https://arxiv.org/abs/1412.7062.
[22] Chen L C, Papandreou G, Kokkinos I, et al. DeepLab:semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4):834-848. DOI:10.1109/TPAMI.2017.2699184.
[23] Chen L C, Papandreou G, Schroff F, et al. Rethinking atrous convolution for semantic image segmentation[EB/OL]. arXiv:1706.05587.(2017-06-17)[2023-02-02]. https://arxiv.org/abs/1706.05587.
[24] Chen L C, Zhu Y K, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Computer Vision-ECCV 2018:15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part VII. New York:ACM, 2018:833-851. DOI:10.1007/978-3-030-01234-2_49.
[25] Guo Z C, Xu J M, Liu A D. Remote sensing image semantic segmentation method based on improved Deeplabv3+[C]//Proc SPIE 11928, International Conference on Image Processing and Intelligent Control (IPIC 2021), 2021, 11928:101-109. DOI:10.1117/12.2611930.
[26] Chollet F. Xception:deep learning with depthwise separable convolutions[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26, 2017, Honolulu, HI, USA. IEEE, 2017:1800-1807. DOI:10.1109/CVPR.2017.195.
[27] 王华俊,葛小三.一种轻量级的DeepLabv3+遥感影像建筑物提取方法[J].自然资源遥感, 2022, 34(2):128-135. DOI:10.6046/zrzyyg.2021219.
[28] Cao J M, Li Y Y, Sun M C, et al. DO-conv:depthwise over-parameterized convolutional layer[J]. IEEE Transactions on Image Processing:a Publication of the IEEE Signal Processing Society, 2022, 31:3726-3736. DOI:10.1109/TIP.2022.3175432.
[29] Lin T Y, Dollár P, Girshick R, et al. Feature pyramid networks for object detection[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26, 2017, Honolulu, HI, USA. IEEE, 2017:936-944. DOI:10.1109/CVPR.2017.106.
[30] Wang Q L, Wu B G, Zhu P F, et al. ECA-net:efficient channel attention for deep convolutional neural networks[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 13-19, 2020, Seattle, WA, USA. IEEE, 2020:11531-11539. DOI:10.1109/CVPR42600.2020.01155.
[31] Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018, Salt Lake City, UT, USA. IEEE, 2018:7132-7141. DOI:10.1109/CVPR.2018.00745.