Building extraction method based on MFF-Deeplabv3+ network for high resolution remote sensing images

doi:10.7523/j.ucas.2023.010

Abstract

Abstract: Automatic extraction of building information from high-resolution remote sensing images is of great significance in the fields of environmental monitoring, earthquake mitigation and land use, and is a research hotspot in the field of high-resolution remote sensing applications. In order to improve the accuracy of building extraction in high-resolution remote sensing images, a building extraction method based on MFF-Deeplabv3+ (Multiscale feature fusion-Deeplabv3+) network for high-resolution remote sensing images is proposed in this paper. First, the multi-scale feature enhancement module is designed to enable the network to capture more scale context information; then, the feature fusion module is designed to effectively fuse deep features with shallow features to reduce the loss of detail information; finally, the attention mechanism module is introduced to select accurate features adaptively. In the comparison experiments of Inria building dataset, MFF-Deeplabv3+ achieved the highest accuracy in PA, MPA, FWIoU, and MIoU metrics with 95.75%, 91.22%, 92.12%, and 85.01%, respectively, while the generalization experiments of WHU building dataset achieved good results. The results show that this method extracts building information in high resolution remote sensing images with high accuracy and has good generalization.

Key words: building extraction, deep learning, attention mechanism, multi-scale feature enhancement, high-resolution remote sensing images

CLC Number:

TP751.1

CHEN Jingwei, LI Yu, CHEN Jun, ZHANG Hongqun. Building extraction method based on MFF-Deeplabv3+ network for high resolution remote sensing images[J]. Journal of University of Chinese Academy of Sciences, DOI: 10.7523/j.ucas.2023.010.

References

[1] Sirmacek B, Unsalan C.Urban-area and building detection using SIFT keypoints and graph theory[J]. IEEE Transactions on Geoscience and Remote Sensing, 2009, 47(4): 1156-1167. DOI: 10.1109/TGRS.2008.2008440.
[2] Lee D S, Shan J, Bethel J S.Class-guided building extraction from ikonos imagery[J]. Photogrammetric Engineering & Remote Sensing, 2003, 69(2): 143-150. DOI: 10.14358/PERS.69.2.143.
[3] Li E, Xu S B, Meng W L, et al.Building extraction from remotely sensed images by integrating saliency cue[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(3): 906-919. DOI: 10.1109/JSTARS.2016.2603184.
[4] Bischof H, Schneider W, Pinz A J.Multispectral classification of Landsat-images using neural networks[J]. IEEE Transactions on Geoscience and Remote Sensing, 1992, 30(3): 482-490. DOI: 10.1109/36.142926.
[5] Zhang F, Du B, Zhang L P.Scene classification via a gradient boosting random convolutional network framework[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(3): 1793-1802. DOI: 10.1109/TGRS.2015.2488681.
[6] Tarabalka Y, Benediktsson J A, Chanussot J.Spectral-spatial classification of hyperspectral imagery based on partitional clustering techniques[J]. IEEE Transactions on Geoscience and Remote Sensing, 2009, 47(8): 2973-2987. DOI: 10.1109/TGRS.2009.2016214.
[7] Huang X, Zhang L P.A multidirectional and multiscale morphological index for automatic building extraction from multispectral GeoEye-1 imagery[J]. Photogrammetric Engineering & Remote Sensing, 2011, 77(7): 721-732. DOI: 10.14358/PERS.77.7.721.
[8] Shi Y L, Li Q Y, Zhu X X.Building segmentation through a gated graph convolutional neural network with deep structured feature embedding[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 159: 184-197. DOI: 10.1016/j.isprsjprs.2019.11.004.
[9] Collobert R, Weston J.A unified architecture for natural language processing: Deep neural networks with multitask learning[C]//Proceedings of the 25th international conference on Machine learning. July 5-9, 2008, Helsinki, Finland. New York: ACM, 2008: 160-167. DOI: 10.1145/1390156.1390177.
[10] Zhu X X, Tuia D, Mou L C, et al.Deep learning in remote sensing: A comprehensive review and list of resources[J]. IEEE Geoscience and Remote Sensing Magazine, 2017, 5(4): 8-36. DOI: 10.1109/MGRS.2017.2762307.
[11] Song S R, Lichtenberg S P, Xiao J X.SUN RGB-D: A RGB-D scene understanding benchmark suite[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 7-12, 2015, Boston, MA, USA. IEEE, 2015: 567-576. DOI:10.1109/CVPR.2015.7298655.
[12] Ji S P, Wei S Q, Lu M.Fully convolutional networks for multisource building extraction from an open aerial and satellite imagery data set[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(1): 574-586. DOI:10.1109/TGRS.2018.2858817.
[13] 杨旭勃, 田金文. 小数据集中的小型建筑物提取方法研究[J]. 测绘通报, 2019(10): 51-55. DOI: 10.13474/j.cnki.11-2246.2019.0317.
[14] Zuo T C, Feng J T, Chen X J.HF-FCN: Hierarchically fused fully convolutional network for robust building extraction[M]//Computer Vision - ACCV 2016. Cham: Springer International Publishing, 2017: 291-302. DOI: 10.1007/978-3-319-54181-5_19.
[15] Ronneberger O, Fischer P, Brox T.U-net: Convolutional networks for biomedical image segmentation[M]//Lecture Notes in Computer Science. Cham: Springer International Publishing, 2015: 234-241. DOI: 10.1007/978-3-319-24574-4_28.
[16] 古煜民, 阎福礼. 基于不同骨架UNet++网络的建筑物提取[J]. 中国科学院大学学报, 2022, 39(4): 512-523. DOI: 10.7523/j.ucas.2020.0040.
[17] Liu J Y, Wang S S, Hou X W, et al.A deep residual learning serial segmentation network for extracting buildings from remote sensing imagery[J]. International Journal of Remote Sensing, 2020, 41(14): 5573-5587. DOI: 10.1080/01431161.2020.1734251.
[18] Yi Y N, Zhang Z J, Zhang W C, et al.Semantic segmentation of urban buildings from VHR remote sensing imagery using a deep convolutional neural network[J]. Remote Sensing, 2019, 11(15): 1774. DOI: 10.3390/rs11151774.
[19] 胡明洪, 李佳田, 姚彦吉, 等. 结合多路径的高分辨率遥感影像建筑物提取SER-UNet算法[J/OL]. 测绘学报: 1-14[2023-02-02]. http://kns.cnki.net/kcms/detail/11.2089.P.20221021.1438.008.html.
[20] 徐佳伟, 刘伟, 单浩宇, 等. 基于PRCUnet的高分遥感影像建筑物提取[J]. 地球信息科学学报, 2021, 23(10): 1838-1849. DOI: 10.12082/dqxxkx.2021.210283.
[21] Chen L C, Papandreou G, Kokkinos I, et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs[EB/OL]. arXiv:1412.7062. (2014-12-22) [2023-02-02]. https://arxiv.org/abs/1412.7062.
[22] Chen L C, Papandreou G, Kokkinos I, et al.DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4): 834-848. DOI: 10.1109/TPAMI.2017.2699184.
[23] Chen L C, Papandreou G, Schroff F, et al. Rethinking atrous convolution for semantic image segmentation[EB/OL]. arXiv:1706.05587. (2017-06-17) [2023-02-02]. https://arxiv.org/abs/1706.05587.
[24] Chen L C, Zhu Y K, Papandreou G, et al.Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Computer Vision - ECCV 2018: 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part VII. New York: ACM, 2018: 833-851. DOI: 10.1007/978-3-030-01234-2_49.
[25] Guo Z C, Xu J M, Liu A D.Remote sensing image semantic segmentation method based on improved Deeplabv3+[C]//Proc SPIE 11928, International Conference on Image Processing and Intelligent Control (IPIC 2021), 2021, 11928: 101-109. DOI: 10.1117/12.2611930.
[26] Chollet F.Xception: deep learning with depthwise separable convolutions[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26, 2017, Honolulu, HI, USA. IEEE, 2017: 1800-1807. DOI: 10.1109/CVPR.2017.195.
[27] 王华俊, 葛小三. 一种轻量级的DeepLabv3+遥感影像建筑物提取方法[J]. 自然资源遥感, 2022, 34(2): 128-135. DOI: 10.6046/zrzyyg.2021219.
[28] Cao J M, Li Y Y, Sun M C, et al.DO-conv: Depthwise over-parameterized convolutional layer[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2022, 31: 3726-3736. DOI: 10.1109/TIP.2022.3175432.
[29] Lin T Y, Dollár P, Girshick R, et al.Feature pyramid networks for object detection[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26, 2017, Honolulu, HI, USA. IEEE, 2017: 936-944. DOI: 10.1109/CVPR.2017.106.
[30] Wang Q L, Wu B G, Zhu P F, et al.ECA-net: Efficient channel attention for deep convolutional neural networks[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). June 13-19, 2020, Seattle, WA, USA. IEEE, 2020: 11531-11539. DOI: 10.1109/CVPR42600.2020.01155.
[31] Hu J, Shen L, Sun G.Squeeze-and-excitation networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018, Salt Lake City, UT, USA. IEEE, 2018: 7132-7141. DOI: 10.1109/CVPR.2018.00745.