Building extraction based on UNet++ network with different backbones

doi:10.7523/j.ucas.2020.0040

Abstract

Abstract: Automatic building extraction methods based on deep learning theory have the technical characteristics of high accuracy and speed,and are of great significance in industrial applications, such as urban planning,disaster prevention and mitigation. This paper introduces the deep learning modules and the traditional remote sensing validation section in the proposed building extraction method in high-resolution remote sensing imageries, forming an operational deep-learning-theory based building extraction technical system that integrates different backbone modules, UNet + + networks,and remote sensing authenticity verification modules. The basic network is transformed through the traditional convolutional network model backbones,such as VGG,ResNet, and Inception to improve the model operational efficiency,strengthen the model feature learning capabilities,verify the effectiveness and applicability of the algorithm through authenticity validation. Taking the Massachusetts building dataset published by Mnih as the data source,a comparative analysis was carried out with the traditional non-full convolutional network model and full convolutional network model. The results show that an increasing in the depth and width of the model can substantially improve the building extraction results. The InceptionV3-UNet + + backbone model has the best performance in recall rate,accuracy,CSI,F1 score,Kappa coefficients, and total accuracy,reaching 85. 14%,90. 50%,0. 781 6,0. 877 4,0. 850 4, and 95. 57%,respectively,and its robustness is also verified on the WHU datasets. This method has significantly improved the extraction accuracy and the details of the buildings extracted, especially on complex and irregular buildings, which will facilitate the building extraction applications in real, complex, and large scene of high-resolution remote sensing imageries.

Key words: deep learning, high-resolution remote sensing image, convolutional neural network, building extraction, image segmentation

CLC Number:

TP753

GU Yumin, YAN Fuli. Building extraction based on UNet++ network with different backbones[J]. Journal of University of Chinese Academy of Sciences, 2022, 39(4): 512-523.

References

[1] 张庆云,赵冬.高空间分辨率遥感影像建筑物提取方法综述[J].测绘与空间地理信息,2015,38(4):78-82.DOI:10.3969/j.issn.1672-5867.2015.04.026.
[2] 钱瑶,唐立娜,赵景柱.基于遥感的建筑物高度快速提取研究综述[J].生态学报,2015,35(12):3886-3895.DOI:10.5846/stxb201309252361.
[3] Myint S W,Gober P,Brazel A,et al. Per-pixel vs. object-based classification of urban land cover extraction using high spatial resolution imagery[J]. Remote Sensing of Environment,2011,115(5):1145-1161.DOI:10.1016/j.rse.2010.12.017.
[4] 李莉.面向对象的高分辨率遥感影像信息提取研究.成都:成都理工大学,2012.
[5] Cleve C, Kelly M, Kearns F R, et al. Classification of the wildland-urban interface:a comparison of pixel-and object-based classifications using high-resolution aerial photography[J]. Computers, Environment and Urban Systems,2008,32(4):317-326.DOI:10.1016/j.compenvurbsys.2007.10.001.
[6] Baatz M,Schäpe A. Multiresolution segmentation:an optimization approach for high quality multi-Scale image segmentation//Strobl J,Blaschke T, Griesbner G. Angewandte Geographische Informations-Verarbeitung,XII,Wichmann-Verlag,Heidelberg,Germany,2000:12-23.
[7] 王丹.一种高分辨率遥感影像建筑物边缘提取方法[J].环境保护与循环经济,2009,29(10):26-28.DOI:10.3969/j.issn.1674-1021.2009.10.010.
[8] Jung C R,Schramm R. Rectangle detection based on a windowed hough transform//Proceedings of the 17th Brazilian Symposium on Computer Graphics and Image Processing. October 20, 2004, Curitiba,Brazil. IEEE, 2004:113-120. DOI:10.1109/SIBGRA.2004.1352951.
[9] 魏德强.高分辨率遥感影像建筑物提取技术研究.郑州:解放军信息工程大学,2013.
[10] 高薇.基于不变矩算法的高分辨率遥感影像建筑物特征提取.成都:电子科技大学,2013.
[11] Dempster A P,Laird N M,Rubin D B. Maximum likelihood from incomplete data via the EM algorithm[J]. Journal of the Royal Statistical Society:Series B (Methodological),1977,39(1):1-22. DOI:10.1111/j.2517-6161.1977.tb01600.x.
[12] Simonyan K,Zisserman A. Very deep convolutional networks for large-scale image recognition[J].ArXiv Preprint, 2014, ArXiv:1409.1556.
[13] Szegedy C,Liu W,Jia Y Q,et al. Going deeper with convolutions//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 7-12,2015, Boston, MA, USA. IEEE, 2015:1-9.DOI:10.1109/CVPR.2015.7298594.
[14] He K M,Zhang X Y,Ren S Q,et al. Deep residual learning for image recognition//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 27-30, 2016, Las Vegas,NV,USA. IEEE, 2016:770-778.DOI:10.1109/CVPR.2016.90.
[15] Mnih. Machine learning for aerial image labeling. Toronto:University of Toronto (Canada),2013.
[16] Saito S,Yamashita T,Aoki Y. Multiple object extraction from aerial imagery with convolutional neural networks[J].Journal of Imaging Science and Technology,2016,60(1):10402.DOI:10.2352/j.imagingsci.technol.2016.60.1.010402.
[17] Long J,Shelhamer E,Darrell T. Fully convolutional networks for semantic segmentation//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 7-12, 2015, Boston, MA, USA. IEEE, 2015:3431-3440.DOI:10.1109/CVPR.2015.7298965.
[18] 刘文涛,李世华,覃驭楚.基于全卷积神经网络的建筑物屋顶自动提取[J].地球信息科学学报,2018,20(11):1562-1570.DOI:10.12082/dqxxkx.2018.180159.
[19] 刘浩,骆剑承,黄波,等.基于特征压缩激活Unet网络的建筑物提取[J].地球信息科学学报,2019,21(11):1779-1789.DOI:10.12082/dqxxkx.2019.190285.
[20] Szegedy C,Ioffe S,Vanhoucke V,et al. Inception-v4,inception-ResNet and the impact of residual connections on learning[J]. ArXiv Preprint, 2016, ArXiv:1602.07261.
[21] Szegedy C,Vanhoucke V,Ioffe S,et al. Rethinking the inception architecture for computer vision//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 27-30, 2016. Las Vegas, NV,USA. IEEE, 2016:2818-2826.DOI:10.1109/CVPR.2016.308.
[22] Ioffe S,Szegedy C. Batch normalization:accelerating deep network training by reducing internal covariate shift[J]. ArXiv Preprint, 2015, ArXiv:1502.03167.
[23] Srivastava N,Hinton G E,Krizhevsky A,et al. Dropout:a simple way to prevent neural networks from overfitting[J]. Journal of Machine Learning Research,2014,15(1):1929-1958.
[24] Milletari F,Navab N,Ahmadi S A. V-net:fully convolutional neural networks for volumetric medical image segmentation//2016 Fourth International Conference on 3D Vision (3DV). October 25-28, 2016, Stanford,CA,USA. IEEE, 2016:565-571.DOI:10.1109/3DV.2016.79.
[25] Ronneberger O,Fischer P,Brox T. U-net:convolutional networks for biomedical image segmentation[J]. ArXiv Preprint, 2015, ArXiv:1505.04597.
[26] Hinton G E,Salakhutdinov R R. Reducing the dimensionality of data with neural networks[J]. Science,2006,313(5786):504-507.DOI:10.1126/science.1127647.
[27] Zhou Z W,Siddiquee M M R,Tajbakhsh N,et al. UNet++:a nested U-Net architecture for medical image segmentation[M]//Stoyanov D,Taylor Z,Carneiro G,et al. Deep learning in medical image analysis and multimodal learning for clinical decision support. Spain:Springer,2018:3-11.
[28] Huang G,Liu Z,van der Maaten L,et al. Densely connected convolutional networks//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26, 2017, Honolulu,HI,USA. IEEE, 2017:2261-2269.DOI:10.1109/XVPR.2017.243.
[29] Lee C Y,Xie S N,Gallagher P W,et al. Deeply-supervised nets[J]. ArXiv Preprint, 2014, ArXiv:1409.5185.
[30] Kingma D P,Ba J. Adam:a method for stochastic optimization[J]. ArXiv Preprint, 2015, ArXiv:1412.6980.
[31] Ji S P,Wei S Q,Lu M. Fully convolutional networks for multisource building extraction from an open aerial and satellite imagery data set[J]. IEEE Transactions on Geoscience and Remote Sensing,2019,57(1):574-586. DOI:10.1109/TGRS.2018.2858817.