Spectral-spatial feature fusion deep network for cloud detection in remote sensing images

doi:10.7523/j.ucas.2021.0074

Abstract

Abstract: Current cloud detection methods fail to fully utilize the spectral-spatial features of remote sensing images. Insufficient use of spectral information results in misclassification of cloud with similar spectral feature, and insufficient use of spatial information makes it difficult to identify broken clouds or thin clouds. Motivated by these issues, we propose a spectral-spatial feature fusion network (SSFF-Net) for cloud detection which leverages the spectral-spatial information of remote sensing images. Firstly, SSFF-Net extracts the spectral features of remote sensing images with the 1×1 convolution kernel, then Transformer-based spatial encoder and decoder is applied to learn long-distance feature, which fully exploits the spectral and spatial information of remote sensing images. In this way, SSFF-Net overcomes the problem that spectral feature extraction depends on the empirical linear combination, and further reduces the loss of spatial position information. We evaluate our proposed model on the Landsat8 Biome and AIR-CD datasets. The results show that SSFF-Net has a good performance for cloud detection, with an accuracy of 97% and 96%, respectively.

Key words: spectral information, spatial information, Transformer, cloud detection, loss of information

CLC Number:

TP751.2

CHEN Siya, JI Luyan, ZHANG Peng, TANG Hairong. Spectral-spatial feature fusion deep network for cloud detection in remote sensing images[J]. Journal of University of Chinese Academy of Sciences, 2023, 40(3): 371-379.

References

[1] Zhang Y C, Rossow W B, Lacis A A, et al. Calculation of radiative fluxes from the surface to top of atmosphere based on ISCCP and other global data sets:refinements of the radiative transfer model and the input data[J]. Journal of Geophysical Research:Atmospheres, 2004, 109(D19):D19105. DOI:10.1029/2003JD004457.
[2] Sun L, Wei J, Wang J, et al. A universal dynamic threshold cloud detection algorithm (UDTCDA) supported by a prior surface reflectance database[J]. Journal of Geophysical Research:Atmospheres, 2016, 121(12):7172-7196. DOI:10.1002/2015JD024722.
[3] 刘庆凤, 刘吉平, 宋开山. 基于MODIS/NDVI时序数据的土地覆盖分类[J]. 中国科学院研究生院学报, 2010, 27(2):163-169. DOI:10.7523/j.issn.2095-6134.2010.2.003.
[4] 段雅鸣, 张锦水, 朱爽. 基于深度卷积神经网络的云检测方法[J]. 测绘通报, 2021(4):33-39. DOI:10.13474/j.cnki.11-2246.2021.0107.
[5] Zhu Z, Woodcock C E. Object-based cloud and cloud shadow detection in Landsat imagery[J]. Remote Sensing of Environment, 2012, 118:83-94. DOI:10.1016/j.rse.2011.10.028.
[6] Zhai H, Zhang H Y, Zhang L P, et al. Cloud/shadow detection based on spectral indices for multi/hyperspectral optical remote sensing imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 144:235-253. DOI:10.1016/j.isprsjprs.2018.07.006.
[7] Frantz D, Haß E, Uhl A, et al. Improvement of the Fmask algorithm for Sentinel-2 images:separating clouds from bright surfaces based on parallax effects[J]. Remote Sensing of Environment, 2018, 215:471-481. DOI:10.1016/j.rse.2018.04.046.
[8] Oishi Y, Ishida H, Nakamura R. A new Landsat 8 cloud discrimination algorithm using thresholding tests[J]. International Journal of Remote Sensing, 2018, 39(23):9113-9133. DOI:10.1080/01431161.2018.1506183.
[9] 张永宏, 蔡朋艳, 陶润喆, 等. 基于改进U-Net网络的遥感图像云检测[J]. 测绘通报, 2020(3):17-20,34. DOI:10.13474/j.cnki.11-2246.2020.0070.
[10] Jeppesen J H, Jacobsen R H, Inceoglu F, et al. A cloud detection algorithm for satellite imagery based on deep learning[J]. Remote Sensing of Environment, 2019, 229:247-259. DOI:10.1016/j.rse.2019.03.039.
[11] Chai D F, Newsam S, Zhang H K, et al. Cloud and cloud shadow detection in Landsat imagery based on deep convolutional neural networks[J]. Remote sensing of environment, 2019, 225:307-316. DOI:10.1016/j.rse.2019.03.007.
[12] Li Z W, Shen H F, Cheng Q, et al. Deep learning based cloud detection for medium and high resolution remote sensing images of different sensors[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 150:197-212. DOI:10.1016/j.isprsjprs.2019.02.017.
[13] 赵敏钧, 赵亚伟, 赵雅捷, 等. 一种新的基于深度学习的重叠关系联合抽取模型[J]. 中国科学院大学学报. DOI:10.7523/j.ucas.2020.0026.
[14] Sha Y, Zhang Y, Ji X, et al. Transformer-Unet:raw image processing with unet[EB/OL]. arXiv:2109.08417v1. (2021-09-17)[2021-11-11]. https://arxiv.org/abs/2109.08417v1.
[15] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[EB/OL]. arXiv:1706.03762v5. (2017-12-06)[2021-11-11]. https://arxiv.org/abs/1706.03762v5.
[16] Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16x16 words:transformers for image recognition at scale[EB/OL]. arXiv:2010.11929v2. (2021-06-03)[2021-11-11]. https://arxiv.org/abs/2010.11929v2.
[17] Carion N, Massa F, Synnaeve G, et al. End-to-end object detection with transformers[C]//Computer Vision-ECCV 2020, 2020:213-229. DOI:10.1007/978-3-030-58452-8_13.
[18] Liu Z, Lin Y T, Cao Y, et al. Swin transformer:hierarchical vision transformer using shifted windows[EB/OL]. arXiv:2103.14030v2. (2021-08-17)[2021-11-11]. https://arxiv.org/abs/2103.14030v2.
[19] Foga S, Scaramuzza P L, Guo S, et al. Cloud detection algorithm comparison and validation for operational Landsat data products[J]. Remote Sensing of Environment, 2017, 194:379-390. DOI:10.1016/j.rse.2017.03.026.
[20] He Q B, Sun X, Yan Z Y, et al. DABNet:deformable contextual and boundary-weighted network for cloud detection in remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 5474, PP(99):1-16. DOI:10.1109/TGRS.2020.3045474.