Super-resolution reconstruction of high-resolution remote sensing images for real scenes

doi:10.7523/j.ucas.2024.054

Abstract

Abstract: Super-resolution technology has become an important tool for reconstructing high-resolution datasets and supplementing the shortage of high-resolution images with its characteristics of flexibility and low cost. Compared with natural images, remote sensing images of real scenes are complexity and specificity, which make super-resolution tasks more difficult. Meanwhile, for remote sensing images, traditional deep learning models can improve the resolution, but there is still a great deficiency of improvement for the details and textures of the ground objects. Therefore, based on the generative adversarial network model, this paper fuses channel-space attention to enhance the feature learning capability of the network and use artifact suppression strategy to distinguish smooth regions from detail-rich regions, so that the network can focus more on detail-rich regions and suppress the generation of artifacts. Extensive experiments on GaoFen satellite data show that the quantitative metrics and visual quality of the method proposed in this paper are better than that of the current mainstream methods.

Key words: Super resolution, Remote sensing images, GAN, Fusing channel-space attention, Artifact suppression

CLC Number:

TP751

ZHAO Jiayi, MA Yong, CHEN Fu, YAO Wutao, SHANG Erping, ZHANG Shuyang, LONG An. Super-resolution reconstruction of high-resolution remote sensing images for real scenes[J]. Journal of University of Chinese Academy of Sciences, DOI: 10.7523/j.ucas.2024.054.

References

[1] Olivier R, Cao H Q.Nearest neighbor value interpolation[J]. International Journal of Advanced Computer Science and Applications, 2012, 3(4):25-30. DOI: 10.14569/ijacsa.2012.030405.
[2] Zhang X G.A new kind of super-resolution reconstruction algorithm based on the ICM and the bilinear interpolation[C]//2008 International Seminar on Future BioMedical Information Engineering. December 18-18, 2008, Wuhan, China. IEEE, 2009: 183-186. DOI: 10.1109/FBIE.2008.44.
[3] Zhang X G.A new kind of super-resolution reconstruction algorithm based on the ICM and the bicubic interpolation[C]//2008 International Symposium on Intelligent Information Technology Application Workshops. December 21-22, 2008, Shanghai, China. IEEE, 2008: 817-820. DOI: 10.1109/IITA.Workshops.2008.12.
[4] Rasti P, Demirel H, Anbarjafari G.Image resolution enhancement by using interpolation followed by iterative back projection[C]//2013 21st Signal Processing and Communications Applications Conference (SIU). April 24-26, 2013, Haspolat, Turkey. IEEE, 2013: 1-4. DOI: 10.1109/SIU.2013.6531593.
[5] Wheeler F W, Hoctor R T, Barrett E B.Super-resolution image synthesis using projections onto convex sets in the frequency domain[C]//Proc SPIE 5674, Computational Imaging III, 2005, 5674: 479-490. DOI: 10.1117/12.605436.
[6] 李利, 尹增山, 石神. 联合L1和L0先验模型的超分辨率重建算法[J]. 中国科学院大学学报, 2022, 39(3): 369-376. DOI: 10.7523/j.ucas.2020.0013.
[7] 浦剑, 张军平, 黄华. 超分辨率算法研究综述[J]. 山东大学学报(工学版), 2009, 39(1): 27-32.
[8] Dong C, Loy C C, He K M, et al.Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 38(2): 295-307. DOI: 10.1109/TPAMI.2015.2439281.
[9] Dong C, Loy C C, Tang X O. Accelerating the super-resolution convolutional neural network[C]//European Conference on Computer Vision. Cham: Springer, 2016: 391-407.10.1007/978-3-319-46475-6_25.
[10] Kim J, Lee J K, Lee K M.Accurate image super-resolution using very deep convolutional networks[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 27-30, 2016, Las Vegas, NV, USA. IEEE, 2016: 1646-1654. DOI: 10.1109/CVPR.2016.182.
[11] Lim B, Son S, Kim H, et al.Enhanced deep residual networks for single image super-resolution[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). July 21-26, 2017, Honolulu, HI, USA. IEEE, 2017: 1132-1140. DOI: 10.1109/CVPRW.2017.151.
[12] Zhang Y L, Li K P, Li K, et al. Image super-resolution using very deep residual channel attention networks[C]//European Conference on Computer Vision. Cham: Springer, 2018: 294-310.10.1007/978-3-030-01234-2_18.
[13] Ledig C, Theis L, Huszár F, et al.Photo-realistic single image super-resolution using a generative adversarial network[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA. IEEE, 2017: 105-114. DOI: 10.1109/CVPR.2017.19.
[14] Wang X T, Yu K, Wu S X, et al. ESRGAN: enhanced super-resolution generative adversarial networks[C]//European Conference on Computer Vision. Cham: Springer, 2019: 63-79.10.1007/978-3-030-11021-5_5.
[15] Wang X T, Yu K, Dong C, et al.Recovering realistic texture in image super-resolution by deep spatial feature transform[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018, Salt Lake City, UT, USA. IEEE, 2018: 606-615. DOI: 10.1109/CVPR.2018.00070.
[16] Szegedy C, Ioffe S, Vanhoucke V, et al.Inception-v4, inception-ResNet and the impact of residual connections on learning[C]//Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. February 4 - 9, 2017, San Francisco, California, USA. New York: ACM, 2017: 4278-4284. DOI: 10.5555/3298023.3298188.
[17] Huang H M, Lin L F, Tong R F, et al.UNet 3: A full-scale connected UNet for medical image segmentation[C]//ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). May 4-8, 2020, Barcelona, Spain. IEEE, 2020: 1055-1059. DOI: 10.1109/ICASSP40776.2020.9053405.
[18] Miyato T, Kataoka T, Koyama M, et al. Spectral normalization for generative adversarial networks[EB/OL].(2018-02-16)[2023-01-30] 2018: arXiv: 1802.05957.https://doi.org/10.48550/arXiv.1802.05957.
[19] Johnson J, Alahi A, Li F F. Perceptual losses for real-time style transfer and super-resolution[C]//European Conference on Computer Vision. Cham: Springer, 2016: 694-711.10.1007/978-3-319-46475-6_43.
[20] Bruna J, Sprechmann P, LeCun Y. Super-resolution with deep convolutional sufficient statistics[EB/OL]. (2015-11-18)[2023-01-30]2015: arXiv: 1511.05666.https://doi.org/10.48550/arXiv.1511.05666.
[21] Dosovitskiy A, Brox T.Generating images with perceptual similarity metrics based on deep networks[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems. New York: ACM, 2016: 658-666. DOI: 10.5555/3157096.3157170.
[22] Zhao H, Gallo O, Frosio I, et al.Loss functions for image restoration with neural networks[J]. IEEE Transactions on Computational Imaging, 2017, 3(1): 47-57. DOI: 10.1109/TCI.2016.2644865.
[23] Wang X T, Xie L B, Dong C, et al.Real-ESRGAN: Training real-world blind super-resolution with pure synthetic data[C]//2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW). October 11-17, 2021, Montreal, BC, Canada. IEEE, 2021: 1905-1914. DOI: 10.1109/ICCVW54120.2021.00217.
[24] Korhonen J, You J Y.Peak signal-to-noise ratio revisited: Is simple beautiful?[C]//2012 Fourth International Workshop on Quality of Multimedia Experience. July 5-7, 2012, Melbourne, VIC, Australia. IEEE, 2012: 37-38. DOI: 10.1109/QoMEX.2012.6263880.
[25] Wang Z, Bovik A C, Sheikh H R, et al.Image quality assessment: From error visibility to structural similarity[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2004, 13(4): 600-612. DOI: 10.1109/tip.2003.819861.
[26] Heusel M, Ramsauer H, Unterthiner T, et al.GANs trained by a two time-scale update rule converge to a local Nash equilibrium[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. December 4 - 9, 2017, Long Beach, California, USA. New York: ACM, 2017: 6629-6640. DOI: 10.5555/3295222.3295408.
[27] Zhang R, Isola P, Efros A A, et al.The unreasonable effectiveness of deep features as a perceptual metric[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018, Salt Lake City, UT, USA. IEEE, 2018: 586-595. DOI: 10.1109/CVPR.2018.00068.
[28] Mittal A, Soundararajan R, Bovik A C.Making a “completely blind” image quality analyzer[J]. IEEE Signal Processing Letters, 2013, 20(3): 209-212. DOI: 10.1109/LSP.2012.2227726.