Super-resolution reconstruction of high-resolution remote sensing images for real scenes

doi:10.7523/j.ucas.2024.054

Abstract

Abstract:

Super-resolution technology has become an important tool for reconstructing high-resolution datasets and supplementing the shortage of high-resolution images with its characteristics of flexibility and low cost. Compared with natural images， remote sensing images of real scenes are complex and specific， which make super-resolution tasks more difficult. Meanwhile， for remote sensing images， traditional deep learning models can improve the resolution， but there is still a great deficiency of improvement for the details and textures of the ground objects. Therefore， based on the generative adversarial network model， this paper fuses channel-space attention to enhance the feature learning capability of the network and use an artifact suppression strategy to distinguish smooth regions from detail-rich regions， so that the network can focus more on detail-rich regions and suppress the generation of artifacts. Extensive experiments on GaoFen satellite data show that the quantitative metrics and visual quality of the method proposed in this paper are better than those of the current mainstream methods.

Key words: super-resolution, remote sensing images, GAN, fusing channel-space attention, artifact suppression

CLC Number:

TP751

Jiayi ZHAO, Yong MA, Fu CHEN, Wutao YAO, Erping SHANG, Shuyan ZHANG, An LONG. Super-resolution reconstruction of high-resolution remote sensing images for real scenes[J]. Journal of University of Chinese Academy of Sciences, 2026, 43(1): 80-92.

Figures/Tables 13

Fig.1 Overall flow chart

Table 1 Satellite and spatial resolution ofthe dataset

卫星	全色波段分辨率	多光谱波段分辨率
GF1	2.0	8.0
GF1B/C/D	2.0	8.0
GF2	0.8（星下点）	3.2（星下点）
GF6	1.0	2.0
GF7	0.8	3.2

Fig.2 Network framework diagram

Fig.3 Internal structure of CSA

Fig.4 Comparison of the HR image with the real LR image

Table 2 Detailed configuration of the model

参数	设置
批次规格	8
训练迭代次数	450 000
优化方法	Adam, $β 1 = 0.9$ , $β 2 = 0.99$
学习率	1e-4

Table 2 Detailed configuration of the model

参数	设置
批次规格	8
训练迭代次数	450 000
优化方法	Adam, $β 1 = 0.9$ , $β 2 = 0.99$
学习率	1e-4

Fig. 5 Comparison of the algorithm in this paper with the real HR image

Table 3 Comparison of metrics of different models on the data set

模型	PSNR	SSIM	FID	LPIPS
Bicubic	16.894 1	0.574 8	0.545 4	71.680 0
SRResNet	26.326 5	0.662 6	0.423 0	66.757 3
EDSR	27.417 6	0.717 2	0.352 0	51.435 7
RCAN	26.685 1	0.682 8	0.399 5	54.029 1
SRGAN	24.772 9	0.592 2	0.289 2	20.051 4
ESRGAN	24.822 0	0.590 6	0.282 9	19.126 6
本文	25.741 8	0.647 4	0.246 5	18.247 6

Fig.6 Comparison of recovery results of different models in different scenarios

Table 4 Comparison of indicators in ablation experiments

添加的模块	PSNR	SSIM	FID	LPIPS
无	24.892 7	0.614 6	0.274 1	18.947 2
伪影抑制	25.472 8	0.632 1	0.254 2	18.627 5
融合注意力	25.501 7	0.638 6	0.253 3	18.872 1
伪影抑制+融合注意力	25.741 8	0.647 4	0.246 5	18.247 6

Fig.7 Comparison of recovery results in ablation experiments

Table 5 Quantitative evaluation of 2x super-resolution reconstruction results of different models in different regions

区域	SRResNet	EDSR	RCAN	SRGAN	ESRGAN	本文
陕西汉中	8.215 6	8.157 9	8.519 6	6.254 9	6.545 5	6.182 7
北京	6.683 7	8.013 9	8.110 7	4.769 0	5.292 6	4.743 9

Fig.8 Results of the migration experiment at different times and regions

References 28

[1]	Olivier R， Cao H Q. Nearest neighbor value interpolation［J］. International Journal of Advanced Computer Science and Applications， 2012， 3（4）：25-30. DOI： 10.14569/ijacsa.2012.030405 .
[2]	Zhang X G. A new kind of super-resolution reconstruction algorithm based on the ICM and the bilinear interpolation［C］//2008 International Seminar on Future BioMedical Information Engineering. December 18-18， 2008， Wuhan， China. IEEE， 2009： 183-186. DOI： 10.1109/FBIE.2008.44 .
[3]	Zhang X G. A new kind of super-resolution reconstruction algorithm based on the ICM and the bicubic interpolation［C］//2008 International Symposium on Intelligent Information Technology Application Workshops. December 21-22， 2008， Shanghai， China. IEEE， 2008： 817-820. DOI： 10.1109/IITA.Workshops.2008.12 .
[4]	Rasti P， Demirel H， Anbarjafari G. Image resolution enhancement by using interpolation followed by iterative back projection［C］//2013 21st Signal Processing and Communications Applications Conference （SIU）. April 24-26， 2013， Haspolat， Turkey. IEEE， 2013： 1-4. DOI： 10.1109/SIU.2013.6531593 .
[5]	Wheeler F W， Hoctor R T， Barrett E B. Super-resolution image synthesis using projections onto convex sets in the frequency domain［C］//Proc SPIE 5674， Computational Imaging III， 2005， 5674： 479-490. DOI： 10.1117/12.605436 .
[6]	李利，尹增山，石神. 联合L1和L0先验模型的超分辨率重建算法［J］. 中国科学院大学学报， 2022， 39（3）： 369-376. DOI： 10.7523/j.ucas.2020.0013 .
[7]	浦剑，张军平，黄华. 超分辨率算法研究综述［J］. 山东大学学报（工学版）， 2009， 39（1）： 27-32.
[8]	Dong C， Loy C C， He K M， et al. Image super-resolution using deep convolutional networks［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2015， 38（2）： 295-307. DOI： 10.1109/TPAMI.2015.2439281 .
[9]	Dong C， Loy C C， Tang X O. Accelerating the super-resolution convolutional neural network［C］//European Conference on Computer Vision. Cham： Springer， 2016： 391-407.10.1007/978-3-319-46475-6_25.
[10]	Kim J， Lee J K， Lee K M. Accurate image super-resolution using very deep convolutional networks［C］//2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. June 27-30， 2016， Las Vegas， NV， USA. IEEE， 2016： 1646-1654. DOI： 10.1109/CVPR.2016.182 .
[11]	Lim B， Son S， Kim H， et al. Enhanced deep residual networks for single image super-resolution［C］//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops （CVPRW）. July 21-26， 2017， Honolulu， HI， USA. IEEE， 2017： 1132-1140. DOI： 10.1109/CVPRW.2017.151 .
[12]	Zhang Y L， Li K P， Li K， et al. Image super-resolution using very deep residual channel attention networks［C］//European Conference on Computer Vision. Cham： Springer， 2018： 294-310.10.1007/978-3-030-01234-2_18.
[13]	Ledig C， Theis L， Huszár F， et al. Photo-realistic single image super-resolution using a generative adversarial network［C］//2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Honolulu， HI， USA. IEEE， 2017： 105-114. DOI： 10.1109/CVPR.2017.19 .
[14]	Wang X T， Yu K， Wu S X， et al. ESRGAN： enhanced super-resolution generative adversarial networks［C］//European Conference on Computer Vision. Cham： Springer， 2019： 63-79.10.1007/978-3-030-11021-5_5.
[15]	Wang X T， Yu K， Dong C， et al. Recovering realistic texture in image super-resolution by deep spatial feature transform［C］//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23， 2018， Salt Lake City， UT， USA. IEEE， 2018： 606-615. DOI： 10.1109/CVPR.2018.00070 .
[16]	Szegedy C， Ioffe S， Vanhoucke V， et al. Inception-v4， inception-ResNet and the impact of residual connections on learning［C］//Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. February 4 - 9， 2017， San Francisco， California， USA. New York： ACM， 2017： 4278-4284. DOI： 10.5555/3298023.3298188 .
[17]	Huang H M， Lin L F， Tong R F， et al. UNet 3： a full-scale connected UNet for medical image segmentation［C］//ICASSP 2020 - 2020 IEEE International Conference on Acoustics， Speech and Signal Processing （ICASSP）. May 4-8， 2020， Barcelona， Spain. IEEE， 2020： 1055-1059. DOI： 10.1109/ICASSP40776.2020.9053405 .
[18]	Miyato T， Kataoka T， Koyama M， et al. Spectral normalization for generative adversarial networks［EB/OL］. arXiv 2018： 1802.05957. （2018-02-16）［2023-01-30］. .
[19]	Johnson J， Alahi A， Li F F. Perceptual losses for real-time style transfer and super-resolution［C］//European Conference on Computer Vision. Cham： Springer， 2016： 694-711.10.1007/978-3-319-46475-6_43.
[20]	Bruna J， Sprechmann P， LeCun Y. Super-resolution with deep convolutional sufficient statistics［EB/OL］. arXiv 2015： 1511.05666. （2015-11-18）［2023-01-30］. .
[21]	Dosovitskiy A， Brox T. Generating images with perceptual similarity metrics based on deep networks［C］//Proceedings of the 30th International Conference on Neural Information Processing Systems. New York： ACM， 2016： 658-666. DOI： 10.5555/3157096.3157170 .
[22]	Zhao H， Gallo O， Frosio I， et al. Loss functions for image restoration with neural networks［J］. IEEE Transactions on Computational Imaging， 2017， 3（1）： 47-57. DOI： 10.1109/TCI.2016.2644865 .
[23]	Wang X T， Xie L B， Dong C， et al. Real-ESRGAN： training real-world blind super-resolution with pure synthetic data［C］//2021 IEEE/CVF International Conference on Computer Vision Workshops （ICCVW）. October 11-17， 2021， Montreal， BC， Canada. IEEE， 2021： 1905-1914. DOI： 10.1109/ICCVW54120.2021.00217 .
[24]	Korhonen J， You J Y. Peak signal-to-noise ratio revisited： Is simple beautiful？［C］//2012 Fourth International Workshop on Quality of Multimedia Experience. July 5-7， 2012， Melbourne， VIC， Australia. IEEE， 2012： 37-38. DOI： 10.1109/QoMEX.2012.6263880 .
[25]	Wang Z， Bovik A C， Sheikh H R， et al. Image quality assessment： from error visibility to structural similarity［J］. IEEE Transactions on Image Processing： a Publication of the IEEE Signal Processing Society， 2004， 13（4）： 600-612. DOI： 10.1109/tip.2003.819861 .
[26]	Heusel M， Ramsauer H， Unterthiner T， et al. GANs trained by a two time-scale update rule converge to a local Nash equilibrium［C］//Proceedings of the 31st International Conference on Neural Information Processing Systems. December 4 - 9， 2017， Long Beach， California， USA. New York： ACM， 2017： 6629-6640. DOI： 10.5555/3295222.3295408 .
[27]	Zhang R， Isola P， Efros A A， et al. The unreasonable effectiveness of deep features as a perceptual metric［C］//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23， 2018， Salt Lake City， UT， USA. IEEE， 2018： 586-595. DOI： 10.1109/CVPR.2018.00068 .
[28]	Mittal A， Soundararajan R， Bovik A C. Making a “completely blind” image quality analyzer［J］. IEEE Signal Processing Letters， 2013， 20（3）： 209-212. DOI： 10.1109/LSP.2012.2227726 .