面向真实场景的高分辨率遥感影像超分辨率重建*

doi:10.7523/j.ucas.2024.054

中国科学院大学学报

面向真实场景的高分辨率遥感影像超分辨率重建^*

赵佳祎^1,2,3, 马勇^1,2†, 陈甫², 姚武韬², 尚二萍², 仉淑艳², 龙安⁴

1.海南省地球观测重点实验室, 中国科学院空天信息研究院海南研究院海南三亚 572029;
2.中国科学院空天信息创新研究院,北京100094;
3.中国科学院大学,北京100049;
4.广西壮族自治区环境应急与事故调查中心 ,南宁 530028

收稿日期:2023-01-30 修回日期:2024-05-21 发布日期:2024-06-11
通讯作者: ^†E-mail：mayong@aircas.ac.cn
基金资助:
^*海南省重点研发计划（ZDYF2021SHFZ260）、海南自然科学青年基金（520QN295）和广西创新驱动发展专项资金项目（桂科AA20302022）资助

Super-resolution reconstruction of high-resolution remote sensing images for real scenes

ZHAO Jiayi^1,2,3, MA Yong^1,2, CHEN Fu², YAO Wutao², SHANG Erping², ZHANG Shuyang², LONG An⁴

1. Key Laboratory of Earth Observation in Hainan Province, Hainan Research Institute, Aerospace Information Research Institute, Chinese Academy of Sciences, Sanya 572029, Hainan, China;
2 Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China;
3 University of Chinese Academy of Sciences, Beijing 100049, China;
4. Guangxi Zhuang Autonomous Region Environmental Emergency and Accident Investigation Center, Nanning 530028, China

Received:2023-01-30 Revised:2024-05-21 Published:2024-06-11

摘要/Abstract

摘要： 超分辨率技术以其灵活、低成本等特点已经成为构建高分辨率数据集以及补充高分辨率影像不足的重要手段。相较于自然图像,反映真实场景的遥感影像由于自身的复杂性与特殊性加大了超分辨率任务难度,同时对于遥感影像,传统的深度学习模型虽然在一定程度上可以提高分辨率,但是对于地物细节和纹理的提升还存在着很大的不足。因此本文在对抗生成网络模型的基础上,融合通道-空间注意力来增强网络的特征学习能力,并使用伪影抑制策略来区分平滑区域和细节丰富区域,让网络专注于细节丰富区域,抑制伪影的产生。在高分卫星数据上的大量实验表明,本文的定量指标和视觉质量优于目前主流的方法。

关键词: 超分辨率, 遥感影像, 对抗生成网络, 融合通道-空间注意力, 伪影抑制

Abstract: Super-resolution technology has become an important tool for reconstructing high-resolution datasets and supplementing the shortage of high-resolution images with its characteristics of flexibility and low cost. Compared with natural images, remote sensing images of real scenes are complexity and specificity, which make super-resolution tasks more difficult. Meanwhile, for remote sensing images, traditional deep learning models can improve the resolution, but there is still a great deficiency of improvement for the details and textures of the ground objects. Therefore, based on the generative adversarial network model, this paper fuses channel-space attention to enhance the feature learning capability of the network and use artifact suppression strategy to distinguish smooth regions from detail-rich regions, so that the network can focus more on detail-rich regions and suppress the generation of artifacts. Extensive experiments on GaoFen satellite data show that the quantitative metrics and visual quality of the method proposed in this paper are better than that of the current mainstream methods.

Key words: Super resolution, Remote sensing images, GAN, Fusing channel-space attention, Artifact suppression

中图分类号:

TP751

赵佳祎, 马勇, 陈甫, 姚武韬, 尚二萍, 仉淑艳, 龙安. 面向真实场景的高分辨率遥感影像超分辨率重建^*[J]. 中国科学院大学学报, DOI: 10.7523/j.ucas.2024.054.

ZHAO Jiayi, MA Yong, CHEN Fu, YAO Wutao, SHANG Erping, ZHANG Shuyang, LONG An. Super-resolution reconstruction of high-resolution remote sensing images for real scenes[J]. Journal of University of Chinese Academy of Sciences, DOI: 10.7523/j.ucas.2024.054.

参考文献

[1] Olivier R, Cao H Q.Nearest neighbor value interpolation[J]. International Journal of Advanced Computer Science and Applications, 2012, 3(4):25-30. DOI: 10.14569/ijacsa.2012.030405.
[2] Zhang X G.A new kind of super-resolution reconstruction algorithm based on the ICM and the bilinear interpolation[C]//2008 International Seminar on Future BioMedical Information Engineering. December 18-18, 2008, Wuhan, China. IEEE, 2009: 183-186. DOI: 10.1109/FBIE.2008.44.
[3] Zhang X G.A new kind of super-resolution reconstruction algorithm based on the ICM and the bicubic interpolation[C]//2008 International Symposium on Intelligent Information Technology Application Workshops. December 21-22, 2008, Shanghai, China. IEEE, 2008: 817-820. DOI: 10.1109/IITA.Workshops.2008.12.
[4] Rasti P, Demirel H, Anbarjafari G.Image resolution enhancement by using interpolation followed by iterative back projection[C]//2013 21st Signal Processing and Communications Applications Conference (SIU). April 24-26, 2013, Haspolat, Turkey. IEEE, 2013: 1-4. DOI: 10.1109/SIU.2013.6531593.
[5] Wheeler F W, Hoctor R T, Barrett E B.Super-resolution image synthesis using projections onto convex sets in the frequency domain[C]//Proc SPIE 5674, Computational Imaging III, 2005, 5674: 479-490. DOI: 10.1117/12.605436.
[6] 李利, 尹增山, 石神. 联合L1和L0先验模型的超分辨率重建算法[J]. 中国科学院大学学报, 2022, 39(3): 369-376. DOI: 10.7523/j.ucas.2020.0013.
[7] 浦剑, 张军平, 黄华. 超分辨率算法研究综述[J]. 山东大学学报(工学版), 2009, 39(1): 27-32.
[8] Dong C, Loy C C, He K M, et al.Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 38(2): 295-307. DOI: 10.1109/TPAMI.2015.2439281.
[9] Dong C, Loy C C, Tang X O. Accelerating the super-resolution convolutional neural network[C]//European Conference on Computer Vision. Cham: Springer, 2016: 391-407.10.1007/978-3-319-46475-6_25.
[10] Kim J, Lee J K, Lee K M.Accurate image super-resolution using very deep convolutional networks[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 27-30, 2016, Las Vegas, NV, USA. IEEE, 2016: 1646-1654. DOI: 10.1109/CVPR.2016.182.
[11] Lim B, Son S, Kim H, et al.Enhanced deep residual networks for single image super-resolution[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). July 21-26, 2017, Honolulu, HI, USA. IEEE, 2017: 1132-1140. DOI: 10.1109/CVPRW.2017.151.
[12] Zhang Y L, Li K P, Li K, et al. Image super-resolution using very deep residual channel attention networks[C]//European Conference on Computer Vision. Cham: Springer, 2018: 294-310.10.1007/978-3-030-01234-2_18.
[13] Ledig C, Theis L, Huszár F, et al.Photo-realistic single image super-resolution using a generative adversarial network[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA. IEEE, 2017: 105-114. DOI: 10.1109/CVPR.2017.19.
[14] Wang X T, Yu K, Wu S X, et al. ESRGAN: enhanced super-resolution generative adversarial networks[C]//European Conference on Computer Vision. Cham: Springer, 2019: 63-79.10.1007/978-3-030-11021-5_5.
[15] Wang X T, Yu K, Dong C, et al.Recovering realistic texture in image super-resolution by deep spatial feature transform[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018, Salt Lake City, UT, USA. IEEE, 2018: 606-615. DOI: 10.1109/CVPR.2018.00070.
[16] Szegedy C, Ioffe S, Vanhoucke V, et al.Inception-v4, inception-ResNet and the impact of residual connections on learning[C]//Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. February 4 - 9, 2017, San Francisco, California, USA. New York: ACM, 2017: 4278-4284. DOI: 10.5555/3298023.3298188.
[17] Huang H M, Lin L F, Tong R F, et al.UNet 3: A full-scale connected UNet for medical image segmentation[C]//ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). May 4-8, 2020, Barcelona, Spain. IEEE, 2020: 1055-1059. DOI: 10.1109/ICASSP40776.2020.9053405.
[18] Miyato T, Kataoka T, Koyama M, et al. Spectral normalization for generative adversarial networks[EB/OL].(2018-02-16)[2023-01-30] 2018: arXiv: 1802.05957.https://doi.org/10.48550/arXiv.1802.05957.
[19] Johnson J, Alahi A, Li F F. Perceptual losses for real-time style transfer and super-resolution[C]//European Conference on Computer Vision. Cham: Springer, 2016: 694-711.10.1007/978-3-319-46475-6_43.
[20] Bruna J, Sprechmann P, LeCun Y. Super-resolution with deep convolutional sufficient statistics[EB/OL]. (2015-11-18)[2023-01-30]2015: arXiv: 1511.05666.https://doi.org/10.48550/arXiv.1511.05666.
[21] Dosovitskiy A, Brox T.Generating images with perceptual similarity metrics based on deep networks[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems. New York: ACM, 2016: 658-666. DOI: 10.5555/3157096.3157170.
[22] Zhao H, Gallo O, Frosio I, et al.Loss functions for image restoration with neural networks[J]. IEEE Transactions on Computational Imaging, 2017, 3(1): 47-57. DOI: 10.1109/TCI.2016.2644865.
[23] Wang X T, Xie L B, Dong C, et al.Real-ESRGAN: Training real-world blind super-resolution with pure synthetic data[C]//2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW). October 11-17, 2021, Montreal, BC, Canada. IEEE, 2021: 1905-1914. DOI: 10.1109/ICCVW54120.2021.00217.
[24] Korhonen J, You J Y.Peak signal-to-noise ratio revisited: Is simple beautiful?[C]//2012 Fourth International Workshop on Quality of Multimedia Experience. July 5-7, 2012, Melbourne, VIC, Australia. IEEE, 2012: 37-38. DOI: 10.1109/QoMEX.2012.6263880.
[25] Wang Z, Bovik A C, Sheikh H R, et al.Image quality assessment: From error visibility to structural similarity[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2004, 13(4): 600-612. DOI: 10.1109/tip.2003.819861.
[26] Heusel M, Ramsauer H, Unterthiner T, et al.GANs trained by a two time-scale update rule converge to a local Nash equilibrium[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. December 4 - 9, 2017, Long Beach, California, USA. New York: ACM, 2017: 6629-6640. DOI: 10.5555/3295222.3295408.
[27] Zhang R, Isola P, Efros A A, et al.The unreasonable effectiveness of deep features as a perceptual metric[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018, Salt Lake City, UT, USA. IEEE, 2018: 586-595. DOI: 10.1109/CVPR.2018.00068.
[28] Mittal A, Soundararajan R, Bovik A C.Making a “completely blind” image quality analyzer[J]. IEEE Signal Processing Letters, 2013, 20(3): 209-212. DOI: 10.1109/LSP.2012.2227726.

面向真实场景的高分辨率遥感影像超分辨率重建^*

Super-resolution reconstruction of high-resolution remote sensing images for real scenes

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

访问统计

联系我们

[1]	石正一, 刘朔, 夏昊. 结合特征信息聚类分区的遥感影像配准方法[J]. 中国科学院大学学报, 2024, 41(1): 97-106.
[2]	郑美艳, 陈俊, 葛小青, 张红. 一种结合局部与半全局几何保持的影像匹配算法[J]. 中国科学院大学学报, 2024, 41(1): 107-116.
[3]	陈若男, 彭玲, 刘玉菲, 卫志超, 吕蓓茹, 陈德跃. 引入空间距离信息的城郊山区道路提取与应用[J]. 中国科学院大学学报, 2022, 39(5): 658-667.
[4]	古煜民, 阎福礼. 基于不同骨架UNet++网络的建筑物提取[J]. 中国科学院大学学报, 2022, 39(4): 512-523.
[5]	蔚瑞, 谢卓辰, 刘洁, 刘会杰. 基于图像超分辨重建的低轨卫星频谱感知数据空间分辨率提高方法[J]. 中国科学院大学学报, 2022, 39(3): 386-392.
[6]	李利, 尹增山, 石神. 联合L₁和L₀先验模型的超分辨率重建算法[J]. 中国科学院大学学报, 2022, 39(3): 369-376.
[7]	李森, 彭玲, 胡媛, 池天河. 基于FD-RCF的高分辨率遥感影像耕地边缘检测[J]. 中国科学院大学学报, 2020, 37(4): 483-489.
[8]	卢凯旋, 李国清, 陈正超, 昝露洋, 李柏鹏, 高建威. 基于负样本多通道优化SSD网络的钢铁厂提取[J]. 中国科学院大学学报, 2020, 37(3): 352-359.
[9]	常敬明, 金铭, 曾江源, 陈锟山. 随机介质场景下的时间反转成像效果分析[J]. 中国科学院大学学报, 2018, 35(1): 59-65.
[10]	刘亮, 姜小光, 刘善伟, 潘微, 雷宇, 张学华. WorldView-2影像的水深探测能力[J]. 中国科学院大学学报, 2017, 34(1): 64-68.
[11]	刘晓, 郭田德, 韩丛英, 李明强. 有去噪和尺度放大的单幅图像超分辨率重建[J]. 中国科学院大学学报, 2016, 33(5): 596-603.
[12]	刘桂林, 张落成, 赵金丽. 景观破碎化区域土地覆被面向对象分类研究——以皖南山区为例[J]. 中国科学院大学学报, 2014, 31(5): 626-631.
[13]	邓富亮, 范协裕, 王刚, 马娟. 一种FNEA影像分割算法中初始对象的快速构建方法[J]. 中国科学院大学学报, 2014, 31(4): 484-491.
[14]	姚振杰, 易卫东. 一种用于车牌识别的图像超分辨率算法[J]. 中国科学院大学学报, 2013, 30(1): 137-143.
[15]	袁建华; 殷学民; 邹谋炎. 一种有效的超分辨率重建计算方法[J]. 中国科学院大学学报, 2006, 23(4): 514-519.