多重残差网络的多光谱遥感图像锐化方法*

doi:10.7523/j.ucas.2023.086

摘要/Abstract

摘要： 针对传统的遥感图像锐化方法通常会导致锐化图像光谱失真的问题与目前基于深度学习的锐化方法利用网络层之间信息不充分的问题,本文结合深度卷积神经网络和残差网络的特性,提出一种多重残差网络的多光谱遥感图像锐化方法。本文方法利用深度卷积网络和残差网络,设计深度残差模块,通过堆叠深度残差模块来提取图像深次层的空间和光谱特征,同时利用残差建立起子块与子块之间的跳跃连接,将梯度信息传递到更深的网络,避免梯度爆炸问题,使网络更加的高效。实验基于WorldView-2的多光谱图像和全色图像进行模拟实验与真实实验,将实验结果与传统方法和现有深度学习方法进行比较。结果表明,本文方法改善了传统方法存在的光谱失真现象;相较于现有的深度学习方法,本文方法能够学习到更深层次的图像特征,更好地保留了图像的空间与光谱信息,本文方法的全局相对光谱损失（error relative globale adimensionnelle de synthèse,ERGAS）、光谱角映射（spectral angle mapper,SAM）、空间相关系数（spatial correlation coefficient,SCC）、整体质量评价指标（universal image quality index,UIQI）和全局融合质量评价指标（${{Q}^{2n}}$）分别比深度卷积锐化网络方法提高了24.3%、26.7%、6.2%、4.6%、6.3%。通过主观视觉评价、客观定量评价和光谱曲线表明,本文方法相比于传统的锐化算法以及常用的深度学习锐化算法,在空间分辨率和光谱分辨率上都有显著的提升,特别是对于地物环境复杂条件下的遥感图像。

关键词: 遥感图像锐化方法, 深度学习, 多光谱遥感图像, 卷积神经网络, 残差网络

Abstract: This paper proposes a multi-spectral remote sensing image sharpening method based on a deep convolutional neural network and residual network. The method addresses the problems of spectral distortion in traditional remote sensing image sharpening methods and insufficient information utilization between network layers in current deep learning-based methods. The proposed method uses the depth convolution and residual network to design the depth residual module to extract the spatial and spectral features of the deep image. Additionally, residual connections between sub-blocks are established to transmit gradient information to deeper networks and avoid gradient explosion problems, making the network more efficient. Experiments are conducted on simulated and real-world multi-spectral images from WorldView-2, and the results are compared with traditional and existing deep learning-based methods. The proposed method improves the spectral distortion phenomenon and learns deeper image features to better preserve the spatial and spectral information of the image. The proposed method outperforms the deep convolutional sharpening network method in terms of various evaluation metrics, including ERGAS, SAM, SCC, UIQI, and the global fusion quality evaluation index. The proposed method improves these metrics by 24.3%, 26.7%,6.2%,4.6% and 6.3% respectively. Subjective and objective evaluations and spectral curve also indicate that the proposed method significantly improves the spatial and spectral resolution of remote sensing images, especially under complex environmental conditions.

Key words: remote sensing image pan-sharpening method, deep learning, multispectral remote sensing image, convolutional neural network, residual network

中图分类号:

TP751

周庆泽, 郭擎. 多重残差网络的多光谱遥感图像锐化方法^*[J]. 中国科学院大学学报, DOI: 10.7523/j.ucas.2023.086.

ZHOU Qingze, GUO Qing. Multispectral remote sensing image pan-sharpening method based on multi-residual network[J]. Journal of University of Chinese Academy of Sciences, DOI: 10.7523/j.ucas.2023.086.

参考文献

[1] 孙洪泉, 窦闻, 易文斌. 遥感图像融合的研究现状、困境及发展趋势探讨[J]. 遥感信息, 2011, 26(1): 104-108. DOI: 10.3969/j.issn.1000-3177.2011.01.021.
[2] Dadrass Javan F, Samadzadegan F, Mehravar S, et al.A review of image fusion techniques for pan-sharpening of high-resolution satellite imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 171: 101-117. DOI: 10.1016/j.isprsjprs.2020.11.001.
[3] 王华伟. 基于IHS变换的多光谱和全色图像融合算法研究[D]. 成都: 西南交通大学, 2016.
[4] Ranchin T, Wald L.Fusion of high spatial and spectral resolution images: the ARSIS concept and its implementation[J]. Photogrammetric Engineering and Remote Sensing, 2000, 66: 49-61.
[5] Aharon M, Elad M, Bruckstein A.K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation[J]. IEEE Transactions on Signal Processing, 2006, 54(11): 4311-4322. DOI: 10.1109/TSP.2006.881199.
[6] Nasr A, Darwis A and Shaheen S. Use of intensity-hue-saturation (IHS) transformations in change detection of multitemporal remotely sensed data[C]//Conference on Image and Signal Processing for Remote Sensing Ⅶ Sep 18-21, 2001, Toulouse, France.National Authority for Remote Sensing and Space Sciences, 23 Joseph Proz Tito st. El-Nozha El-Gedida, P.O. Box: 1564 Alf-Maskan, Cairo, Egypt, 2001.DOI:10.1117/12.454166
[7] Laben C A, Brower B V. Process for enhancing the spatial resolution of multispectral imagery using pan-sharpening: US6011875[P].2000-01-04.
[8] Tu T M, Su S C, Shyu H C, et al.A new look at IHS-like image fusion methods[J]. Information Fusion, 2001, 2(3): 177-186. DOI: 10.1016/S1566-2535(01)00036-7.
[9] Abdolahpoor A, Kabiri P.New texture-based pansharpening method using wavelet packet transform and PCA[J]. International Journal of Wavelets, Multiresolution and Information Processing, 2020, 18(4): 2050025. DOI: 10.1142/s0219691320500253.
[10] Vivone G.Robust band-dependent spatial-detail approaches for panchromatic sharpening[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(9): 6421-6433. DOI: 10.1109/TGRS.2019.2906073.
[11] Aiazzi B, Baronti S, Selva M.Improving component substitution pansharpening through multivariate regression of MS $ $Pan data[J]. IEEE Transactions on Geoscience and Remote Sensing, 2007, 45(10): 3230-3239. DOI: 10.1109/TGRS.2007.901007.
[12] Choi J, Yu K, Kim Y.A new adaptive component-substitution-based satellite image fusion by using partial replacement[J]. IEEE Transactions on Geoscience and Remote Sensing, 2011, 49(1): 295-309. DOI: 10.1109/TGRS.2010.2051674.
[13] Vivone G, Alparone L, Chanussot J, et al.A critical comparison among pansharpening algorithms[J]. IEEE Transactions on Geoscience and Remote Sensing,2015,53(5): 2565-2586.DOI: 10.1109/TGRS.2014.2361734.
[14] Kaplan N H, Erer I.Bilateral filtering-based enhanced pansharpening of multispectral satellite images[J]. IEEE Geoscience and Remote Sensing Letters, 2014, 11(11): 1941-1945. DOI: 10.1109/LGRS.2014.2314389.
[15] Liu J M, Liang S L.Pan-sharpening using a guided filter[J]. International Journal of Remote Sensing, 2016, 37(8): 1777-1800. DOI: 10.1080/01431161.2016.1163749.
[16] Fang F M, Li F, Shen C M, et al.A variational approach for pan-sharpening[J]. IEEE Transactions on Image Processing, 2013, 22(7): 2822-2834. DOI: 10.1109/TIP.2013.2258355.
[17] Azarang A, Ghassemian H.A new pansharpening method using multi resolution analysis framework and deep neural networks[C]//2017 3rd International Conference on Pattern Recognition and Image Analysis (IPRIA). April 19-20, 2017, Shahrekord, Iran. IEEE, 2017: 1-6. DOI: 10.1109/PRIA.2017.7983017.
[18] Zhong J Y, Yang B, Huang G Y, et al.Remote sensing image fusion with convolutional neural network[J]. Sensing and Imaging, 2016, 17(1): 10. DOI: 10.1007/s11220-016-0135-6.
[19] Masi G, Cozzolino D, Verdoliva L, et al.Pansharpening by convolutional neural networks[J]. Remote Sensing, 2016, 8(7): 594. DOI: 10.3390/rs8070594.
[20] Rao Y Z, He L, Zhu J W.A residual convolutional neural network for pan-shaprening[C]//2017 International Workshop on Remote Sensing with Intelligent Processing (RSIP). May 18-21, 2017, Shanghai, China. IEEE, 2017: 1-4. DOI: 10.1109/RSIP.2017.7958807.
[21] Wei Y C, Yuan Q Q, Shen H F, et al.Boosting the accuracy of multispectral image pansharpening by learning a deep residual network[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(10): 1795-1799. DOI: 10.1109/LGRS.2017.2736020.
[22] Yang J F, Fu X Y, Hu Y W, et al.PanNet: a deep network architecture for pan-sharpening[C]//2017 IEEE International Conference on Computer Vision (ICCV). October 22-29, 2017, Venice, Italy. IEEE, 2017: 1753-1761. DOI: 10.1109/ICCV.2017.193.
[23] Liu X Y, Wang Y H, Liu Q J. Remote sensing image fusion based on two-stream fusion network[C]//International Conference on Multimedia Modeling. Cham: Springer, 2018: 428-439.10.1007/978-3-319-73603-7_35.
[24] He K M, Sun J.Convolutional neural networks at constrained time cost[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 7-12, 2015, Boston, MA, USA. IEEE, 2015: 5353-5360. DOI: 10.1109/CVPR.2015.7299173.
[25] Stephan M, Hoffman Matthew D, Blei David M.Stochastic gradient descent as approximate Bayesian inference[J]. Journal of Machine Learning Research, 2017, 18: 4873-4907.
[26] Wald L, Ranchin T, Mangolini M.Fusion of satellite images of different spatial resolutions: assessing the quality of resulting images[J]. Photogrammetric Engineering and Remote Sensing, 1997, 63(6): 691-699.

多重残差网络的多光谱遥感图像锐化方法^*

Multispectral remote sensing image pan-sharpening method based on multi-residual network

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

访问统计

联系我们

[1]	李雪源, 韩丛英. Actor-critic框架下的二次指派问题求解方法[J]. 中国科学院大学学报, 2024, 41(2): 275-284.
[2]	李元, 张栖, 朱建明, 焦建彬. 基于多层次深度模型的社交网络核心谣言传播节点识别[J]. 中国科学院大学学报, 2024, 41(1): 136-144.
[3]	董文豪, 张怀. 基于迁移学习的岩屑岩性识别[J]. 中国科学院大学学报, 2023, 40(6): 743-750.
[4]	张萌, 潘志刚. 基于分层模糊聚类和小波卷积神经网络的SAR图像变化检测算法[J]. 中国科学院大学学报, 2023, 40(5): 637-646.
[5]	肖俊, 石光田. 三维点云去噪技术[J]. 中国科学院大学学报, 2023, 40(5): 577-595.
[6]	姚沐风, 昝露洋, 李柏鹏, 李庆亭, 陈正超. 基于CAR-Siamese网络的高分辨率遥感图像建筑物变化检测[J]. 中国科学院大学学报, 2023, 40(3): 380-387.
[7]	盛晓光, 王颖, 张迎伟, 项若曦, 付红萍. WILS：面向学业预警的非均衡增量式学习方法[J]. 中国科学院大学学报, 2023, 40(3): 422-432.
[8]	庄子俊, 袁晓兵, 裴俊, 王国辉, 刘建坡. 基于无监督表达学习的森林地貌特征建模及林火易发性评估[J]. 中国科学院大学学报, 2023, 40(2): 227-239.
[9]	霍鑫怡, 李焱磊, 陈龙永, 张福博, 孙巍. 基于卷积注意力和胶囊网络的 SAR少样本目标识别方法[J]. 中国科学院大学学报, 2022, 39(6): 783-792.
[10]	陈若男, 彭玲, 刘玉菲, 卫志超, 吕蓓茹, 陈德跃. 引入空间距离信息的城郊山区道路提取与应用[J]. 中国科学院大学学报, 2022, 39(5): 658-667.
[11]	古煜民, 阎福礼. 基于不同骨架UNet++网络的建筑物提取[J]. 中国科学院大学学报, 2022, 39(4): 512-523.
[12]	林大权, 范睿, 张良峰. 基于高维特征的图像对抗攻击算法[J]. 中国科学院大学学报, 2022, 39(3): 421-431.
[13]	赵敏钧, 赵亚伟, 赵雅捷, 罗刚. 一种新的基于深度学习的重叠关系联合抽取模型（英文）[J]. 中国科学院大学学报, 2022, 39(2): 240-251.
[14]	周龙, 王伟强, 吕科. FACR：一种快速且准确的车辆识别器[J]. 中国科学院大学学报, 2021, 38(1): 130-136.
[15]	胡莉娜, 高盛华. 基于层次化标签的人体解析[J]. 中国科学院大学学报, 2020, 37(6): 820-827.