分块尺寸对大场景遥感图像空间超分应用的影响分析*

doi:10.7523/j.ucas.2024.019

中国科学院大学学报

分块尺寸对大场景遥感图像空间超分应用的影响分析^*

孙瑞奇^1,2, 张文娟^1,†, 李震¹, 马雪松³, 梅君林^1,2

1 中国科学院空天信息创新研究院,北京 100089;
2 中国科学院大学,北京 100049;
3 中国矿业大学（北京）, 北京 100083

收稿日期:2024-01-19 修回日期:2024-04-03 发布日期:2024-05-09
通讯作者: † E-mail: zhangwj@aircas.ac.cn
基金资助:
* 国家自然科学基金（42201503）资助

Analysis of the effect of patch size on super-resolution of large scene remote sensing images

SUN Ruiqi^1,2, ZHANG Wenjuan^1,†, LI Zhen¹, MA Xuesong³, MEI Junlin^1,2

1 Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China;
2 University of Chinese Academy of Sciences, Beijing 100049, China;
3 China University of Mining and Technology-Beijing, Beijing 100083, China

Received:2024-01-19 Revised:2024-04-03 Published:2024-05-09

摘要/Abstract

摘要： 遥感图像覆盖范围广、数据量大,深度学习超分算法需分块处理。目前分块尺寸对整图超分精度和效率的影响缺乏研究。本文以高空间分辨率（高分）遥感图像为对象,采用3种典型超分模型进行了9组不同分块尺寸的实验,综合分析结果如下：（1）分块超分导致拼接缝,在较小分块时,拼接缝呈现斑块效应,不一致性更为显著。（2）随着分块尺寸增大,模型的超分精度和计算效率均有所提高,当分块比大于1时,耗时与精度趋于稳定。（3）整图输入的计算可行性、精度和模型关系密切。ESPCN模型在整图输入时精度最优,RDBPN模型由于图像非方阵而导致精度下降,HSENET模型对算力要求较高,无法进行整图计算。综上,本文的研究为遥感超分工程化应用的分块大小选取提供了实验依据。

关键词: 深度学习超分辨率, 大场景遥感图像, 空间分辨率, 输入尺寸, 分块大小

Abstract: Single image Super-resolution (SISR) can improve the resolution of remote sensing images (RSIs), thereby improving the application value of data. At present, the number of pixels of RSIs generally reaches hundreds of millions, and it is usually necessary to divide the image into patches when performing SISR. However, there is a lack of relevant research on how to effectively determine the patch size and whether different sizes affect the results. In this paper, taking a large-scale high-resolution RSIs as the experiment data, 3 typical SISR models are selected, 9 groups of SR experiments under different patch sizes are carried out, and the super resolution (SR) results for the whole of the large-scaled RSI are analyzed comprehensively both qualitatively and quantitatively. The results show that: (1) Cutting of the patches results in stitching seams at the stitching place. In particular, when the patch size is small, a large number of stitching seams show a block effect and the inconsistency is more obvious. (2) With the increase of the patch size, the SR accuracy of the three models is improved, and the overall computational efficiency is also improved. When the test patches are larger than the training patches, the elapsed time and accuracy stabilize. (3) The feasibility and accuracy of the whole RSI input are closely related to the model. The ESPCN model has the best accuracy when inputting the whole RSI, the RDBPN model may cause the accuracy to decrease due to the non-square matrix of the RSI, and the HSENET model has high requirements for computing power and cannot calculate the whole RSI. In conclusion, this paper provides an experimental basis for the selection of patch size for RSI SR engineering applications.

Key words: deep learning super-resolution, large scene remote sensing images, spatial resolution, input size, patch size

中图分类号:

TP751

孙瑞奇, 张文娟, 李震, 马雪松, 梅君林. 分块尺寸对大场景遥感图像空间超分应用的影响分析^*[J]. 中国科学院大学学报, DOI: 10.7523/j.ucas.2024.019.

SUN Ruiqi, ZHANG Wenjuan, LI Zhen, MA Xuesong, MEI Junlin. Analysis of the effect of patch size on super-resolution of large scene remote sensing images[J]. Journal of University of Chinese Academy of Sciences, DOI: 10.7523/j.ucas.2024.019.

参考文献

[1] 张兵. 当代遥感科技发展的现状与未来展望[J].中国科学院院刊,2017,32(7):774-784.DOI:10.16418/j.issn.1000-3045.2017.07.012.
[2] 黄昕, 张良培, 李平湘. 高空间分辨率遥感图像分类的SSMC方法[J]. 中国图象图形学报, 2006, 11(4): 529-534, 插4. DOI: 10.3969/j.issn.1006-8961.2006.04.014.
[3] Martins V S, Kaleita A L, Gelder B K, et al.Exploring multiscale object-based convolutional neural network (multi-OCNN) for remote sensing image classification at high spatial resolution[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 168: 56-73. DOI: 10.1016/j.isprsjprs.2020.08.004.
[4] 张鑫龙, 陈秀万, 李飞, 等. 高分辨率遥感影像的深度学习变化检测方法[J]. 测绘学报, 2017, 46(8): 999-1008. DOI: 10.11947/j.AGCS.2017.20170036.
[5] Fytsilis A L, Prokos A, Koutroumbas K D, et al.A methodology for near real-time change detection between Unmanned Aerial Vehicle and wide area satellite images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 119: 165-186. DOI: 10.1016/j.isprsjprs.2016.06.001.
[6] 林祥国, 张继贤. 面向对象的形态学建筑物指数及其高分辨率遥感影像建筑物提取应用[J]. 测绘学报, 2017, 46(6): 724-733. DOI: 10.11947/j.AGCS.2017.20170068.
[7] Sun X, Wang P J, Yan Z Y, et al.FAIR1M: a benchmark dataset for fine-grained object recognition in high-resolution remote sensing imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2022, 184: 116-130. DOI: 10.1016/j.isprsjprs.2021.12.004.
[8] 郑凯旋, 林兴稳, 闻建光, 等. 集合卡尔曼滤波方法的高时空分辨率山区地表反照率反演[J]. 遥感学报, 2022, 26(12): 2568-2581. DOI: 10.11834/jrs.20210322.
[9] Huang L, Han X Y, Wang X L, et al.Coupling with high-resolution remote sensing data to evaluate urban non-point source pollution in Tongzhou, China[J]. The Science of the Total Environment, 2022, 831: 154632. DOI: 10.1016/j.scitotenv.2022.154632.
[10] 左超, 陈钱. 计算光学成像: 何来, 何处, 何去, 何从?[J]. 红外与激光工程, 2022, 51(2): 150-330. DOI: 10.3788/IRLA20220110.
[11] 左超, 陈钱. 分辨率, 超分辨率与空间带宽积拓展—从计算光学成像角度的一些思考[J]. 中国光学(中英文), 2022(6): 1105-1166. DOI: 10.37188/CO.2022-0105.
[12] 郑珂. 基于深度学习的高光谱图像超分辨率重建与分类方法研究[D]. 北京: 中国矿业大学 (北京), 2020. DOI: 10.27624/d.cnki.gzkbu.2020.000027.
[13] 李佳星, 赵勇先, 王京华. 基于深度学习的单幅图像超分辨率重建算法综述[J]. 自动化学报, 2021, 47(10): 2341-2363. DOI: 10.16383/j.aas.c190859.
[14] Keys R.Cubic convolution interpolation for digital image processing[J]. IEEE Transactions on Acoustics, Speech, and Signal Processing, 1981, 29(6): 1153-1160. DOI: 10.1109/TASSP.1981.1163711.
[15] Wang Q M, Shi W Z, Atkinson P M.Sub-pixel mapping of remote sensing images based on radial basis function interpolation[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2014, 92: 1-15. DOI: 10.1016/j.isprsjprs.2014.02.012.
[16] Ma J L, Chan J C W, Canters F. Robust locally weighted regression for superresolution enhancement of multi-angle remote sensing imagery[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014, 7(4): 1357-1371. DOI: 10.1109/JSTARS.2014.2312887.
[17] Wang P, Yao H Y, Li C, et al.Multiresolution analysis based on dual-scale regression for pansharpening[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 60: 5406319. DOI: 10.1109/tgrs.2021.3131477.
[18] Pan Z X, Yu J, Huang H J, et al.Super-resolution based on compressive sensing and structural self-similarity for remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2013, 51(9): 4864-4876. DOI: 10.1109/TGRS.2012.2230270.
[19] Hou B, Zhou K, Jiao L C.Adaptive super-resolution for remote sensing images based on sparse representation with global joint dictionary model[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(4): 2312-2327. DOI: 10.1109/TGRS.2017.2778191.
[20] Jiang K, Wang Z Y, Yi P, et al.Edge-enhanced GAN for remote sensing image superresolution[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(8): 5799-5812. DOI: 10.1109/TGRS.2019.2902431.
[21] Xu M Z, Ma J, Zhu Y Y.Dual-Diffusion: dual conditional denoising diffusion probabilistic models for blind super-resolution reconstruction in RSIs[J]. IEEE Geoscience and Remote Sensing Letters, 2023, 20: 6008505. DOI: 10.1109/LGRS.2023.3304418.
[22] Dong C, Loy C C, He K M, et al.Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2): 295-307. DOI: 10.1109/TPAMI.2015.2439281.
[23] Shi W Z, Caballero J, Huszár F, et al.Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA. IEEE, 2016: 1874-1883. DOI: 10.1109/CVPR.2016.207.
[24] Pan Z X, Ma W, Guo J Y, et al.Super-resolution of single remote sensing image based on residual dense backprojection networks[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(10): 7918-7933. DOI: 10.1109/TGRS.2019.2917427.
[25] Lei S, Shi Z W.Hybrid-scale self-similarity exploitation for remote sensing image super-resolution[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 60: 5401410. DOI: 10.1109/TGRS.2021.3069889.
[26] Ji S P, Wei S Q, Lu M.Fully convolutional networks for multisource building extraction from an open aerial and satellite imagery data set[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(1): 574-586. DOI: 10.1109/TGRS.2018.2858817.
[27] Odena A, Dumoulin V, Olah C.Deconvolution and checkerboard artifacts[J]. Distill, 2016, 1(10): e3. DOI: 10.23915/distill.00003.

分块尺寸对大场景遥感图像空间超分应用的影响分析^*

Analysis of the effect of patch size on super-resolution of large scene remote sensing images

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 1

编辑推荐

Metrics

本文评价

访问统计

联系我们