Arbitrary-scale super-resolution reconstruction of remote sensing images based on meta-learning and residual dense attention mechanism

WEI Xiaoyuan; MENG Gang; ZHANG Haopeng; JIANG Zhiguo

doi:10.11834/jrs.20233267

Satellite Information Intelligent Processing Technology | Views : 0 下载量: 1895 CSCD: 0

R-PDF
PDF
Export
Share
Collection
Album

Arbitrary-scale super-resolution reconstruction of remote sensing images based on meta-learning and residual dense attention mechanism
Vol. 28, Issue 7, Pages: 1735-1745(2024)
Received：11 July 2023，

Published：07 July 2024
DOI： 10.11834/jrs.20233267
稿件说明：

移动端阅览

魏小源，孟钢，张浩鹏，姜志国.2024.基于元学习和密集残差注意力的遥感图像任意尺度超分辨率重建.遥感学报，28（7）： 1735-1745 DOI： 10.11834/jrs.20233267.

Wei X Y，Meng G，Zhang H P and Jiang Z G. 2024. Arbitrary-scale super-resolution reconstruction of remote sensing images based on meta-learning and residual dense attention mechanism. National Remote Sensing Bulletin， 28（7）：1735-1745 DOI： 10.11834/jrs.20233267.

摘要

超分辨率重建技术在卫星遥感图像信息智能处理领域中有重要的应用。现有面向遥感图像超分辨率重建的深度学习方法大多只能处理一种比例因子的超分辨率重建任务，在多尺度层面上缺少泛化性，难以满足真实遥感图像多倍率连续放大的超分辨率重建任务需求。为解决遥感图像超分辨率重建过程中的多尺度放大问题，本文采用元学习的方法，在构建单一自适应模型的基础上实现对遥感图像的任意尺度超分辨率重建，提升遥感图像的空间分辨率，利用密集残差网络和通道注意力机制重建遥感图像中地物纹理、目标边缘等丰富细节信息。在真实遥感图像上的定量实验表明，本文所提方法重建结果的峰值信噪比能达到40 dB以上，同时在多种数据上的定量和定性实验结果证明了本文方法的有效性。

Abstract

Super-resolution reconstruction technology plays an important role in the intelligent processing of satellite remote sensing images. Existing deep learning methods for super-resolution remote sensing image reconstruction can only handle super-resolution tasks with a single scale factor

lacking generalization at the multiscale level and failing to meet the requirements of real super-resolution remote sensing image reconstruction for continuous zooming at multiple magnification levels.

To address the problem of arbitrary-scale super-resolution reconstruction and enhance the quality of reconstructed real remote sensing images

this paper proposes a super-resolution reconstruction method called the Meta-RDCAN

which utilizes meta-learning and residual dense channel attention network.The proposed method employs a meta-upscale module that incorporates three functions: weight prediction

location projection

and feature mapping. The module adaptively adjusts the internal parameters of a model according to different scale factors for arbitrary-scale super-resolution reconstruction. From the perspective of extracting detailed information of local land objects in a remote sensing image

a dense residual network with an attention mechanism is used as a feature extractor

enabling the reconstructed results to possess clear and distinguishable details.Extensive experiments are conducted on various datasets

including DIV2K

AID

UCMerced

WIDS

Set5

and real remote sensing image from Macao Science Popularization Satellite. The influence of variations in spatial resolution on the super-resolution reconstruction results is analyzed

and the effectiveness of the training scheme

which involves pretraining on a general dataset followed by fine-tuning on a remote sensing dataset

is validated using the loss curve.

The test results with different scale factors demonstrate that the proposed model is suitable for arbitrary-scale super-resolution reconstruction tasks in remote sensing images

having scale factors of up to 4.0. Comparative experimental results show that the improved model with added channel attention achieves enhanced performance in terms of Peak Signal-to-Noise Ratio (PSNR) and Structural SIMilarity (SSIM)

compared with the baseline model. On real remote sensing data

the reconstruction results of the proposed model achieve a PSNR of over 40 dB and an SSIM of over 0.95. The comparison based on the no-reference quality indicator NIQE confirms that the perceived quality of the super-resolution reconstruction results of the improved model surpasses that of the baseline model.

The proposed method for arbitrary-scale super-resolution reconstruction is effective for remote sensing images by utilizing meta-learning and dense residual channel attention. The main contributions of this paper include two aspects. First

for the arbitrary-scale super-resolution reconstruction of remote sensing image

the meta-learning approach is employed to adaptively adjust the internal parameters of a model. This approach enables continuous integer- and non-integer-scale super-resolution reconstruction of a single remote sensing image with a single model. Second

to address issues

such as missing details and unclear edges of geographic features

in reconstruction results

a channel-attention mechanism is applied to enhance the dense residual network and improve the quality of super-resolution reconstruction results.

关键词

Keywords

references

Agustsson E and Timofte R . 2017 . NTIRE 2017 challenge on single image super-resolution: dataset and study // 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) . Honolulu : IEEE: 1122 - 1131 [ DOI: 10.1109/cvprw.2017.150 http://dx.doi.org/10.1109/cvprw.2017.150 ]

Bavirisetti D P and Dhuli R . 2016 . Fusion of infrared and visible sensor images based on anisotropic diffusion and Karhunen-Loeve transform . IEEE Sensors Journal , 16 ( 1 ): 203 - 209 [ DOI: 10.1109/jsen.2015.2478655 http://dx.doi.org/10.1109/jsen.2015.2478655 ]

Dosovitskiy A and Brox T . 2016 . Inverting visual representations with convolutional networks // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Las Vegas : IEEE: 4829 - 4837 [ DOI: 10.1109/cvpr.2016.522 http://dx.doi.org/10.1109/cvpr.2016.522 ]

He Z and He D . 2020 . Deep learning-based super-resolution for GF-4 satellite imagery . Journal of Remote Sensing (in Chinese) , 24 ( 12 ): 1500 - 1510

贺智 , 贺丹 . 2020 . 基于深度学习的高分四号卫星图像超分辨率重建 . 遥感学报 , 24 ( 12 ): 1500 - 1510 [ DOI: 10.11834/jrs.20208397 http://dx.doi.org/10.11834/jrs.20208397 ]

Hu X C , Mu H Y , Zhang X Y , Wang Z L , Tan T N and Sun J . 2019 . Meta-SR: a magnification-arbitrary network for super-resolution // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Long Beach : IEEE: 1575 - 1584 [ DOI: 10.1109/CVPR.2019.00167 http://dx.doi.org/10.1109/CVPR.2019.00167 ]

Kulkarni K , Lohit S , Turaga P , Kerviche R and Ashok A . 2016 . ReconNet: non-iterative reconstruction of images from compressively sensed measurements // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Las Vegas : IEEE: 449 - 458 [ DOI: 10.1109/cvpr.2016.55 http://dx.doi.org/10.1109/cvpr.2016.55 ]

Liu E , Logan J , Lee M , Roy S , Wang S , Valerie V M and Venkatesh V . 2019 . WiDS Datathon 2019 . Kaggle . https://kaggle.com/competitions/widsdatathon2019 https://kaggle.com/competitions/widsdatathon2019

Mittal A , Soundararajan R and Bovik A C . 2013 . Making a “completely blind” image quality analyzer . IEEE Signal Processing Letters , 20 ( 3 ): 209 - 212 [ DOI: 10.1109/lsp.2012.2227726 http://dx.doi.org/10.1109/lsp.2012.2227726 ]

Pratt L and Thrun S . 1997 . Guest editors’ introduction . Machine Learning , 28 ( 1 ): 5 [ DOI: 10.1023/A:1007322005825 http://dx.doi.org/10.1023/A:1007322005825 ]

Ren S B , Meng Q and Wu Z . 2022 . Flow-based super-resolution reconstruction of remote sensing images . Chinese Space Science and Technology , 42 ( 6 ): 99 - 106

任术波 , 孟倩 , 吴钻 . 2022 . 基于流的遥感图像超分辨率重建 . 中国空间科学技术 , 42 ( 6 ): 99 - 106 [ DOI: 10.16708/j.cnki.1000-758X.2022.0088 http://dx.doi.org/10.16708/j.cnki.1000-758X.2022.0088 ]

Thrun S and Pratt L . 1998 . Learning to learn: introduction and overview // Learning to Learn . New York : Springer: 3 - 17 [ DOI: 10.1007/978-1-4615-5529-2_1 http://dx.doi.org/10.1007/978-1-4615-5529-2_1 ]

Vilalta R and Drissi Y . 2002 . A perspective view and survey of meta-learning . Artificial Intelligence Review , 18 ( 2 ): 77 - 95 [ DOI: 10.1023/A:1019956318069 http://dx.doi.org/10.1023/A:1019956318069 ]

Xia G S , Hu J W , Hu F , Shi B G , Bai X , Zhong Y F , Zhang L P and Lu X Q . 2017 . AID: a benchmark data set for performance evaluation of aerial scene classification . IEEE Transactions on Geoscience and Remote Sensing , 55 ( 7 ): 3965 - 3981 [ DOI: 10.1109/tgrs.2017.2685945 http://dx.doi.org/10.1109/tgrs.2017.2685945 ]

Yang X , Li F , Lu M , Xin L , Lu X T and Zhang N . 2022 . New super-resolution reconstruction method based on mixed sparse representations . National Remote Sensing Bulletin , 26 ( 8 ): 1685 - 1697

杨雪 , 李峰 , 鹿明 , 辛蕾 , 鲁啸天 , 张南 . 2022 . 混合稀疏表示模型的超分辨率重建 . 遥感学报 , 26 ( 8 ): 1685 - 1697 [ DOI: 10.11834/jrs.20219409 http://dx.doi.org/10.11834/jrs.20219409 ]

Yang Y and Newsam S . 2010 . Bag-of-visual-words and spatial extensions for land-use classification // Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems . San Jose : Association for Computing Machinery: 270 - 279 [ DOI: 10.1145/1869790.1869829 http://dx.doi.org/10.1145/1869790.1869829 ]

Zhang Y L , Li K P , Li K , Wang L C , Zhong B N and Fu Y . 2018a . Image super-resolution using very deep residual channel attention networks // 15th European Conference on Computer Vision . Munich : Springer: 294 - 310 [ DOI: 10.1007/978-3-030-01234-2_18 http://dx.doi.org/10.1007/978-3-030-01234-2_18 ]

Zhang Y L , Tian Y P , Kong Y , Zhong B N and Fu Y . 2018b . Residual dense network for image super-resolution // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE: 2472 - 2481 [ DOI: 10.1109/CVPR.2018.00262 http://dx.doi.org/10.1109/CVPR.2018.00262 ]

Zhao H Y and Li Y H . 2022 . Super-resolution analysis of Sentinel-2 images using generative adversarial networks . Journal of Computer Applications , 42 ( S1 ): 298 - 304

赵慧岩 , 李云鹤 . 2022 . 基于生成对抗网络的Sentinel-2遥感图像超分辨率分析 . 计算机应用 , 42 ( S1 ): 298 - 304 [ DOI: 10.11772/j.issn.1001-9081.2021060953 http://dx.doi.org/10.11772/j.issn.1001-9081.2021060953 ]

Alert me when the article has been cited

提交

Low-Cost Fine-Grained interpretation method of remote sensing images based on weakly supervised instance segmentation

Dual-branch remote sensing shadow detection with the fusion of Transformer and CNN

ADC-CPANet：A remote sensing image classification method based on local-global feature fusion

Remote sensing image detection method based on brain-inspired spiking neural networks