Arbitrary-scale super-resolution reconstruction of remote sensing images based on meta-learning and residual dense attention mechanism

  • role: First author第一作者
  • Affiliation:

    School of Astronautics, Beihang University, Beijing 102206, China

  • Email:19375140@buaa.edu.cn
  • Introduction:魏小源,研究方向为遥感图像超分辨率重建。E-mail: 19375140@buaa.edu.cn
WEI Xiaoyuan1,  
  • Affiliation:

    Beijing Institute of Remote Sensing Information, Beijing 100192, China

MENG Gang2,  
  • role: Corresponding author通信作者
  • Affiliation:

    School of Astronautics, Beihang University, Beijing 102206, China

  • Email:zhanghaopeng@buaa.edu.cn
  • Introduction:张浩鹏,研究方向为遥感图像处理与解译、空间目标信息处理。E-mail:zhanghaopeng@buaa.edu.cn
ZHANG Haopeng1*,  
  • Affiliation:

    School of Astronautics, Beihang University, Beijing 102206, China

JIANG Zhiguo1

résumé

Super-resolution reconstruction technology plays an important role in the intelligent processing of satellite remote sensing images. Existing deep learning methods for super-resolution remote sensing image reconstruction can only handle super-resolution tasks with a single scale factor, lacking generalization at the multiscale level and failing to meet the requirements of real super-resolution remote sensing image reconstruction for continuous zooming at multiple magnification levels.To address the problem of arbitrary-scale super-resolution reconstruction and enhance the quality of reconstructed real remote sensing images, this paper proposes a super-resolution reconstruction method called the Meta-RDCAN, which utilizes meta-learning and residual dense channel attention network.The proposed method employs a meta-upscale module that incorporates three functions: weight prediction, location projection, and feature mapping. The module adaptively adjusts the internal parameters of a model according to different scale factors for arbitrary-scale super-resolution reconstruction. From the perspective of extracting detailed information of local land objects in a remote sensing image, a dense residual network with an attention mechanism is used as a feature extractor, enabling the reconstructed results to possess clear and distinguishable details.Extensive experiments are conducted on various datasets, including DIV2K, AID, UCMerced, WIDS, Set5, and real remote sensing image from Macao Science Popularization Satellite. The influence of variations in spatial resolution on the super-resolution reconstruction results is analyzed, and the effectiveness of the training scheme, which involves pretraining on a general dataset followed by fine-tuning on a remote sensing dataset, is validated using the loss curve.The test results with different scale factors demonstrate that the proposed model is suitable for arbitrary-scale super-resolution reconstruction tasks in remote sensing images, having scale factors of up to 4.0. Comparative experimental results show that the improved model with added channel attention achieves enhanced performance in terms of Peak Signal-to-Noise Ratio (PSNR) and Structural SIMilarity (SSIM), compared with the baseline model. On real remote sensing data, the reconstruction results of the proposed model achieve a PSNR of over 40 dB and an SSIM of over 0.95. The comparison based on the no-reference quality indicator NIQE confirms that the perceived quality of the super-resolution reconstruction results of the improved model surpasses that of the baseline model.The proposed method for arbitrary-scale super-resolution reconstruction is effective for remote sensing images by utilizing meta-learning and dense residual channel attention. The main contributions of this paper include two aspects. First, for the arbitrary-scale super-resolution reconstruction of remote sensing image, the meta-learning approach is employed to adaptively adjust the internal parameters of a model. This approach enables continuous integer- and non-integer-scale super-resolution reconstruction of a single remote sensing image with a single model. Second, to address issues, such as missing details and unclear edges of geographic features, in reconstruction results, a channel-attention mechanism is applied to enhance the dense residual network and improve the quality of super-resolution reconstruction results.

mots-clés

super-resolution reconstruction;remote sensing image;arbitrary-scale;meta-learning;residual dense network;channel attention mechanism

References

  1. 1.
    Agustsson E and Timofte R. 2017. NTIRE 2017 challenge on single image super-resolution: dataset and study//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Honolulu: IEEE: 1122-1131
  2. 2.
    Bavirisetti D P and Dhuli R. 2016. Fusion of infrared and visible sensor images based on anisotropic diffusion and Karhunen-Loeve transform. IEEE Sensors Journal, 16(1): 203-209
  3. 3.
    Dosovitskiy A and Brox T. 2016. Inverting visual representations with convolutional networks//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE: 4829-4837
  4. 4.
    He Z and He D. 2020. Deep learning-based super-resolution for GF-4 satellite imagery. Journal of Remote Sensing (in Chinese), 24(12): 1500-1510
  5. 5.
    Hu X C, Mu H Y, Zhang X Y, Wang Z L, Tan T N and Sun J. 2019. Meta-SR: a magnification-arbitrary network for super-resolution//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach: IEEE: 1575-1584
  6. 6.
    Kulkarni K, Lohit S, Turaga P, Kerviche R and Ashok A. 2016. ReconNet: non-iterative reconstruction of images from compressively sensed measurements//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE: 449-458
  7. 7.
    Liu E, Logan J, Lee M, Roy S, Wang S, Valerie V M and Venkatesh V. 2019. WiDS Datathon 2019. Kaggle.
  8. 8.
    Mittal A, Soundararajan R and Bovik A C. 2013. Making a “completely blind” image quality analyzer. IEEE Signal Processing Letters, 20(3): 209-212
  9. 9.
    Pratt L and Thrun S. 1997. Guest editors’ introduction. Machine Learning, 28(1): 5
  10. 10.
    Ren S B, Meng Q and Wu Z. 2022. Flow-based super-resolution reconstruction of remote sensing images. Chinese Space Science and Technology, 42(6): 99-106
  11. 11.
    Thrun S and Pratt L. 1998. Learning to learn: introduction and overview//Learning to Learn. New York: Springer: 3-17
  12. 12.
    Vilalta R and Drissi Y. 2002. A perspective view and survey of meta-learning. Artificial Intelligence Review, 18(2): 77-95
  13. 13.
    Xia G S, Hu J W, Hu F, Shi B G, Bai X, Zhong Y F, Zhang L P and Lu X Q. 2017. AID: a benchmark data set for performance evaluation of aerial scene classification. IEEE Transactions on Geoscience and Remote Sensing, 55(7): 3965-3981
  14. 14.
    Yang X, Li F, Lu M, Xin L, Lu X T and Zhang N. 2022. New super-resolution reconstruction method based on mixed sparse representations. National Remote Sensing Bulletin, 26(8): 1685-1697
  15. 15.
    Yang Y and Newsam S. 2010. Bag-of-visual-words and spatial extensions for land-use classification//Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems. San Jose: Association for Computing Machinery: 270-279
  16. 16.
    Zhang Y L, Li K P, Li K, Wang L C, Zhong B N and Fu Y. 2018a. Image super-resolution using very deep residual channel attention networks//15th European Conference on Computer Vision. Munich: Springer: 294-310
  17. 17.
    Zhang Y L, Tian Y P, Kong Y, Zhong B N and Fu Y. 2018b. Residual dense network for image super-resolution//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 2472-2481
  18. 18.
    Zhao H Y and Li Y H. 2022. Super-resolution analysis of Sentinel-2 images using generative adversarial networks. Journal of Computer Applications, 42(S1): 298-304

Lire l'article complet

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website