A remote sensing image cloud removal network based on attention mechanism and generate adversarial network

Zheng Qiguang; Wang Renfang; Qiu Hong; Wang Feng; GAO Guang; Wu Dun

doi:10.11834/jrs.20243372

Views : 0 下载量: 24 CSCD: 0 更多指标

R-PDF
PDF
Export
Share
Collection
Album

A remote sensing image cloud removal network based on attention mechanism and generate adversarial network
Pages: 1-10(2024)
Published Online： 11 March 2024 ，
DOI： 10.11834/jrs.20243372

扫描看全文

郑其光，王仁芳，邱虹，王峰，高广，吴敦.XXXX.基于注意力机制和生成对抗网络的遥感影像云去除.遥感学报，XX（XX）： 1-10

Zheng Qiguang，Wang Renfang，Qiu Hong，Wang Feng，GAO Guang，Wu Dun. XXXX. A remote sensing image cloud removal network based on attention mechanism and generate adversarial network. National Remote Sensing Bulletin， XX（XX）：1-10
郑其光，王仁芳，邱虹，王峰，高广，吴敦.XXXX.基于注意力机制和生成对抗网络的遥感影像云去除.遥感学报，XX（XX）： 1-10 DOI： 10.11834/jrs.20243372.

Zheng Qiguang，Wang Renfang，Qiu Hong，Wang Feng，GAO Guang，Wu Dun. XXXX. A remote sensing image cloud removal network based on attention mechanism and generate adversarial network. National Remote Sensing Bulletin， XX（XX）：1-10 DOI： 10.11834/jrs.20243372.

摘要

受成像环境、传感设备等因素的限制，高分辨率卫星遥感影像存在云干扰的现象，其严重影响了影像质量及后续的智能解译。深度学习因其强大的特征学习能力，近年来在在遥感影像去云中得到了广泛应用。针对现有遥感影像去云方法往往难以更好地保留图像边缘纹理细节的问题，本文提出一种基于注意力机制的生成对抗网络去云算法。首先设计傅里叶残差模块同时集成云层影像的低频和高频残差信息；然后利用注意力机制从通道和空间两个角度进行特征关注而获得云层空间分布信息，并引导云层去除；最后在构造的联合损失函数约束下训练模型，确保生成图像和真实影像在内容上的一致性。在遥感影像数据集RICE上的实验结果表明，本文提出的基于注意力机制和生成对抗网络的遥感影像云去除方法能够有效地保留影像边缘纹理细节，且达到了优异的定量指标和视觉效果。

Abstract

High-resolution satellite remote sensing images are often compromised by cloud interference due to constraints such as imaging conditions and sensing equipment. This significantly degrades image quality and hinders intelligent interpretation. In recent years

deep learning has been extensively applied to cloud removal in remote sensing images

thanks to its powerful feature learning capabilities. This paper presents an attention-based generative adversarial network for cloud removal in remote sensing images

which aims to address the challenge of preserving textures. Initially

a Fourier residual block is designed to integrate both low-frequency and high-frequency of cloud images. Subsequently

attention mechanisms are employed from both channel and spatial to capture the spatial distribution information of clouds that guides the network in effectively removing clouds. Finally

the model is trained to minimize the joint loss function

ensuring the coherence of generated images with real images in terms of content. Experimental results derived from the RICE dataset substantiate that the proposed cloud removal method

which integrates attention mechanisms and generative adversarial networks

is capable of effectively eradicating clouds while preserving textures.The following conclusions can be drawn (1) The proposed Fourier residual block plays a pivotal role in adeptly capturing the details of both low-frequency and high-frequency residual information present in cloud images. By merging these diverse features

the block bolsters the network’s ability to extract rich features; (2) The attention mechanisms utilized extract features from both spatial and channel dimensions. This enables the model to discern the significance of various features across different dimensions. Moreover

the cloud information obtained through these attention mechanisms allows the model to adaptively present the changes in cloud

thereby enhancing the model’s precision and efficiency in cloud removal; (3) Experimental results on the RICE dataset provide compelling evidence of the proposed method’s effectiveness in terms of cloud removal and texture preservation.

关键词

遥感影像去云傅里叶残差通道-空间注意力生成对抗网络云层空间分布

Keywords

remote sensing imagery cloud removalFourier residual blockchannel-spatial attentiongenerative adversarial networksspatial distribution of clouds

references

Chen S L, Chen X H, Chen X, Chen J, Cao X, Shen M G, Yang W and Cui X H. 2018. A Novel Cloud Removal Method Based on IHOT and the Cloud Trajectories for Landsat Imagery. Remote Sensing, 10(7): 1040 [DOI: 10.3390/rs10071040http://dx.doi.org/10.3390/rs10071040]

Ding H D, Zi Y and Xie F Y. 2023. Uncertainty-Based Thin Cloud Removal Network via Conditional Variational Autoencoders. Asian Conference on Computer Vision. Berlin: Springer: 52-68 [DOI: 10.1007/978-3-031-26313-2_4http://dx.doi.org/10.1007/978-3-031-26313-2_4]

Dong H, Pan J S, Xiang L, Hu Z, Zhang X Y, Wang F and Yang M H. 2020. Multi-Scale Boosted Dehazing Network With Dense Feature Fusion. IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Seattle: IEEE: 2154-2164 [DOI: 10.1109/CVPR42600.2020.00223http://dx.doi.org/10.1109/CVPR42600.2020.00223]

Fan X N，Yan W，Shi P F and Zhang X W. 2022. Remote sensing image target detection based on a multi-scale deep feature fusion network. National Remote Sensing Bulletin，26（11）：2292-2303

范新南，严炜，史朋飞，张学武. 2022. 多尺度深度特征融合网络的遥感图像目标检测. 遥感学报，26（11）：2292-2303［DOI：10.11834/jrs.20210170http://dx.doi.org/10.11834/jrs.20210170］

Hadjimitsis D G and Clayton C R I. 2008. The use of an improved atmospheric correction algorithm for removing atmospheric effects from remotely sensed images using an atmosphere-surface simulation and meteorological data. Meteorological Applications, 15(3): 381-387 [DOI: 10.1002/met.80http://dx.doi.org/10.1002/met.80]

He K M, Zhang X Y, Ren S Q and Sun J. 2015. Deep Residual Learning for Image Recognition. IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Las Vegas: IEEE: 770-778 [DOI: 10.1109/CVPR.2016.90http://dx.doi.org/10.1109/CVPR.2016.90]

He Y F, Li C L and Li X. 2023. Remote Sensing Image Dehazing Using Heterogeneous Atmospheric Light Prior. IEEE Access, (11): 18805-18820. [DOI: 10.1109/ACCESS.2023.3247967http://dx.doi.org/10.1109/ACCESS.2023.3247967]

Hu J, Shen L and Sun G. 2018. Squeeze-and-Excitation Networks. IEEE Transactions on Pattern Analysis Machine Intelligence 42(8), 2011-2023 [DOI: 10.1109/CVPR.2018.00745http://dx.doi.org/10.1109/CVPR.2018.00745]

Hu X W, Fu C W, Zhu L, Qin J and Heng P A. 2020. Direction-aware Spatial Context Features for Shadow Detection and Removal. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(11): 2795-2808 [DOI: 10.1109/TPAMI.2019.2919616http://dx.doi.org/10.1109/TPAMI.2019.2919616]

Judith C, Schneider J V, Schmidt M, Ortega R, Gaviria J and Zizka G. 2013. Using high-resolution remote sensing data for habitat suitability models of Bromeliaceae in the city of Mérida, Venezuela. Landscape and Urban Planning, 120: 107-118 [DOI: 10.1016/j.landurbplan.2013.08.012http://dx.doi.org/10.1016/j.landurbplan.2013.08.012]

Li B Y, Peng X L, Wang Z Y, Xu J Z and Feng D. 2017. AOD-Net: All-in-One Dehazing Network. International Conference on Computer Vision (ICCV). Venice IEEE: 4780-4788 [DOI: 10.1109/ICCV.2017.511http://dx.doi.org/10.1109/ICCV.2017.511]

Li S T，Li C Y and Kang X D. 2021. Development status and future prospects of multi-source remote sensing image fusion. National Remote Sensing Bulletin，25（1）：148-166

李树涛，李聪妤，康旭东. 2021. 多源遥感图像融合发展现状与未来展望.遥感学报，25（1）：148-166 [DOI：10.11834/jrs.20210259http://dx.doi.org/10.11834/jrs.20210259]

Li Y S, Wei F Y, Zhang Y J, Chen W, Ma J Y. 2023. HS2P: Hierarchical spectral and structure-preserving fusion network for multimodal remote sensing image cloud and shadow removal. Information Fusion, 94: 215-228 [DOI: 10.1016/j.inffus.2023.02.002http://dx.doi.org/10.1016/j.inffus.2023.02.002]

Lin D Y, Xu G L, Wang X K, Wang Y, Xian S and Fu K. 2019. A Remote Sensing Image Dataset for Cloud Removal. arXiv:1901.00600

Liu X H, Ma Y R, Shi Z H and Chen J. 2019. GridDehazeNet: Attention-Based Multi-Scale Network for Image Dehazing. International Conference on Computer Vision (ICCV). Seoul: IEEE: 7313-7322 [DOI: 10.1109/ICCV.2019.00741http://dx.doi.org/10.1109/ICCV.2019.00741]

Liu X, Zhang H, Tang Y Y and Du J X. 2016. Scene‐adaptive single image dehazing via opening dark channel model. IET Image Processing, 10(11): 877-884 [DOI: 10.1049/iet-ipr.2016.0138http://dx.doi.org/10.1049/iet-ipr.2016.0138]

Mao X T, Liu Y M, Liu F Z, Li Q L, Shen W and Wang Y. 2022. Intriguing Findings of Frequency Selection for Image Deblurring. AAAI Conference on Artificial Intelligence, 37: 1905-1913 [DOI: 10.1609/aaai.v37i2.25281http://dx.doi.org/10.1609/aaai.v37i2.25281]

Ming Y S，Liu Q H，Bai H and Huang C. 2023. Classification and change detection of vegetation in the Ruoergai Wetland using optical and SAR remote sensing data. National Remote Sensing Bulletin，27（6）：1414-1425

明義森，刘启航，柏荷，黄昌. 2023. 利用光学和SAR遥感数据的若尔盖湿地植被分类与变化监测. 遥感学报，27（6）：1414-1425[DOI：10. 11834/jrs.20221767http://dx.doi.org/10.11834/jrs.20221767］

Mirza M and Osindero S. 2014. Conditional Generative Adversarial Nets. [DOI: 10.48550/arXiv.1411.1784http://dx.doi.org/10.48550/arXiv.1411.1784]

Pan H. 2020. Cloud Removal for Remote Sensing Imagery vai Spatial Attention Generative Adversarial Network. [DOI: arXiv:2009.13015http://dx.doi.org/arXiv:2009.13015]

Ren W Q, Ma L, Zhang J W, Pan J S, Cao X C, Liu W and Yang M H. 2018. Gated Fusion Network for Single Image Dehazing. IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Seattle: IEEE: 3253-3261 [DOI: 10.1109/CVPR.2018.00343http://dx.doi.org/10.1109/CVPR.2018.00343]

Shen H F, Li H F, Qian Y, Zhang L P and Yuan Q Q. 2014. An effective thin cloud removal procedure for visible remote sensing images. ISPRS Journal of Photogrammetry and Remote Sensing, 96: 224-235 [DOI: 10.1016/j.isprsjprs.2014.06.011http://dx.doi.org/10.1016/j.isprsjprs.2014.06.011]

Singh P and Komodakis N. 2018. Cloud-Gan: Cloud Removal for Sentinel-2 Imagery Using a Cyclic Consistent Generative Adversarial Networks. International Geoscience and Remote Sensing Symposium. Valencia: IEEE: 1772-1775 [DOI: 10.1109/IGARSS.2018.8519033http://dx.doi.org/10.1109/IGARSS.2018.8519033]

Wan Y Q, Fan Y H and Jin M S. 2021. Application of hyperspectral remote sensing for supplementary investigation of polymetallic deposits in Huaniushan ore region, northwestern China. Scientific Reports, 11(1): 440 [DOI: 10.1038/s41598-020-79864-0http://dx.doi.org/10.1038/s41598-020-79864-0]

Wang X X, Jiang L S, Chen Y P and Wang J L. 2013. Thin cloud removal from remote sensing images with adaptive thresholds of wavelet transforms. Journal of University of Electronic Science and Technology of China, 42(03): 71-74 [DOI: 10.3969/j.issn.1001-0548.2013.03.014http://dx.doi.org/10.3969/j.issn.1001-0548.2013.03.014]

Wang Q L, Wu B G, Zhu P F, Li P H, Zuo W M and Hu Q H. 2020. ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle: IEEE: 11531-11539 [DOI: 10.1109/CVPR42600.2020.01155http://dx.doi.org/10.1109/CVPR42600.2020.01155]

Wei W, Polap D, Li X H, Wozniak M and Liu J Z. 2018. Study on Remote Sensing Image Vegetation Classification Method Based on Decision Tree Classifier. Symposium Series on Computational Intelligence (SSCI). Piscataway: IEEE: 2292–2297 [DOI: 10.1109/SSCI.2018.8628721http://dx.doi.org/10.1109/SSCI.2018.8628721]

Wu X, Song W, Zhang X K, Lin G H, Wang H M and Deng Y Y. 2022. Algorithm Development of Cloud Removal from Solar Images Based on Pix2Pix Network. Computers, Materials & Continua, 71(2): 3497-3512 [DOI: 10.32604/cmc.2022.022325http://dx.doi.org/10.32604/cmc.2022.022325]

Yang X，Li F，Lu M，Xin L，Lu X T and Zhang N. 2022. New super-resolution reconstruction method based on Mixed Sparse Representations. National Remote Sensing Bulletin，26（8）：1685-1697

杨雪，李峰，鹿明，辛蕾，鲁啸天，张南. 2022. 混合稀疏表示模型的超分辨率重建. 遥感学报，26（8）：1685-1697［DOI：10.11834/jrs.20219409http://dx.doi.org/10.11834/jrs.20219409］

Zhou Y, Jing W P, Wang J, Chen G S, Scherer R and Damaševičius R. 2022. MSAR-DefogNet: Lightweight cloud removal network for high resolution remote sensing images based on multi scale convolution. IET Image Processing, 16(3): 659-668 [DOI: 10.1049/ipr2.12224http://dx.doi.org/10.1049/ipr2.12224]

Zi Y, Ding H D, Xie F Y, Jiang Z G and Song X D. 2023. Wavelet Integrated Convolutional Neural Network for Thin Cloud Removal in Remote Sensing Images. Remote Sensing, 15(3): 781 [DOI: 10.3390/rs15030781http://dx.doi.org/10.3390/rs15030781]

Alert me when the article has been cited

提交

Unsupervised domain adaptive classification for hyperspectral remote sensing by adversary coupled with distillation