层级特征交互与增强感受野双分支遥感图像去雾网络

孙航; 方帅领; 但志平; 任东; 余梅; 孙水发

doi:10.11834/jrs.20232333

AI+遥感 | 浏览量 : 0 下载量: 2093 CSCD: 1

R-PDF
PDF
导出
分享
收藏
专辑

层级特征交互与增强感受野双分支遥感图像去雾网络
A two-branch remote sensing image dehazing network based on hierarchical feature interaction and enhanced receptive field
2023年27卷第12期页码：2831-2846
收稿：2022-07-06，

纸质出版：2023-12-07
DOI： 10.11834/jrs.20232333
稿件说明：

移动端阅览

孙航，方帅领，但志平，任东，余梅，孙水发.2023.层级特征交互与增强感受野双分支遥感图像去雾网络.遥感学报，27（12）： 2831-2846 DOI： 10.11834/jrs.20232333.

Sun H，Fang S L，Dan Z P，Ren D，Yu M and Sun S F. 2023. A two-branch remote sensing image dehazing network based on hierarchical feature interaction and enhanced receptive field. National Remote Sensing Bulletin， 27（12）：2831-2846 DOI： 10.11834/jrs.20232333.

摘要

近年来，深度学习的去雾方法在图像去雾领域取得了显著的成绩。然而，大多数基于U型网络的去雾方法将编码层特征直接传递到对应解码层，缺乏浅层和深层特征之间的信息交互。此外，基于非U型网络的去雾方法存在感受野受限问题，无法有效的利用上下文信息。从而导致这些方法在场景尺度变化较大的遥感图像去雾中无法取得理想效果。为此，本文提出了一种层级特征交互与增强感受野的双分支遥感图像去雾网络，该方法包含层级特征交互子网和多尺度信息提取子网。其中，层级特征交互子网利用层级特征交互融合模块，逐层的在浅层特征中引入语义信息，深层特征中引入空间细节信息，从而增强编码层中不同层级特征之间的信息交互。多尺度信息提取子网利用多尺度残差空洞卷积模块，融合不同感受野的特征，从而获取对于遥感图像去雾至关重要的上下文信息。在两个公开数据集上的实验结果表明，本文提出的去雾方法相比现有的9种优秀的去雾算法，取得了最好的客观评价指标和视觉效果。

Abstract

In recent years

deep learning-based dehazing methods have achieved remarkable results in the field of image dehazing. However

most dehazing methods based on U-shaped networks directly transfer the features of the encoding layer to the corresponding decoding layers

which lacks information interaction between the low- and high-level features. Meanwhile

the network model designed based on the U-shaped structure may destroy the detailed information important for the restored image in the process of downsampling. As a result

the restored clear image lacks detailed texture and structure information. In addition

the dehazing method based on non-U-shaped network has limited receptive field

which hinders its capability to effectively utilize contextual information. As a result

these methods cannot achieve ideal dehazing results in remote sensing images with large scene scale changes. Therefore

this study proposes a two-branch remote sensing image dehazing network based on hierarchical feature interaction and enhanced receptive field. This network includes hierarchical feature interaction sub-net and multi-scale information extraction sub-net. The hierarchical feature interaction sub-net uses the hierarchical feature interaction fusion module to introduce semantic information into low-level features and spatial details into high-level features layer by layer. This way enhances the information interaction between features at different levels in the encoding layer. The multi-scale information extraction sub-net uses the multi-scale residual dilated convolution module to fuse the features of different receptive fields for obtaining contextual information

which is crucial for remote sensing image dehazing. The experiment on two public datasets show that the dehazing method proposed in this study achieves the best evaluation compared with the existing nine excellent dehazing algorithms. Among them

in the three sub-test sets of the public remote sensing dataset Haze1k

the quantitative index PSNR values of this study reach 27.362

28.171

and 25.137 dB. In the two sub-test sets of the public remote sensing dataset RICE

the quantitative index PSNR values of this study reach 37.79 and 35.367 dB. In addition

the method proposed in this study is the closest to ground truth in terms of subjective visual qualities such as color

saturation

and sharpness

while still achieving the dehazing effect. The following conclusions can be drawn: (1) through the proposed hierarchical feature interaction fusion module

the deep semantic information in the coding stage is gradually interactively fused with the shallow detailed texture information

which enhances the expressive ability of the network and restores clear images with higher quality. (2) Through the multi-scale residual dilated convolution module

the dehazing network proposed in this study can increase the receptive field of the network without changing the size of the feature map. The contextual information of different scales can also be fused. (3) In two public remote sensing image dehazing datasets

namely

Haze1k and RICE

the dehazing method proposed in this study outperforms nine recently proposed excellent dehazing algorithms in terms of objective evaluation indexes and subjective visual effects.

关键词

Keywords

references

Cai B L , Xu X M , Jia K , Qing C and Tao D C . 2016 . DehazeNet: an end-to-end system for single image haze removal . IEEE Transactions on Image Processing , 25 ( 11 ): 5187 - 5198 [ DOI: 10.1109/TIP.2016.2598681 http://dx.doi.org/10.1109/TIP.2016.2598681 ]

Cox L J . 1977 . Optics of the atmosphere-scattering by molecules and particles . Optica Acta : International Journal of Optics , 24 ( 7 ): 779 - 779 [ DOI: 10.1080/713819629 http://dx.doi.org/10.1080/713819629 ]

Deng J , Dong W , Socher R , Li L J , Li K and Fei-Fei L . 2009 . ImageNet: a large-scale hierarchical image database // 2009 IEEE Conference on Computer Vision and Pattern Recognition . Miami : IEEE: 248 - 255 [ DOI: 10.1109/CVPR.2009.5206848 http://dx.doi.org/10.1109/CVPR.2009.5206848 ]

Dong Y , Liu Y H , Zhang H , Chen S F and Qiao Y . 2020 . FD-GAN: generative adversarial networks with fusion-discriminator for single image dehazing . Proceedings of the AAAI Conference on Artificial Intelligence , 34 ( 7 ): 10729 - 10736 [ DOI: 10.1609/aaai.v34i07.6701 http://dx.doi.org/10.1609/aaai.v34i07.6701 ]

Engin D , Gen A and Ekenel H K . 2018 . Cycle-Dehaze: enhanced CycleGAN for single image dehazing // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops . Salt Lake City : IEEE: 825 - 833 [ DOI: 10.1109/CVPRW.2018.00127 http://dx.doi.org/10.1109/CVPRW.2018.00127 ]

Gao S H , Cheng M M , Zhao K , Zhang X Y , Yang M H and Torr P . 2021 . Res2Net: a new multi-scale backbone architecture . IEEE Transactions on Pattern Analysis and Machine Intelligence , 43 ( 2 ): 652 - 662 [ DOI: 10.1109/TPAMI.2019.2938758 http://dx.doi.org/10.1109/TPAMI.2019.2938758 ]

He K M , Sun J and Tang X O . 2011 . Single image haze removal using dark channel prior . IEEE Transactions on Pattern Analysis and Machine Intelligence , 33 ( 12 ): 2341 - 2353 [ DOI: 10.1109/TPAMI.2010.168 http://dx.doi.org/10.1109/TPAMI.2010.168 ]

He K M , Zhang X Y , Ren S Q and Sun J . 2016 . Deep residual learning for image recognition // 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas : IEEE: 770 - 778 [ DOI: 10.1109/CVPR.2016.90 http://dx.doi.org/10.1109/CVPR.2016.90 ]

Hu J , Shen L , Albanie S , Sun G and Wu E H . 2020 . Squeeze-and-excitation networks . IEEE Transactions on Pattern Analysis and Machine Intelligence , 42 ( 8 ): 2011 - 2023 [ DOI: 10.1109/TPAMI.2019.2913372 http://dx.doi.org/10.1109/TPAMI.2019.2913372 ]

Huang B H , Li Z , Yang C , Sun F C and Song Y X . 2020 . Single satellite optical imagery dehazing using SAR image prior based on conditional generative adversarial networks // 2020 IEEE Winter Conference on Applications of Computer Vision . Snowmass : IEEE: 1795 - 1802 [ DOI: 10.1109/WACV45572.2020.9093471 http://dx.doi.org/10.1109/WACV45572.2020.9093471 ]

Huang Y F and Chen X . 2021 . Single remote sensing image dehazing using a dual-step cascaded residual dense network // 2021 IEEE International Conference on Image Processing . Anchorage : IEEE: 3852 - 3856 [ DOI: 10.1109/ICIP42928.2021.9506603 http://dx.doi.org/10.1109/ICIP42928.2021.9506603 ]

Kim U H , Kim S H and Kim J H . 2022 . SimVODIS: simultaneous visual odometry, object detection, and instance segmentation . IEEE Transactions on Pattern Analysis and Machine Intelligence , 44 ( 1 ): 428 - 441 [ DOI: 10.1109/TPAMI.2020.3007546 http://dx.doi.org/10.1109/TPAMI.2020.3007546 ]

Ledig C , Theis L , Huszar F , Caballero J , Cunningham A , Acosta A , Aitken A , Tejani A , Totz J , Wang Z H and Shi W Z . 2017 . Photo-realistic single image super-resolution using a generative adversarial network // 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu : IEEE: 105 - 114 [ DOI: 10.1109/CVPR.2017.19 http://dx.doi.org/10.1109/CVPR.2017.19 ]

Li B , Luo H , Zhang H X , Tan S Q and Ji Z Z . 2017 . A multi-branch convolutional neural network for detecting double JPEG compression . arXiv : 1710 . 05477 [ DOI: 10.48550/arXiv.1710.05477 http://dx.doi.org/10.48550/arXiv.1710.05477 ]

Li Y F and Chen X . 2021 . A coarse-to-fine two-stage attentive network for haze removal of remote sensing images . IEEE Geoscience and Remote Sensing Letters , 18 ( 10 ): 1751 - 1755 [ DOI: 10.1109/LGRS.2020.3006533 http://dx.doi.org/10.1109/LGRS.2020.3006533 ]

Li Z L , Wang L Y , Jiang S , Wu Y H and Zhang Q J . 2021 . On orbit extraction method of ship target in SAR images based on ultra-lightweight network . National Remote Sensing Bulletin , 25 ( 3 ): 765 - 775

李宗凌 , 汪路元 , 蒋帅 , 吴雨航 , 张庆君 . 2021 . 超轻量网络的SAR图像舰船目标在轨提取 . 遥感学报 , 25 ( 3 ): 765 - 775 [ DOI: 10.11834/jrs.20210160 http://dx.doi.org/10.11834/jrs.20210160 ]

Lin D Y , Xu G L , Wang X K , Wang Y , Sun X and Fu K . 2019 . A remote sensing image dataset for cloud removal . arXiv : 1901 . 00600 [ DOI: 10.48550/arXiv.1901.00600 http://dx.doi.org/10.48550/arXiv.1901.00600 ]

Mehta A , Sinha H , Mandal M and Narang P . 2021 . Domain-aware unsupervised hyperspectral reconstruction for aerial image dehazing // 2021 IEEE Winter Conference on Applications of Computer Vision . Waikoloa : IEEE: 413 - 422 [ DOI: 10.1109/WACV48630.2021.00046 http://dx.doi.org/10.1109/WACV48630.2021.00046 ]

Mou F . 2021 . Research on Thin Cloud Removal and Application of Remote Sensing Image Based on Deep Learning . Chengdu : University of Electronic Science and Technology of China (牟范 . 2021. 基于深度学习的遥感影像去薄云及应用研究 . 成都: 电子科技大学) [DOI: 10.27005/d.cnki.gdzku.2021.001113]

Qin X , Wang Z L , Bai Y C , Xie X D and Jia H Z . 2020 . FFA-Net: feature fusion attention network for single image dehazing . Proceedings of the AAAI Conference on Artificial Intelligence , 34 ( 7 ): 11908 - 11915 [ DOI: 10.1609/aaai.v34i07.6865 http://dx.doi.org/10.1609/aaai.v34i07.6865 ]

Qu Y Y , Chen Y Z , Huang J Y and Xie Y . 2019 . Enhanced Pix2pix dehazing network // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach : IEEE: 8152 - 8160 [ DOI: 10.1109/CVPR.2019.00835 http://dx.doi.org/10.1109/CVPR.2019.00835 ]

Ren W Q , Liu S , Zhang H , Pan J S , Cao X C and Yang M H . 2016 . Single image dehazing via multi-scale convolutional neural networks // 14th European Conference on Computer Vision . Amsterdam : Springer: 154 - 169 [ DOI: 10.1007/978-3-319-46475-6_10 http://dx.doi.org/10.1007/978-3-319-46475-6_10 ]

Ren W Q , Ma L , Zhang J W , Pan J S , Cao X C , Liu W and Yang M H . 2018 . Gated fusion network for single image dehazing // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE: 3253 - 3261 [ DOI: 10.1109/CVPR.2018.00343 http://dx.doi.org/10.1109/CVPR.2018.00343 ]

Shi W Z , Caballero J , Huszár F , Totz J , Aitken A P , Bishop R , Rueckert D and Wang Z H . 2016 . Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network // 2016 IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas : IEEE: 1874 - 1883 [ DOI: 10.1109/CVPR.2016.207 http://dx.doi.org/10.1109/CVPR.2016.207 ]

Simonyan K and Zisserman A . 2015 . Very deep convolutional networks for large-scale image recognition . arXiv : 1409 . 1556 [ DOI: 10.48550/arXiv.1409.1556 http://dx.doi.org/10.48550/arXiv.1409.1556 ]

Song Y F , Li J , Wang X G and Chen X W . 2018 . Single image dehazing using ranking convolutional neural network . IEEE Transactions on Multimedia , 20 ( 6 ): 1548 - 1560 [ DOI: 10.1109/TMM.2017.2771472 http://dx.doi.org/10.1109/TMM.2017.2771472 ]

Sun X and Xu J D . 2021 . Remote sensing image dehazing method based on cascaded generative adversarial network . Journal of Computer Applications , 41 ( 8 ): 2440 - 2444

孙潇 , 徐金东 . 2021 . 基于级联生成对抗网络的遥感图像去雾方法 . 计算机应用 , 41 ( 8 ): 2440 - 2444 [ DOI: 10.11772/j.issn.1001-9081.2020101563 http://dx.doi.org/10.11772/j.issn.1001-9081.2020101563 ]

Tan R T . 2008 . Visibility in bad weather from a single image // 2008 IEEE Conference on Computer Vision and Pattern Recognition . Anchorage : IEEE: 1 - 8 [ DOI: 10.1109/CVPR.2008.4587643 http://dx.doi.org/10.1109/CVPR.2008.4587643 ]

Tang K T , Yang J C and Wang J . 2014 . Investigating haze-relevant features in a learning framework for image dehazing // 2014 IEEE Conference on Computer Vision and Pattern Recognition . Columbus : IEEE: 2995 - 3002 [ DOI: 10.1109/CVPR.2014.383 http://dx.doi.org/10.1109/CVPR.2014.383 ]

Wang L W , Li Y , Huang J and Lazebnik S . 2019 . Learning two-branch neural networks for image-text matching tasks . IEEE Transactions on Pattern Analysis and Machine Intelligence , 41 ( 2 ): 394 - 407 [ DOI: 10.1109/TPAMI.2018.2797921 http://dx.doi.org/10.1109/TPAMI.2018.2797921 ]

Wei X S , Xu Y Y and Yang J . 2022 . Review of webly-supervised fine-grained image recognition . Journal of Image and Graphics , 27 ( 7 ): 2057 - 2077

魏秀参 , 许玉燕 , 杨健 . 2022 . 网络监督数据下的细粒度图像识别综述 . 中国图象图形学报 , 27 ( 7 ): 2057 - 2077 [ DOI: 10.11834/jig.210188 http://dx.doi.org/10.11834/jig.210188 ]

Wu H Y , Liu J , Xie Y , Qu Y Y and Ma L Z . 2020 . Knowledge transfer dehazing network for NonHomogeneous dehazing // 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops . Seattle : IEEE: 1975 - 1983 [ DOI: 10.1109/CVPRW50498.2020.00247 http://dx.doi.org/10.1109/CVPRW50498.2020.00247 ]

Wu H Y , Qu Y Y , Lin S H , Zhou J , Qiao R Z , Zhang Z Z , Xie Y and Ma L Z . 2021 . Contrastive learning for compact single image dehazing // 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Nashville : IEEE: 10546 - 10555 [ https://doi.org/10.1109/CVPR46437.2021.01041 https://doi.org/10.1109/CVPR46437.2021.01041 ]

Yang S T , Wang P F , Wang J , Lou H Z and Gong T L . 2021 . River flow estimation method based on UAV aerial photogrammetry . National Remote Sensing Bulletin , 25 ( 6 ): 1284 - 1293

杨胜天 , 王鹏飞 , 王娟 , 娄和震 , 巩同梁 . 2021 . 结合无人机航空摄影测量的河道流量估算 . 遥感学报 , 25 ( 6 ): 1284 - 1293 [ DOI: 10.11834/jrs.20209082 http://dx.doi.org/10.11834/jrs.20209082 ]

Yu F and Koltun V . 2016 . Multi-scale context aggregation by dilated convolutions . arXiv : 1511 . 07122 [ DOI: 10.48550/arXiv.1511.07122 http://dx.doi.org/10.48550/arXiv.1511.07122 ]

Yu Y K , Liu H , Fu M H , Chen J , Wang X Y and Wang K Y . 2021 . A two-branch neural network for non-homogeneous dehazing via ensemble learning // 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops . Nashville : IEEE: 193 - 202 [ DOI: 10.1109/CVPRW53098.2021.00028 http://dx.doi.org/10.1109/CVPRW53098.2021.00028 ]

Yuan F N , Li Z Q , Shi J T , Xia X and Li Y . 2021 . Image defogging algorithm using a two-phase feature extraction strategy . Journal of Image and Graphics , 26 ( 3 ): 568 - 580

袁非牛 , 李志强 , 史劲亭 , 夏雪 , 李雅 . 2021 . 两阶段特征提取策略的图像去雾 . 中国图象图形学报 , 26 ( 3 ): 568 - 580 [ DOI: 10.11834/jig.200057 http://dx.doi.org/10.11834/jig.200057 ]

Zhang H and Patel V M . 2018 . Densely connected pyramid dehazing network // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE: 3194 - 3203 [ DOI: 10.1109/CVPR.2018.00337 http://dx.doi.org/10.1109/CVPR.2018.00337 ]

Zhang Z L , Zhang X Y , Peng C , Xue X Y and Sun J . 2018 . ExFuse: enhancing feature fusion for semantic segmentation // 15th European Conference on Computer Vision . Munich : Springer: 273 - 288 [ DOI: 10.1007/978-3-030-01249-6_17 http://dx.doi.org/10.1007/978-3-030-01249-6_17 ]

Zhao W D , Li S S , Li A , Zhang B and Chen J . 2021 . Deep fusion of hyperspectral images and multi-source remote sensing data for classification with convolutional neural network . National Remote Sensing Bulletin , 25 ( 7 ): 1489 - 1502

赵伍迪 , 李山山 , 李安 , 张兵 , 陈俊 . 2021 . 结合深度学习的高光谱与多源遥感数据融合分类 . 遥感学报 , 25 ( 7 ): 1489 - 1502 [ DOI: 10.11834/jrs.20219117 http://dx.doi.org/10.11834/jrs.20219117 ]

Zhu Q S , Mai J and Shao L . 2014 . Single image dehazing using color attenuation prior // Proceedings of the British Machine Vision Conference 2014 . Nottingham : BMVA [ DOI: 10.5244/C.28.114 http://dx.doi.org/10.5244/C.28.114 ]

文章被引用时，请邮件提醒。

提交

集成主体边缘分离和多尺度信息提取的双分支建筑物提取网络

岩溶湿地无人机LiDAR点云精细3D制图及植被空间分异研究

LAE-Transformer：融合局部感知增强的机载LiDAR点云分割网络