Anterior|Siguiente

A two-branch remote sensing image dehazing network based on hierarchical feature interaction and enhanced receptive field

  • role： First author第一作者
  • Affiliation：

    College of Computer and Information Technology, China Three Gorges University, Yichang, 443002, China

    Hubei Key Laboratory of Intelligent Vision Based Monitoring for Hydroelectric Engineering, China Three Gorges University, Yichang, 443002, China

  • Email：sunhang0418@whu.edu.cn
  • Introduction：孙航，研究方向为计算机视觉、图像去雾、图像增强。E-mail： sunhang0418@whu.edu.cn
SUN Hang13,  
  • Affiliation：

    College of Computer and Information Technology, China Three Gorges University, Yichang, 443002, China

FANG Shuailing1,  
  • role： Corresponding author通信作者
  • Affiliation：

    College of Computer and Information Technology, China Three Gorges University, Yichang, 443002, China

    Hubei Key Laboratory of Intelligent Vision Based Monitoring for Hydroelectric Engineering, China Three Gorges University, Yichang, 443002, China

  • Email：zp_dan@ctgu.edu.cn
  • Introduction：但志平，研究方向为计算机视觉、自然语言处理。E-mail： zp_dan@ctgu.edu.cn
DAN Zhiping13*,  
  • Affiliation：

    College of Computer and Information Technology, China Three Gorges University, Yichang, 443002, China

    Hubei Engineering Technology Research Center for Farmland Environmental, China Three Gorges University, Yichang, 443002, China

REN Dong12,  
  • Affiliation：

    College of Computer and Information Technology, China Three Gorges University, Yichang, 443002, China

    Hubei Engineering Technology Research Center for Farmland Environmental, China Three Gorges University, Yichang, 443002, China

YU Mei12,  
  • Affiliation：

    College of Computer and Information Technology, China Three Gorges University, Yichang, 443002, China

    Hubei Key Laboratory of Intelligent Vision Based Monitoring for Hydroelectric Engineering, China Three Gorges University, Yichang, 443002, China

SUN Shuifa13

resumen

In recent years, deep learning-based dehazing methods have achieved remarkable results in the field of image dehazing. However, most dehazing methods based on U-shaped networks directly transfer the features of the encoding layer to the corresponding decoding layers, which lacks information interaction between the low- and high-level features. Meanwhile, the network model designed based on the U-shaped structure may destroy the detailed information important for the restored image in the process of downsampling. As a result, the restored clear image lacks detailed texture and structure information. In addition, the dehazing method based on non-U-shaped network has limited receptive field, which hinders its capability to effectively utilize contextual information. As a result, these methods cannot achieve ideal dehazing results in remote sensing images with large scene scale changes. Therefore, this study proposes a two-branch remote sensing image dehazing network based on hierarchical feature interaction and enhanced receptive field. This network includes hierarchical feature interaction sub-net and multi-scale information extraction sub-net. The hierarchical feature interaction sub-net uses the hierarchical feature interaction fusion module to introduce semantic information into low-level features and spatial details into high-level features layer by layer. This way enhances the information interaction between features at different levels in the encoding layer. The multi-scale information extraction sub-net uses the multi-scale residual dilated convolution module to fuse the features of different receptive fields for obtaining contextual information, which is crucial for remote sensing image dehazing. The experiment on two public datasets show that the dehazing method proposed in this study achieves the best evaluation compared with the existing nine excellent dehazing algorithms. Among them, in the three sub-test sets of the public remote sensing dataset Haze1k, the quantitative index PSNR values of this study reach 27.362, 28.171, and 25.137 dB. In the two sub-test sets of the public remote sensing dataset RICE, the quantitative index PSNR values of this study reach 37.79 and 35.367 dB. In addition, the method proposed in this study is the closest to ground truth in terms of subjective visual qualities such as color, saturation, and sharpness, while still achieving the dehazing effect. The following conclusions can be drawn: (1) through the proposed hierarchical feature interaction fusion module, the deep semantic information in the coding stage is gradually interactively fused with the shallow detailed texture information, which enhances the expressive ability of the network and restores clear images with higher quality. (2) Through the multi-scale residual dilated convolution module, the dehazing network proposed in this study can increase the receptive field of the network without changing the size of the feature map. The contextual information of different scales can also be fused. (3) In two public remote sensing image dehazing datasets, namely, Haze1k and RICE, the dehazing method proposed in this study outperforms nine recently proposed excellent dehazing algorithms in terms of objective evaluation indexes and subjective visual effects.

palabra clave

deep learning;remote sensing image dehazing;hierarchical feature interaction;receptive field;two-branch

References

  1. 1.
    Cai B L, Xu X M, Jia K, Qing C and Tao D C. 2016. DehazeNet: an end-to-end system for single image haze removal. IEEE Transactions on Image Processing, 25(11): 5187-5198
  2. 2.
    Cox L J. 1977. Optics of the atmosphere-scattering by molecules and particles. Optica Acta: International Journal of Optics, 24(7): 779-779
  3. 3.
    Deng J, Dong W, Socher R, Li L J, Li K and Fei-Fei L. 2009. ImageNet: a large-scale hierarchical image database//2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami: IEEE: 248-255
  4. 4.
    Dong Y, Liu Y H, Zhang H, Chen S F and Qiao Y. 2020. FD-GAN: generative adversarial networks with fusion-discriminator for single image dehazing. Proceedings of the AAAI Conference on Artificial Intelligence, 34(7): 10729-10736
  5. 5.
    Engin D, Gen A and Ekenel H K. 2018. Cycle-Dehaze: enhanced CycleGAN for single image dehazing//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Salt Lake City: IEEE: 825-833
  6. 6.
    Gao S H, Cheng M M, Zhao K, Zhang X Y, Yang M H and Torr P. 2021. Res2Net: a new multi-scale backbone architecture. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(2): 652-662
  7. 7.
    He K M, Sun J and Tang X O. 2011. Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(12): 2341-2353
  8. 8.
    He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE: 770-778
  9. 9.
    Hu J, Shen L, Albanie S, Sun G and Wu E H. 2020. Squeeze-and-excitation networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(8): 2011-2023
  10. 10.
    Huang B H, Li Z, Yang C, Sun F C and Song Y X. 2020. Single satellite optical imagery dehazing using SAR image prior based on conditional generative adversarial networks//2020 IEEE Winter Conference on Applications of Computer Vision. Snowmass: IEEE: 1795-1802
  11. 11.
    Huang Y F and Chen X. 2021. Single remote sensing image dehazing using a dual-step cascaded residual dense network//2021 IEEE International Conference on Image Processing. Anchorage: IEEE: 3852-3856
  12. 12.
    Kim U H, Kim S H and Kim J H. 2022. SimVODIS: simultaneous visual odometry, object detection, and instance segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(1): 428-441
  13. 13.
    Ledig C, Theis L, Huszar F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z H and Shi W Z. 2017. Photo-realistic single image super-resolution using a generative adversarial network//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 105-114
  14. 14.
    Li B, Luo H, Zhang H X, Tan S Q and Ji Z Z. 2017. A multi-branch convolutional neural network for detecting double JPEG compression. arXiv:1710.05477
  15. 15.
    Li Y F and Chen X. 2021. A coarse-to-fine two-stage attentive network for haze removal of remote sensing images. IEEE Geoscience and Remote Sensing Letters, 18(10): 1751-1755
  16. 16.
    Li Z L, Wang L Y, Jiang S, Wu Y H and Zhang Q J. 2021. On orbit extraction method of ship target in SAR images based on ultra-lightweight network. National Remote Sensing Bulletin, 25(3): 765-775
  17. 17.
    Lin D Y, Xu G L, Wang X K, Wang Y, Sun X and Fu K. 2019. A remote sensing image dataset for cloud removal. arXiv:1901.00600
  18. 18.
    Mehta A, Sinha H, Mandal M and Narang P. 2021. Domain-aware unsupervised hyperspectral reconstruction for aerial image dehazing//2021 IEEE Winter Conference on Applications of Computer Vision. Waikoloa: IEEE: 413-422
  19. 19.
    Mou F. 2021. Research on Thin Cloud Removal and Application of Remote Sensing Image Based on Deep Learning. Chengdu: University of Electronic Science and Technology of China (牟范. 2021. 基于深度学习的遥感影像去薄云及应用研究. 成都: 电子科技大学) [DOI: 10.27005/d.cnki.gdzku.2021.001113]
  20. 20.
    Qin X, Wang Z L, Bai Y C, Xie X D and Jia H Z. 2020. FFA-Net: feature fusion attention network for single image dehazing. Proceedings of the AAAI Conference on Artificial Intelligence, 34(7): 11908-11915
  21. 21.
    Qu Y Y, Chen Y Z, Huang J Y and Xie Y. 2019. Enhanced Pix2pix dehazing network//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE: 8152-8160
  22. 22.
    Ren W Q, Liu S, Zhang H, Pan J S, Cao X C and Yang M H. 2016. Single image dehazing via multi-scale convolutional neural networks//14th European Conference on Computer Vision. Amsterdam: Springer: 154-169
  23. 23.
    Ren W Q, Ma L, Zhang J W, Pan J S, Cao X C, Liu W and Yang M H. 2018. Gated fusion network for single image dehazing//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 3253-3261
  24. 24.
    Shi W Z, Caballero J, Huszár F, Totz J, Aitken A P, Bishop R, Rueckert D and Wang Z H. 2016. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE: 1874-1883
  25. 25.
    Simonyan K and Zisserman A. 2015. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556
  26. 26.
    Song Y F, Li J, Wang X G and Chen X W. 2018. Single image dehazing using ranking convolutional neural network. IEEE Transactions on Multimedia, 20(6): 1548-1560
  27. 27.
    Sun X and Xu J D. 2021. Remote sensing image dehazing method based on cascaded generative adversarial network. Journal of Computer Applications, 41(8): 2440-2444
  28. 28.
    Tan R T. 2008. Visibility in bad weather from a single image//2008 IEEE Conference on Computer Vision and Pattern Recognition. Anchorage: IEEE: 1-8
  29. 29.
    Tang K T, Yang J C and Wang J. 2014. Investigating haze-relevant features in a learning framework for image dehazing//2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE: 2995-3002
  30. 30.
    Wang L W, Li Y, Huang J and Lazebnik S. 2019. Learning two-branch neural networks for image-text matching tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(2): 394-407
  31. 31.
    Wei X S, Xu Y Y and Yang J. 2022. Review of webly-supervised fine-grained image recognition. Journal of Image and Graphics, 27(7): 2057-2077
  32. 32.
    Wu H Y, Liu J, Xie Y, Qu Y Y and Ma L Z. 2020. Knowledge transfer dehazing network for NonHomogeneous dehazing//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Seattle: IEEE: 1975-1983
  33. 33.
    Wu H Y, Qu Y Y, Lin S H, Zhou J, Qiao R Z, Zhang Z Z, Xie Y and Ma L Z. 2021. Contrastive learning for compact single image dehazing//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE: 10546-10555 []
  34. 34.
    Yang S T, Wang P F, Wang J, Lou H Z and Gong T L. 2021. River flow estimation method based on UAV aerial photogrammetry. National Remote Sensing Bulletin, 25(6): 1284-1293
  35. 35.
    Yu F and Koltun V. 2016. Multi-scale context aggregation by dilated convolutions. arXiv:1511.07122
  36. 36.
    Yu Y K, Liu H, Fu M H, Chen J, Wang X Y and Wang K Y. 2021. A two-branch neural network for non-homogeneous dehazing via ensemble learning//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Nashville: IEEE: 193-202
  37. 37.
    Yuan F N, Li Z Q, Shi J T, Xia X and Li Y. 2021. Image defogging algorithm using a two-phase feature extraction strategy. Journal of Image and Graphics, 26(3): 568-580
  38. 38.
    Zhang H and Patel V M. 2018. Densely connected pyramid dehazing network//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 3194-3203
  39. 39.
    Zhang Z L, Zhang X Y, Peng C, Xue X Y and Sun J. 2018. ExFuse: enhancing feature fusion for semantic segmentation//15th European Conference on Computer Vision. Munich: Springer: 273-288
  40. 40.
    Zhao W D, Li S S, Li A, Zhang B and Chen J. 2021. Deep fusion of hyperspectral images and multi-source remote sensing data for classification with convolutional neural network. National Remote Sensing Bulletin, 25(7): 1489-1502
  41. 41.
    Zhu Q S, Mai J and Shao L. 2014. Single image dehazing using color attenuation prior//Proceedings of the British Machine Vision Conference 2014. Nottingham: BMVA

Leer el texto completo

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website