سابِق|تال

A dual-attention capsule network for building extraction from high-resolution remote sensing imagery

  • role： First author第一作者
  • Affiliation：

    School of Remote Sensing and Geomatics Engineering, Nanjing University of Information Science and Technology, Nanjing 210044, China

  • Email：xuzs@nuist.edu.cn
  • Introduction：许正森，研究方向为高分辨率遥感影像处理。E-mail： xuzs@nuist.edu.cn
XU Zhengsen1,  
  • role： Corresponding author通信作者
  • Affiliation：

    School of Remote Sensing and Geomatics Engineering, Nanjing University of Information Science and Technology, Nanjing 210044, China

  • Email：guanhy.nj@nuist.edu.cn
  • Introduction：管海燕，研究方向为遥感数据智能处理。E-mail： guanhy.nj@nuist.edu.cn
GUAN Haiyan1*,  
  • Affiliation：

    School of Remote Sensing and Geomatics Engineering, Nanjing University of Information Science and Technology, Nanjing 210044, China

PENG Daifeng1,  
  • Affiliation：

    Faculty of Computer and Software Engineering, Huaiyin Institute of Technology, Huaian 223003, China

YU Yongtao2,  
  • Affiliation：

    School of Remote Sensing and Geomatics Engineering, Nanjing University of Information Science and Technology, Nanjing 210044, China

LEI Xiangda1,  
  • Affiliation：

    School of Remote Sensing and Geomatics Engineering, Nanjing University of Information Science and Technology, Nanjing 210044, China

ZHAO Haohao1

ملخص

Automatic extraction of buildings from high-resolution remote sensing images is greatly important in disaster prevention and mitigation, disaster loss estimation, urban planning, and topographic map making. With the advancement of optical remote sensing techniques in image resolutions and qualities, remote sensing images have provided an important data source for assisting the rapid updating of building footprint database. Despite the large number of algorithms proposed with enhanced performance, fulfilling highly accurate and fully automated extraction of buildings from remote sensing images is still difficult due to the considerable challenging scenarios of buildings, such as color diversities, topology variations, occlusions, and shadow covers. Thus, exploiting advanced and high-performance techniques to further improve the accuracy and automation level of building extraction is greatly meaningful and urgently required by a large variety of applications.To overcome the issues of strong variability and weak homogeneity of traditional convolutional neural networks, we propose a novel dual-attention capsule encoder–decoder network DA-CapsNet for extracting buildings. In this network, a deep capsule encoder–decoder network, along with the channel-spatial attention blocks, is developed to enhance the capability of extracting high-level feature information from very high resolution remote sensed images. Thus, this model has the ability to extract buildings covered by shadows and discriminate buildings from non-building impervious surfaces. Specifically, we initially employ a deep capsule encoder–decoder network to extract and fuse multiscale building capsule features, resulting in a high-quality building feature representation. Moreover, spatial attention and channel attention modules are designed to further rectify and enhance the captured contextual information to obtain a competitive performance in processing buildings in the diverse challenging scenarios. The contributions include the following: (1) the deep capsule encoder–decoder network is designed to generate a high-quality feature representation; (2) the channel and spatial feature attention modules are designed to highlight channel-wise salient features and focus on class-specific spatial features.The proposed DA-CapsNet was evaluated on three datasets: one Google Building Dataset and two publicly-available datasets (Wuhan and Massachusetts). The experimental results achieved a competitive performance with an average precision, recall, and F1-score of 92.15%, 92.07%, and 92.18%, respectively, in handling buildings of varying challenging scenarios. Considering the overall accuracy of F1-score, the DA-CapsNet achieved the values of 92.70%, 94.01%, and 89.84% for Google, WUH, and MA datasets, respectively. Comparative studies also confirmed the robust applicability and superior performance of the DA-CapsNet in building extraction tasks.

مفهوم

building extraction;deep learning;channel feature attention;spatial feature attention;encoder-decoder network;capsule network

References

  1. 1.
    Bi Q, Qin K, Zhang H, Zhang Y, Li Z L and Xu K. 2019. A multi-scale filtering building index for building extraction in very high-resolution satellite imagery. Remote Sensing, 11(5): 482
  2. 2.
    Chen J, Wang G B, Luo L B, Gong W P and Cheng Z. 2021. Building area estimation in drone aerial images based on Mask R-CNN. IEEE Geoscience and Remote Sensing Letters, 18(5): 891-894
  3. 3.
    Chen K Q, Gao X, Yan M L, Zhang Y and Sun X. 2020. Building extraction in pixel level from aerial imagery with a deep encoder-decoder network. Journal of Remote Sensing, 24(9): 1134-1142
  4. 4.
    Chen L C, Zhu Y K, Papandreou G, Schroff F and Adam H. 2018. Encoder-decoder with atrous separable convolution for semantic image segmentation//15th European Conference on Computer Vision. Munich: Springer: 833-851
  5. 5.
    Chen S X, Shi W Z, Zhou M T, Zhang M and Chen P F. 2020. Automatic building extraction via adaptive iterative segmentation with LiDAR data and high spatial resolution imagery fusion. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13: 2081-2095 [DOI:
  6. 6.
    Cui W H, Xiong B Y and Zhang L Y. 2019. Multi-scale fully convolutional neural network for building extraction. Acta Geodaetica et Cartographica Sinica, 48(5): 597-608
  7. 7.
    Fan R S, Chen Y, Xu Q H and Wang J X. 2019. A high-resolution remote sensing image building extraction method based on deep learning. Acta Geodaetica et Cartographica Sinica, 48(1): 34-41
  8. 8.
    Gao X J, Wang M W, Yang Y W and Li G Q. 2018. Building extraction from RGB VHR images using shifted shadow algorithm. IEEE Access, 6: 22034-22045
  9. 9.
    Guo M Q, Liu H, Xu Y Y and Huang Y. 2020. Building extraction based on U-Net with an attention block and multiple losses. Remote Sensing, 12(9): 1400
  10. 10.
    Hao L C, Zhang Y and Cao Z M. 2019. Active cues collection and integration for building extraction with high-resolution color remote sensing imagery. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(8): 2675-2694
  11. 11.
    Hu J, Shen L and Sun G. 2018. Squeeze-and-excitation networks//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT: IEEE: 7132-7141
  12. 12.
    Hu R M, Huang X B and Huang Y C. 2014. An enhanced morphological building index for building extraction from high-resolution images. Acta Geodaetica et Cartographica Sinica, 43(5): 514-520
  13. 13.
    Ji S P and Wei S Q. 2019. Building extraction via convolutional neural networks from an open remote sensing building dataset. Acta Geodaetica et Cartographica Sinica, 48(4): 448-459
  14. 14.
    Ji S P, Wei S Q and Lu M. 2019. Fully convolutional networks for multisource building extraction from an open aerial and satellite imagery data set. IEEE Transactions on Geoscience and Remote Sensing, 57(1): 574-586
  15. 15.
    Jing R, Gong Z N, Zhu W D, Guan H L, Zhao W J and Zhang T. 2020. Extraction of buildings from remote sensing imagery based on multi-scale SLIC-GMRF and FCNSVM. Journal of Remote Sensing, 24(1): 11-26
  16. 16.
    Konstantinidis D, Stathaki T, Argyriou V and Grammalidis N. 2017. Building detection using enhanced HOG–LBP features and region refinement processes. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 10(3): 888-905
  17. 17.
    Li Q Y, Shi Y L, Huang X and Zhu X X. 2020. Building footprint generation by integrating convolution neural network with feature pairwise conditional random field (FPCRF). IEEE Transactions on Geoscience and Remote Sensing, 58(11): 7502-7519
  18. 18.
    Lin X G and Zhang J X. 2017. Object-based morphological building index for building extraction from high resolution remote sensing imagery. Acta Geodaetica et Cartographica Sinica, 46(6): 724-733
  19. 19.
    Mnih V. 2013. Machine Learning for Aerial Image Labeling. Toronto, ON: University of Toronto
  20. 20.
    Pan X R, Yang F, Gao L R, Chen Z C, Zhang B, Fan H R and Ren J C. 2019. Building extraction from high-resolution aerial imagery using a generative adversarial network with spatial and channel attention mechanisms. Remote Sensing, 11(8): 917
  21. 21.
    Qin X B, He S D, Yang X C, Dehghan M, Qin Q M and Martin J. 2018. Accurate outline extraction of individual building from very high-resolution optical images. IEEE Geoscience and Remote Sensing Letters, 15(11): 1775-1779
  22. 22.
    Rajasegaran J, Jayasundara V, Jayasekara S, Jayasekara H, Seneviratne S and Rodrigo R. 2019. DeepCaps: going deeper with capsule networks//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA: IEEE: 10717-10725
  23. 23.
    Ronneberger O, Fischer P and Brox T. 2015. U-Net: convolutional networks for biomedical image segmentation//18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich: Springer: 234-241
  24. 24.
    Sabour S, Frosst N and Hinton G E. 2017. Dynamic routing between capsules. arXiv: 1710.09829
  25. 25.
    Shao Z F, Tang P H, Wang Z Y, Saleem N, Yam S and Sommai C. 2020. BRRNet: a fully convolutional neural network for automatic building extraction from high-resolution remote sensing images. Remote Sensing, 12(6): 1050
  26. 26.
    Shelhamer E, Long J and Darrell T. 2017. Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4): 640-651
  27. 27.
    Shi Y L, Li Q Y and Zhu X X. 2019. Building footprint generation using improved generative adversarial networks. IEEE Geoscience and Remote Sensing Letters, 16(4): 603-607
  28. 28.
    Shi Y L, Li Q Y and Zhu X X. 2020. Building segmentation through a gated graph convolutional neural network with deep structured feature embedding. ISPRS Journal of Photogrammetry and Remote Sensing, 159: 184-197
  29. 29.
    Wang S S, Hou X W and Zhao X. 2020. Automatic building extraction from high-resolution aerial imagery via fully convolutional encoder-decoder network with non-local block. IEEE Access, 8: 7313-7322
  30. 30.
    Wang Y H, Gu L J, Li X F and Ren R Z. 2021. Building extraction in multitemporal high-resolution remote sensing imagery using a multifeature LSTM network. IEEE Geoscience and Remote Sensing Letters, 18(9): 1645-1649
  31. 31.
    Wang Z Q, Zhou Y, Wang S X, Wang F T and Xu Z Y. 2021. House building extraction from high-resolution remote sensing images based on IEU-Net. Journal of Remote Sensing, 25(11): 2245-2254
  32. 32.
    Woo S, Park J, Lee J Y and Kweon I S. 2018. CBAM: convolutional block attention module//15th European Conference on Computer Vision. Munich: Springer: 3-19
  33. 33.
    Xu S B, Pan X J, Li E, Wu B Y, Bu S H, Dong W M, Xiang S M and Zhang X P. 2018. Automatic building rooftop extraction from aerial images via hierarchical RGB-D priors. IEEE Transactions on Geoscience and Remote Sensing, 56(12): 7369-7387
  34. 34.
    You Y F, Wang S Y, Wang B, Ma Y X, Shen M, Liu W H and Xiao L. 2019. Study on hierarchical building extraction from high resolution remote sensing imagery. Journal of Remote Sensing, 23(1): 125-136
  35. 35.
    Yu B S and Tao D C. 2019. Deep metric learning with tuplet margin loss//2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul: IEEE: 6489-6498
  36. 36.
    Yu Y T, Ren Y F, Guan H Y, Li D L, Yu C H, Jin S H and Wang L F. 2021. Capsule feature pyramid network for building footprint extraction from high-resolution aerial imagery. IEEE Geoscience and Remote Sensing Letters, 18(5) 895-899
  37. 37.
    Zeiler M D and Fergus R. 2014. Visualizing and understanding convolutional networks//13th European Conference on Computer Vision. Zurich: Springer: 818-833
  38. 38.
    Zhang Q, Huang X and Zhang G X. 2016. A morphological building detection framework for high-resolution optical imagery over urban areas. IEEE Geoscience and Remote Sensing Letters, 13(9): 1388-1392
  39. 39.
    Zhu Q Q, Zhang Y N, Wang L Z, Zhong Y F, Guan Q F, Lu X Y, Zhang L P and Li D R. 2021. A Global Context-aware and Batch-independent Network for road extraction from VHR satellite imagery. ISPRS Journal of Photogrammetry and Remote Sensing, 175: 353-365

قراءة النص الكامل

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website