Anterior|Siguiente

Building extraction in pixel level from aerial imagery with a deep encoder-decoder network

  • role： First author第一作者
  • Affiliation：

    Key Laboratory of Network Information System Technology, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China

    University of Chinese Academy of Sciences, Beijing 100049, China

  • Email：chenkaiqiang14@mails.ucas.ac.cn
  • Introduction：陈凯强， 男，1992年生，博士研究生， 研究方向为深度学习、语义分割、遥感影像建筑物提取和地物分类。E-mail: chenkaiqiang14@mails.ucas.ac.cn
CHEN Kaiqiang12,  
  • Affiliation：

    Key Laboratory of Network Information System Technology, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China

GAO Xin1,  
  • Affiliation：

    Key Laboratory of Network Information System Technology, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China

YAN Menglong1,  
  • Affiliation：

    Key Laboratory of Network Information System Technology, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China

ZHANG Yue1,  
  • role： Corresponding author通信作者
  • Affiliation：

    Key Laboratory of Network Information System Technology, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China

  • Email：sunxian@aircas.ac.cn
  • Introduction：孙显，男，1981年生，研究员，研究方向为计算机视觉与遥感图像理解。E-mail: sunxian@aircas.ac.cn
SUN Xian1*

resumen

Building extraction plays a significant role in land use analysis like urban planning. Classical methods based on hand-crafted features fail to derive prominent building extraction results due to the limited representation capacity of the hand-crafted features. In this paper, we achieve building extraction in pixel level based on a deep Convolutional Neural Network (CNN) with an encoder-decoder structure. In contrast to the hand-crafted features that require professional knowledge and have a poor representation capacity, convolutional neural networks are equipped with a high representation capacity and able to learn highly abstract and distinguishing features from data. The encoder is used to derived a space compressed representation of the input raw image. This compressed representation is also called a feature of the input image and it is assumed to be abstract and distinguishing. The decoder uses the feature as input and recover the space resolution to the size of the input image. Thereby, the encoder-decoder network achieves pixel-wise building extraction in an end-to-end way from the raw image to the building extraction result.Applying the encoder-decoder network to building extraction will cause a Marginal Phenomenon (MP). Specifically, the prediction accuracy near the edges of a patch is usually lower than that near the central area. Marginal phenomenon will lead to the reduction of building extraction accuracy. To alleviate this effect, we propose the usage of the Field of View Enhancement (FoVE) method. The FoVE method includes two parts: enlarging the patch size and cropping patches with overlaps when making predictions. Therefore, the FoVE method contains two hyper-parameters, which are patch size and overlapping size. Extensive experiments on two building extraction datasets are conducted to analyze the impact of the two hyper-parameters through the Precision-Recall Curves (PRC) and some interesting conclusions are derived from the the analysis: (1) Enlarging the input patch size when making prediction can effectively improve the building extraction performance while the improvement saturates as the overlapping size increases; (2) Cropping patches with an overlap when making prediction can improve the building extraction performance while the improvement saturates as the input patch size increases; (3) The FoVE can effectively improve building extraction accuracy but this improvement from the FoVE has a limit; (4) The convolutional neural network for building extraction plays the key role and further attentions should be focused on the network design.In addition to the numerical analysis of the FoVE experimental results, we attempt to explain why FoVE works and why it has a limit. We blame them on the Field of View (FoV) and that is reason why the method is call FoVE. FoV plays an important role in building extraction and a larger FoV is beneficial to building extraction. Firstly, the marginal phenomenon is caused by the lack of context information of the marginal pixels. FoVE improves the overall accuracy through abandoning the unreliable predictions of the marginal pixels. Secondly, enlarging input patches can enlarge the FoV of each pixel and thus improves the accuracy. Thirdly, the the improvement from FoVE has a limit because that when the field of view is large enough, the improvement derived from more contextual information can be ignore.

palabra clave

remote sensing;building extraction;convolutional neural network;deep learning;aerial imagery

References

  1. 1.
    Chen K Q, Fu K, Yan M L, Gao X, Sun X and Wei X. 2018a. Semantic segmentation of aerial images with shuffling convolutional neural networks. IEEE Geoscience and Remote Sensing Letters, 15(2): 173-177
  2. 2.
    Chen K Q, Weinmann M, Gao X, Yan M L, Hinz S, Jutzi B and Weinmann M, 2018b. Residual shuffling convolutional neural networks for deep semantic image segmentation using multi-modal data. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 4(2): 65-72
  3. 3.
    Chen T Q, Li M, Li Y T, Lin M, Wang N Y, Wang M J, Xiao T J, Xu B, Zhang C Y and Zhang Z. 2015. Mxnet: a flexible and efficient machine learning library for heterogeneous distributed systems. arXiv:1512.01274
  4. 4.
    Decatur S E. 1989. Application of neural networks to terrain classification//1989 International Joint Conference on Neural Networks. Washing: IEEE: 283-288
  5. 5.
    Grabner H, Nguyen T T, Gruber B and Bischof H. 2008. On-line boosting-based car detection from aerial images. ISPRS Journal of Photogrammetry and Remote Sensing, 63(3): 382-396
  6. 6.
    Haralick R M, Shanmugam K and Dinstein I H. 1973. Textural features for image classification. IEEE Transactions on Systems, Man, and Cybernetics, SMC-3(6): 610-621
  7. 7.
    He K M, Zhang X Y, Ren S Q and Sun J. 2015. Delving deep into rectifiers: surpassing human-level performance on imagenet classification//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago, Chile: IEEE: 1026-1034
  8. 8.
    He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA: IEEE
  9. 9.
    Huang C, Davis L S and Townshend J R G. 2002. An assessment of support vector machines for land cover classification. International Journal of Remote Sensing, 23(4): 725-749
  10. 10.
    Krizhevsky A, Sutskever I and Hinton G E. 2012. Imagenet classification with deep convolutional neural networks//Proceedings of the 25th International Conference on Neural Information Processing Systems. Red Hook, NY: ACM: 1097-1105
  11. 11.
    Lee J, Weger R C, Sengupta S K and Welch R M. 1990. A neural network approach to cloud classification. IEEE Transactions on Geoscience and Remote Sensing, 28(5): 846-855
  12. 12.
    Mnih V. 2013. Machine Learning for Aerial Image Labeling. Toronto: University of Toronto
  13. 13.
    Paisitkriangkrai S, Sherrah J, Janney P and van Den Hengel A. 2015. Effective semantic pixel labelling with convolutional networks and conditional random fields//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Boston, MA, USA: IEEE: 36-43
  14. 14.
    Paola J D and Schowengerdt R A. 1995. A detailed comparison of backpropagation neural network and maximum-likelihood classifiers for urban land use classification. IEEE Transactions on Geoscience and Remote Sensing, 33(4): 981-996
  15. 15.
    Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S A, Huang Z H, Karpathy A, Khosla A, Bernstein M, Berg A C and Li F F. 2015. ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 115(3): 211-252
  16. 16.
    Saito S and Aoki Y. 2015. Building and road detection from large aerial imagery//Proceedings of SPIE 9405, Image Processing: Machine Vision Applications VIII. San Francisco: SPIE
  17. 17.
    Saito S, Yamashita T and Aoki Y. 2016. Multiple object extraction from aerial imagery with convolutional neural networks. Journal of Imaging Science and Technology, 60(1): 010402
  18. 18.
    Sherrah J. 2016. Fully convolutional networks for dense semantic labelling of high-resolution aerial imagery. arXiv:1606.02585
  19. 19.
    Simonyan K and Zisserman A. 2014. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556
  20. 20.
    Song X M, Fan G L and Rao M. 2005. Automatic CRP mapping using nonparametric machine learning approaches. IEEE Transactions on Geoscience and Remote Sensing, 43(4): 888-897
  21. 21.
    Srivastava R K, Greff K and Schmidhuber J. 2015. Highway networks. arXiv:1505.00387
  22. 22.
    Volpi M and Tuia D. 2017. Dense semantic labeling of subdecimeter resolution images with convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 55(2): 881-893
  23. 23.
    Zeiler M D, Krishnan D, Taylor G W and Fergus R. 2010. Deconvolutional networks//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco, CA, USA: IEEE: 2528-2535
  24. 24.
    Zhang H, Lei H and Chen K Q. 2018. Application of densely connected deconvolutional neural network in building extraction from remote sensing images. Computer Engineering and Applications, 54(11): 140-144, 152

Leer el texto completo

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website