Précédent|Prochain

Building segmentation in high-resolution remote sensing image through deep neural network and conditional random fields

  • role： First author第一作者
  • Affiliation：

    School of Surveying and Land Information Engineering, Henan Polytechnic University, Jiaozuo 454000, China

    Field Scientific Observation and Research Base of Ministry of Land and Resources, Henan Polytechnic University, Jiaozuo 454000, China

  • Email：wangyu@hpu.edu.cn
  • Introduction：王宇，1978年生，讲师，研究方向为土地利用规划、遥感图像处理等。E-mail：wangyu@hpu.edu.cn
WANG Yu12,  
  • Affiliation：

    School of Electrical Engineering and Automation, Henan Polytechnic University, Jiaozuo 454000, China

YANG Yi3,  
  • Affiliation：

    School of Surveying and Land Information Engineering, Henan Polytechnic University, Jiaozuo 454000, China

WANG Baoshan1,  
  • Affiliation：

    School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, China

WANG Tian4,  
  • Affiliation：

    School of Electrical Engineering and Automation, Henan Polytechnic University, Jiaozuo 454000, China

BU Xuhui3,  
  • Affiliation：

    School of Computer Science, Shenyang Aerospace University, Shenyang 110136, China

WANG Chuanyun5

résumé

The core of building segmentation in high-resolution remote sensing image is to establish the mapping from an image feature space to a segmentation result with high dimension and strong nonlinearity. In a high-resolution remote sensing image, a building frequently emerges at any location in the entire image, thereby indicating that non-neighborhood pixels may be related to the current semantic segmentation pixel. The segmentation precision and generalization are significantly improved by adopting a Deep Neural Network (DNN) to extract the features and learn the nonlinear mapping in image segmentation. However, the non-neighborhood feature cannot be directly extracted by the DNN. This study presents an encoder–decoder deep learning architecture with ResNet and Conditional Random Field (CRF) for building semantic segmentation in a high-resolution remote sensing image to obtain high segmentation precision and reduce the obstacles from roads, staggered floors, and shadows. In the DNN, ResNet is used to establish the encoder for automatically extracting the building features, in which ResNet avoids the problems of vanishing and exploding gradient and accelerates the convergence of DNN weights. Before each convolution operation, batch normalization is adopted to normalize the sampling data and reduce the training difficulty of the DNN. Then, transposed convolution is applied to establish the decoder for reconstructing the image while segmenting the buildings. At the end of the DNN, the CRF is used to adjust the raw segmentation produced by the decoder. The value of a unary potential function in the CRF is given by the raw result of the decoder, and the pairwise potential function denotes the feature of pixel pairs in the entire image, which constructs a fully connected CRF (FCCRF). Considering that the calculation of FCCRF is considerable, a mean field algorithm is used to approximate the pairwise potential function value. Thus, convolution is used to obtain the pairwise potential function value, and a high-dimensional Gaussian filter is applied to implement the convolution operation. The mean field algorithm is implemented through an RNN mechanism. Thus, FCCRF becomes a part of the DNN, and the parameters of the CRF are trained with the encoder and decoder simultaneously. Experiments are conducted to validate the effectiveness of the proposed methodology. The remote sensing image dataset is Inria Aerial Image Labeling Dataset. A total of 4500 samples with 1000×1000×3 pixels are found in each sample, in which their resolution is 0.3 m. The typical kinds of building, such as building with order, single building with complicated roof, and building without order, are segmented through VGG, ResNet, and the proposed methodology (denoted as ResNetCRF), correspondingly. The results show that ResNetCRF overcomes the interruption of roads in which their color features are similar to the building and effectively reduces the disturbance of staggered floors and shadows. Thus, ResNetCRF obtains the optimal segmentation precision. The multi-resolution experiment demonstrates that ResNetCRF has a strong generalization under a limited range of resolution change. Accurate mapping of building segmentation is established to reduce the disturbance of roads, shadows, and staggered floors by introducing CRFs in the encoder–decoder based on ResNet to segment the building in a high-resolution remote sensing image. In the future work, we will investigate the reduction of FCCRF calculation, overcome the missing segmentation of small buildings, and reduce the segmentation errors of a building whose color feature is similar to the background without a noticeable edge.

mots-clés

high resolution remote sensing image;deep neural network;conditional random fields;building segmentation

References

  1. 1.
    Adams A, Baek J and Davis M A. 2010. Fast high‐dimensional filtering using the permutohedral lattice. Computer Graphics Forum, 29(2): 753–762
  2. 2.
    Alshehhi R, Marpu P R, Woon W L and Mura M D. 2017. Simultaneous extraction of roads and buildings in remote sensing imagery with convolutional neural networks. ISPRS Journal of Photogrammetry and Remote Sensing, 130: 139–149
  3. 3.
    Audebert N, Le Saux B and Lefèvre S. 2018. Beyond RGB: very high resolution urban remote sensing with multimodal deep networks. ISPRS Journal of Photogrammetry and Remote Sensing, 140: 20–32
  4. 4.
    Badrinarayanan V, Kendall A and Cipolla R. 2017. SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(12): 2481–2495
  5. 5.
    Bittner K, Cui S Y and Reinartz P. 2017. Building extraction from remote sensing data using fully convolutional Networks//International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Hannover, Germany: ISPR: 481–486
  6. 6.
    Chatzis S P, Kosmopoulos D I and Doliotis P. 2013. A conditional random field-based model for joint sequence segmentation and classification. Pattern Recognition, 46(6): 1569–1578
  7. 7.
    Chen J, Deng M, Xiao P F, Yang M H, Mei X M and Liu H M. 2011. Multi-scale watershed segmentation of high-resolution multi-spectral remote sensing image using wavelet transform. Journal of Remote Sensing, 15(5): 908–926
  8. 8.
    Chen L C, Papandreou G, Kokkinos I, Murphy K and Yuille A L. 2014. Semantic image segmentation with deep convolutional nets and fully connected CRFs. arVix preprint arXiv: 1412.7062(2014)
  9. 9.
    Chen L C, Papandreou G, Kokkinos I, Murphy K and Yuille A L. 2017. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4): 834–848
  10. 10.
    Dumoulin V and Visin F. 2016. A guide to convolution arithmetic for deep learning. arXiv preprint arXiv: 1603.07285(2016)
  11. 11.
    Glorot X, Bordes A and Bengio Y. 2011. Deep sparse rectifier neural networks//Proceedings of the 14th International Conference on Artificial Intelligence and Statistics. Fort Lauderdale, FL, USA: [s.n.]
  12. 12.
    He K M, Zhang X Y, Ren S Q and Sun J. 2015. Deep residual learning for image recognition. arXiv preprint arXiv: 1512.03385(2015)
  13. 13.
    Ioffe S and Szegedy C. 2015. Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv: 1502.03167(2015)
  14. 14.
    Jiao L C, Liang M M, Chen H, Yang S Y, Liu H Y and Cao X H. 2017. Deep fully convolutional network-based spatial distribution prediction for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 55(10): 5585–5599
  15. 15.
    Krähenbühl P and Koltun V. 2012. Efficient inference in fully connected CRFs with gaussian edge potentials. arXiv preprint arXiv: 1210.5644(2012)
  16. 16.
    Li H. 2012. Statistical Learning Method. Beijing: Tsinghua University Press
  17. 17.
    Lin H N, Shi Z W and Zou Z X. 2017. Fully convolutional network with task partitioning for inshore ship detection in optical remote sensing images. IEEE Geoscience and Remote Sensing Letters, 14(10): 1665–1669
  18. 18.
    Liu Y, Carbonell J, Weigele P and Gopalakrishnan V. 2005. Segmentation conditional random fields (SCRFs): a new approach for protein fold recognition//Proceedings of the 9th Annual International Conference on Research in Computational Molecular Biology. Cambridge, MA, USA: Springer: 408–422
  19. 19.
    Liu Y S, Piramanayagam S, Monteiro S T and Saber E. 2017. Dense semantic labeling of very-high-resolution aerial imagery and LiDAR with fully-convolutional neural networks and higher-order CRFs//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Honolulu, HI, USA: IEEE: 1561–1570
  20. 20.
    Noh H, Hong S and Han B. 2015. Learning deconvolution network for semantic segmentation//Proceedings of 2015 IEEE International Conference on Computer Vision (ICCV). Santiago, Chile: IEEE: 1520–1528
  21. 21.
    Ronneberger O, Fischer P and Brox T. 2015. U-Net: convolutional networks for biomedical image segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich, Germany: Springer
  22. 22.
    Shelhamer E, Long J and Darrell T. 2017. Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4): 640–651
  23. 23.
    Shen J X. 2004. Connectionism in AI and grammatical theories. Journal of Foreign Languages(3): 2–10
  24. 24.
    Simonyan K and Zisserman A. 2014. Visual geometry group[EB/OL]. (2014)
  25. 25.
    Wang Y, Li Y and Zhao Q H. 2018. A region-based multiscale segmentation of panchromatic remote sensing image. Control and Decision, 33(3): 535–541
  26. 26.
    Wang Y, Wang B S, Wang T and Yang Y. 2018. Image entropy active contour models towards water area segmentation in remote sensing image. Optics and Precision Engineering, 26(3): 698–707
  27. 27.
    Wang Y Y, Wang C and Zhang H. 2017. Integrating H-A-α with fully convolutional networks for fully PolSAR classification//Proceedings of 2017 International Workshop on Remote Sensing with Intelligent Processing (RSIP). Shanghai, China: IEEE: 1-4
  28. 28.
    Xu L L, Shafiee M J, Wong A and Clausi D A. 2016. Fully connected continuous conditional random field with stochastic cliques for dark-spot detection in SAR imagery. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 9(7): 2882–2890
  29. 29.
    Zhang J L and Wang B. 2017. SAR image change detection method of DSSRM based on cascade segmentation. Journal of Remote Sensing, 21(4): 614–621
  30. 30.
    Zheng S, Jayasumana S, Romera-Paredes B, Vineet V, Su Z Z, Du D L, Huang C and Torr P H S. 2015. Conditional random fields as recurrent neural networks//Proceedings of 2015 IEEE International Conference on Computer Vision (ICCV). Santiago, Chile: IEEE: 1529–1537
  31. 31.
    Zhong Y F, Han X B and Zhang L P. 2018. Multi-class geospatial object detection based on a position-sensitive balancing framework for high spatial resolution remote sensing imagery. ISPRS Journal of Photogrammetry and Remote Sensing, 138: 281–294
  32. 32.
    Zhou D G, Gao C and Guo Y C. 2014. Adaptive simplified PCNN parameter setting for image segmentation. Acta Automatica Sinica, 40(6): 1191–1197

Lire l'article complet

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website