Précédent|Prochain

Semantic segmentation of remote sensing image based on deep fusion networks and conditional random field

  • role： First author第一作者
  • Affiliation：

    Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences, Beijing 100094, China

    University of Chinese Academy of Sciences, Beijing 100049, China

  • Email：xiaocj@radi.ac.cn
  • Introduction：肖春姣，1994年生,女,硕士研究生, 研究方向为遥感图像处理、深度学习。E-mail: xiaocj@radi.ac.cn
XIAO Chunjiao12,  
  • role： Corresponding author通信作者
  • Affiliation：

    Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences, Beijing 100094, China

  • Email：liyu@radi.ac.cn
  • Introduction：李宇，1986年生，女，助理研究员，研究方向为遥感图像处理。E-mail：liyu@radi.ac.cn
LI Yu1*,  
  • Affiliation：

    Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences, Beijing 100094, China

ZHANG Hongqun1,  
  • Affiliation：

    Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences, Beijing 100094, China

CHEN Jun1

résumé

Image semantic segmentation refers to segmenting an image into several groups of pixel regions with different specific semantic meanings and identifying the categories of each region. In recent years, the common semantic segmentation methods that are based on Convolutional Neural Networks (CNN) have realised the pixel-to-pixel image semantic segmentation. They can avoid the problems of artificial design and selection of features in traditional image semantic segmentation methods. As a result of the pooling operation and lack of context information, the detailed information of images is neglected, the precision of the final image semantic segmentation result is low and the segmentation edge is inaccurate. Therefore, this study proposes a semantic segmentation method for remote sensing image on the basis of Deep Fusion Networks (DFN) combined with a conditional random field model.The method initially builds a DFN model in a Fully Convolutional Network (FCN) framework with a deconvolutional fusion structure. On the one hand, the multiscale features can be extracted through the deep networks, which can avoid the artificial design and selection of features to improve the generalisation ability of the model. On the other hand, the multiscale information is used in the model with the help of the deconvolutional fusion structure. The processing accuracy of the model is also improved by fusing the shallow detail information and deep semantic information. Fundamentally, the fully connected conditional random field is introduced to supplement the spatial context information towards precisely locating the boundary and obtaining final semantic segmentation results.From this study, we can draw the following(1)With the increase in the depth of the fusion layer, detailed information becomes abundant, the semantic segmentation results become refined and the edge contour becomes close to the label image;(2) The fully connected conditional random field model synthesises the global and local information of the remote sensing image and further improves the efficiency and accuracy of the final semantic segmentation results.

mots-clés

remote sensing image semantic segmentation;fully convolutional networks;conditional random field;fusion structure;deconvolution

References

  1. 1.
    Arbeláez P, Hariharan B, Gu C H, Gupta S, Bourdev L and Malik J. 2012. Semantic segmentation using regions and parts//Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, RI, USA: IEEE: 3378-3385 [DOI: 10.1109/CVPR.2012.6248077]
  2. 2.
    Chang L, Deng X M, Zhou M Q, Wu Z K, Yuan Y, Yang S and Wang H A. 2016. Convolutional neural networks in image understanding. Acta Automatica Sinica, 42(9): 1300-1312
  3. 3.
    Chen L C, Papandreou G, Kokkinos L, Murphy K and Yuille A L. 2018. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4): 834-848
  4. 4.
    Cheng G, Han J W and Lu X Q. 2017. Remote sensing image scene classification: benchmark and state of the art. Proceedings of the IEEE, 105(10): 1865-1883
  5. 5.
    Krähenbühl P and Koltun V. 2012. Efficient inference in fully connected CRFs with Gaussian edge potentials. Advances in Neural Information Processing Systems 24: 109-117
  6. 6.
    Krizhevsky A, Sutskever I and Hinton G E. 2012. ImageNet classification with deep convolutional neural networks//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, Nevada Curran Associates Inc: 1097-1105
  7. 7.
    Kumar S and Hebert M. 2003. Discriminative random fields: a discriminative framework for contextual interaction in classification//Proceedings of the 9th IEEE International Conference on Computer Vision. Nice, France: IEEE: 1150-1157
  8. 8.
    Lafferty J D, McCallum A and Pereira F C N. 2001. Conditional random fields: probabilistic models for segmenting and labeling sequence data//Proceedings of the Eighteenth International Conference on Machine Learning. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc: 282-289
  9. 9.
    Li S Z. 2009. Markov Random Field Modeling in Image Analysis. 3rd ed. London: Springer
  10. 10.
    Li Z C, Ma J W, Zhang R and Li L W. 2011. Classifying hyperspectral data using support vector machine conditional random field. Geomatics and Information Science of Wuhan University, 36(3): 306-310
  11. 11.
    Liu W, Zhou T, Yuan H and Zhao H. 2017. 3D spatial layout understanding from image based on multiple CRFs. Acta Electronica Sinica, 45(2): 328-336
  12. 12.
    Long J, Shelhamer E and Darrell T. 2015. Fully convolutional networks for semantic segmentation//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA, USA: IEEE: 3431-3440
  13. 13.
    Lü Y B, Zhao J W and Cao F L. 2017. Image denoising algorithm based on composite convolutional neural network. Pattern Recognition and Artificial Intelligence, 30(2): 97-105
  14. 14.
    Montoya-Zegarra J A, Wegner J D, Ladický L and Schindler K. 2015. Semantic segmentation of aerial images in urban areas with class-specific higher-order cliques. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, II-3/W4: 127-133 [DOI:
  15. 15.
    Shotton J, Johnson M and Cipolla R. 2008. Semantic texton forests for image categorization and segmentation//Proceedings of 2008 IEEE Conference on Computer Vision and Pattern Recognition. Anchorage, AK, USA: IEEE: 1-8
  16. 16.
    Smeulders A W M, Worring M, Santini S, Gupta A and Jain R. 2000. Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(12): 1349-1380
  17. 17.
    Sun W W and Wang R S. 2018. Fully convolutional networks for semantic segmentation of very high resolution remotely sensed images combined with DSM. IEEE Geoscience and Remote Sensing Letters, 15(3): 474-478
  18. 18.
    Tang Z C, Zhang K J, Li C, Sun S Q, Huang Q and Zhang S Y. 2017. Motor imagery classification based on deep convolutional neural network and its application in exoskeleton controlled by EGG. Chinese Journal of Computers, 40(6): 1367-1378
  19. 19.
    Volpi M and Ferrari V. 2015. Semantic segmentation of urban scenes by learning local class interactions//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Boston, MA, USA: IEEE: 1-9
  20. 20.
    Wei X, Guo Y J, Gao X, Yan M L and Sun X. 2017. A new semantic segmentation model for remote sensing images//Proceedings of 2017 IEEE International Geoscience and Remote Sensing Symposium. Fort Worth, TX, USA: IEEE: 1776-1779
  21. 21.
    Wei Y C and Zhao Y. 2016. A review on image semantic segmentation based on DCNN. Journal of Beijing Jiaotong University, 40(4): 82-91
  22. 22.
    Xiao J S, Liu E Y, Zhu L and Lei J F. 2017. Improved image super-resolution algorithm based on convolutional neural network. Acta Optica Sinica, 37(3): 0318011
  23. 23.
    Yang W, Triggs B, Dai D X and Xia G S. 2010. Scene Segmentation with low-dimensional semantic representations and conditional random fields. EURASIP Journal on Advances in Signal Processing,2010: 196036
  24. 24.
    Yang Y and Newsam S. 2010. Bag-of-visual-words and spatial extensions for land-use classification//Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems. San Jose, California: ACM: 270-279
  25. 25.
    Zhang H Q, Liu X Y, Yang S and Li Y. 2017. Retrieval of remote sensing images based on semisupervised deep learning. Journal of Remote Sensing, 21(3): 406-414
  26. 26.
    Zhang K, Hei B Q, Zhou Z and Li S Y. 2018. CNN with coefficient of variation-based dimensionality reduction for hyperspectral remote sensing images classification. Journal of Remote Sensing, 22(1): 87-96
  27. 27.
    Zhang R, Li G Y, Li M L and Wang L. 2018. Fusion of images and point clouds for the semantic segmentation of large-scale 3D scenes based on deep learning. ISPRS Journal of Photogrammetry and Remote Sensing, 143: 85-96
  28. 28.
    Zhao J, Zhong Y F and Zhang L P. 2015. Detail-preserving smoothing classifier based on conditional random fields for high spatial resolution remote sensing imagery. IEEE Transactions on Geoscience and Remote Sensing, 53(5): 2440-2452
  29. 29.
    Zheng S, Jayasumana S, Romera-Paredes B, Vineet V, Su Z Z, Du D L, Huang C and Torr P H S. 2015. Conditional random fields as recurrent neural networks//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago, Chile: IEEE: 1529-1537

Lire l'article complet

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website