предыдущий|следующий

High resolution remote sensing image segmentation based on dual-modal efficient feature learning

  • role： First author第一作者
  • Affiliation：

    School of Electronic and Information Engineering, Wuxi University, Wuxi 214105, China

    School of Electronic and Information Engineering, Nanjing University of Information Science and Technology, Nanjing 210044, China

  • Email：yorkzhang@nuist.edu.cn
  • Introduction：张银胜，研究方向为遥感图像处理、深度学习。E-mail： yorkzhang@nuist.edu.cn
ZHANG Yinsheng12,  
  • role： Corresponding author通信作者
  • Affiliation：

    School of Electronic and Information Engineering, Nanjing University of Information Science and Technology, Nanjing 210044, China

  • Email：20211249643@nuist.edu.cn
  • Introduction：吉茹，研究方向为遥感图像分割、深度学习。E-mail： 20211249643@nuist.edu.cn
JI Ru2*,  
  • Affiliation：

    School of Electronic and Information Engineering, Nanjing University of Information Science and Technology, Nanjing 210044, China

TONG Junyi2,  
  • Affiliation：

    School of Electronic and Information Engineering, Nanjing University of Information Science and Technology, Nanjing 210044, China

YANG Yulong2,  
  • Affiliation：

    School of Electronic and Information Engineering, Wuxi University, Wuxi 214105, China

HU Yuxiang1,  
  • Affiliation：

    School of Electronic and Information Engineering, Wuxi University, Wuxi 214105, China

    School of Electronic and Information Engineering, Nanjing University of Information Science and Technology, Nanjing 210044, China

SHAN Huilin12

реферат

With the rapid development of spatial technology, the resolution of remote sensing images gradually improves. The detailed information and spatial information contained in remote-sensing images are also richer. The ensuing problems are that the difference between various categories becomes and the difference between the same categories becomes larger, i.e., the phenomenon of the same spectrum of foreign objects and the different spectrum of the same objects is serious. However, the existing dual-modal segmentation methods do not extract the dual-modal feature information of remote-sensing images separately, and the fusion features are insufficient. The details of upsampling recovery are also insufficient, resulting in the inability to accurately and efficiently learn remote-sensing image information, thereby resulting in segmentation errors, edge blur, and other problems.This study proposes a high resolution remote-sensing image segmentation based on dual-modal efficient feature learning. The algorithm designs appropriate encoders for different modal remote sensing images, efficiently extracts dual-modal features, and reduces the differences between different path features through interactive reinforcement modules. Then, the dual-modal feature aggregation module and the deep feature-extraction module are proposed to further fuse and extract the dual-modal features. As a result, the network can fully learn the complementary information of the dual-modal. Finally, a multi-layer feature upsampling module is proposed, which uses high-level features with rich semantic information to weight the low-level features with rich detail information. Gradual upsampling is then conducted to achieve efficient feature recovery and improve segmentation performance.In this paper, experiments on the Potsdam and Vaihingen datasets demonstrate that the overall accuracy reaches 94.52% and 90.45%, respectively. Experimental results show that the segmentation effect of the proposed algorithm is better than that of existing algorithms. The proposed algorithm can efficiently extract and fuse the multi-modal complementary features of high resolution remote-sensing images and improve the segmentation accuracy of remote-sensing images.This study proposes a high-resolution remote-sensing image segmentation based on dual-modal efficient feature learning. Experiments on the ISPRS Potsdam and Vaihingen datasets show that the proposed model is more suitable for segmenting low vegetation and trees, buildings, and roads with very similar spectral features. It can also achieve the accurate segmentation of small targets, such as cars. However, the complexity of the model needs to be further reduced, and much room for improvement in accuracy remains. In the future, a better segmentation network will be designed to fuse more than two modal features and thus obtain more feature information to achieve more accurate remote sensing image segmentation.

ключеви́че слова́

remote sensing image segmentation;efficient feature extraction;integration;dual-modal feature aggregation;deep feature extraction;multilayer feature upsampling

References

  1. 1.
    Ahmed O S, Shemrock A, Chabot D, Dillon C, Williams G, Wasson R and Franklin S E. 2017. Hierarchical land cover and vegetation classification using multispectral data acquired from an unmanned aerial vehicle. International Journal of Remote Sensing, 38(8/10): 2037-2052
  2. 2.
    Audebert N, Le Saux B and Lefèvre S. 2018. Beyond RGB: very high resolution urban remote sensing with multimodal deep networks. ISPRS Journal of Photogrammetry and Remote Sensing, 140: 20-32
  3. 3.
    Chen B Y, Xia M and Huang J Q. 2021a. MFANet: a multi-level feature aggregation network for semantic segmentation of land cover. Remote Sensing, 13(4): 731
  4. 4.
    Chen L C, Papandreou G, Kokkinos I, Murphy K and Yuille A L. 2018. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4): 834-848
  5. 5.
    Chen S T, Wu C Q, Mukherjee M and Zheng Y J. 2021b. HA-MPPNet: height aware-multi path parallel network for high spatial resolution remote sensing image semantic segmentation. ISPRS International Journal of Geo-Information, 10(10): 672
  6. 6.
    Farabet C, Couprie C, Najman L and LeCun Y. 2013. Learning hierarchical features for scene labeling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8): 1915-1929
  7. 7.
    Gerard F, Petit S, Smith G, Thomson A, Brown N, Manchester S, Wadsworth R, Bugar G, Halada L, Bezák P, Boltiziar M, De Badts E, Halabuk A, Mojses M, Petrovic F, Gregor M, Hazeu G, Mücher C A, Wachowicz M, Huitu H, Tuominen S, Köhler R, Olschofsky K, Ziese H, Kolar J, Sustera J, Luque S, Pino J, Pons X, Roda F, Roscher M and Feranec J. 2010. Land cover change in Europe between 1950 and 2000 determined employing aerial photography. Progress in Physical Geography: Earth and Environment, 34(2): 183-205
  8. 8.
    Hazirbas C, Ma L N, Domokos C and Cremers D. 2016. FuseNet: incorporating depth into semantic segmentation via fusion-based CNN architecture//Proceedings of the 13th Asian Conference on Computer Vision. Taipei, China: Springer: 213-228
  9. 9.
    He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE: 770-778
  10. 10.
    Hu J, Shen L, Albanie S, Sun G and Wu E H. 2020. Squeeze-and-excitation networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(8): 2011-2023
  11. 11.
    Li H C, Xiong P F, An J and Wang L X. 2018. Pyramid attention network for semantic segmentation. arXiv preprint arXiv: 1805.10180
  12. 12.
    Marcos D, Volpi M, Kellenberger B and Tuia D. 2018. Land cover mapping at very high resolution with rotation equivariant CNNs: towards small yet accurate models. ISPRS Journal of Photogrammetry and Remote Sensing, 145: 96-107
  13. 13.
    Marmanis D, Wegner J D, Galliani S, Schindler K, Datcu M and Stilla U. 2016. Semantic segmentation of aerial images with an ensemble of CNNs. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, III-3: 473-480
  14. 14.
    Mnih V and Hinton G E. 2010. Learning to detect roads in high-resolution aerial images//Proceedings of the 11th European Conference on Computer Vision. Heraklion: Springer: 210-223
  15. 15.
    Paisitkriangkrai S, Sherrah J, Janney P and Van-Den Hengel A. 2015. Effective semantic pixel labelling with convolutional networks and Conditional Random Fields//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Boston: IEEE: 36-43
  16. 16.
    Ronneberger O, Fischer P and Brox T. 2015. U-net: convolutional networks for biomedical image segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich: Springer: 234-241
  17. 17.
    Shelhamer E, Long J and Darrell T. 2017. Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4): 640-651
  18. 18.
    Sun H Q, Pan C, He L M and Xu Z J. 2022. Remote sensing image semantic segmentation network based on multimodal feature fusion. Computer Engineering and Applications, 58(24): 256-264
  19. 19.
    Sun J X and Li Y J. 2021. Multi-feature fusion network for road scene semantic segmentation. Computers and Electrical Engineering, 92: 107155
  20. 20.
    Wang S L. 2005. The magic eye of modern war-military application of satellite remote sensing image. China Surveying and Mapping, (1): 34-37
  21. 21.
    Wang X L, Girshick R, Gupta A and He K M. 2018. Non-local neural networks//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 7794-7803
  22. 22.
    Wu G M, Shao X W, Guo Z L, Chen Q, Yuan W, Shi X D, Xu Y W and Shibasaki R. 2018. Automatic building segmentation of aerial imagery using multi-constraint fully convolutional networks. Remote Sensing, 10(3): 407
  23. 23.
    Xu Y Y, Wu L, Xie Z and Chen Z L. 2018. Building extraction in very high resolution remote sensing imagery using deep learning and guided filters. Remote Sensing, 10(1): 144
  24. 24.
    Yang X, Li S S, Chen Z C, Chanussot J, Jia X P, Zhang B, Li B P and Chen P. 2021. An attention-fused network for semantic segmentation of very-high-resolution remote sensing imagery. ISPRS Journal of Photogrammetry and Remote Sensing, 177: 238-262
  25. 25.
    Zhang W K, Liu W J, Sun X, Xu G L and Fu K. 2022. Multi-source features adaptation fusion network for semantic segmentation in high-resolution remote sensing images. Journal of Image and Graphics, 27(8): 2516-2526
  26. 26.
    Zhang X J and Wang X L. 2020. Image segmentation models of remote sensing using full residual connection and multiscale feature fusion. Journal of Remote Sensing (Chinese), 24(9): 1120-1133
  27. 27.
    Zhou W Q, Huang G L and Cadenasso M L. 2011. Does spatial configuration matter? Understanding the effects of land cover pattern on land surface temperature in urban landscapes. Landscape and Urban Planning, 102(1): 54-63

Читать полностью

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website