سابِق|تال

Edge-perception enhanced segmentation method for high-resolution remote sensing image

  • role： First author第一作者
  • Affiliation：

    Center of Hyperspectral Imaging in Remote Sensing of Dalian Maritime University, Dalian 116026, China

  • Email：yuchunyan1997@126.com
  • Introduction：于纯妍，研究方向为高光谱遥感智能解译方法。E-mail：yuchunyan1997@126.com
YU Chunyan1,  
  • Affiliation：

    Center of Hyperspectral Imaging in Remote Sensing of Dalian Maritime University, Dalian 116026, China

LI Donglin1,  
  • role： Corresponding author通信作者
  • Affiliation：

    Center of Hyperspectral Imaging in Remote Sensing of Dalian Maritime University, Dalian 116026, China

  • Email：smping@163.com
  • Introduction：宋梅萍，研究方向为高光谱遥感智能解译方法。E-mail：smping@163.com
SONG Meiping1*,  
  • Affiliation：

    Center of Hyperspectral Imaging in Remote Sensing of Dalian Maritime University, Dalian 116026, China

YU Haoyang1,  
  • Affiliation：

    Center of Hyperspectral Imaging in Remote Sensing of Dalian Maritime University, Dalian 116026, China

    Remote Sensing Signal and Image Processing Laboratory, Department of Computer Science and Electrical Engineering, University of Maryland, Baltimore County, Baltimore, MD 21250, USA

Chein-I Chang12

ملخص

The semantic segmentation method for high-resolution remote sensing images that is based on Deep Convolutional Neural Network (DCNN) has achieved remarkable progress, but problems still exist in the extraction and expression of the edge features of segmented objects. As a result, the edge segmentation effect of occluded and small target objects is unsatisfactory, which affects the overall accuracy of the semantic segmentation method. To solve these problems, this study proposes an edge-aware enhanced semantic segmentation method for high-resolution remote sensing images.First, we utilize the Transformer-DCNN collaborative feature extraction mechanism to extract the global self-attention features and spatial context information of remote sensing images. In this way, the proposed model makes full use of the advantages of Transformer and DCNN to extract global context information and spatial local context information, respectively. The proposed model extracts a highly accurate ground object semantic feature expression and designs a simple but effective feature extraction fusion module to fuse the features extracted by DCNN and Transformer. Second, we construct an edge-aware enhancement module composed of an edge-enhanced decoder and an uncertain point-enhanced decoder. This module enhances the edge information processing ability of the remote sensing image semantic segmentation model from two perspectives, namely, uncertain points and entity edges. Last, the semantic segmentation decoder effectively employs the feature codes containing edge information to improve the accuracy and completeness of segmented object edge prediction, which guarantees that the semantic segmentation effect of remote sensing images is improved overall.Comparative experiments are conducted on two public datasets, namely, Potsdam and Vaihingen. In comparison with the classical Unet++ network, the proposed method demonstrates improvements of 3.44% and 4.01% in mean intersection over union for the two datasets. The average F1 score and overall accuracy exhibit varying degrees of improvement. Furthermore, compared with the Transformer-based TransUNet model, the proposed method achieves better results.Enhancing the feature extraction of edge information in remote sensing objects leads to remarkable improvements in the edge and overall semantic segmentation accuracies of high-resolution remote sensing images. The proposed edge perception enhancement module improves the model’s ability to process edge information from two perspectives, namely, uncertain points and entity edges, thus effectively enhancing the edge segmentation accuracy for complex terrain objects. The results of commonly used evaluation indicators demonstrate the effectiveness and robustness of the developed model.

مفهوم

remote sensing image;semantic segmentation;Edge perception;feature extraction;Encoder;Decoder

References

  1. 1.
    Bai H W, Cheng J, Huang X, Liu S Y and Deng C J. 2022. HCANet: a hierarchical context aggregation network for semantic segmentation of high-resolution remote sensing images. IEEE Geoscience and Remote Sensing Letters, 19: 6002105
  2. 2.
    Bokhovkin A and Burnaev E. 2019. Boundary loss for remote sensing imagery semantic segmentation//16th International Symposium on Neural Networks. Moscow: Springer: 388-401
  3. 3.
    Chen J N, Lu Y Y, Yu Q H, Luo X D, Adeli E, Wang Y, Lu L, Yuille A L and Zhou Y Y. 2021. TransUNet: transformers make strong encoders for medical image segmentation. arXiv: 2102.04306
  4. 4.
    Chen L C, Zhu Y K, Papandreou G, Schroff F and Adam H. 2018b. Encoder-decoder with atrous separable convolution for semantic image segmentation//15th European Conference on Computer Vision. Munich: Springer: 833-851
  5. 5.
    Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X H, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S, Uszkoreit J and Houlsby N. 2021. An image is worth 16x16 words: transformers for image recognition at scale//9th International Conference on Learning Representations. [s.l.]: OpenReview.net
  6. 6.
    Han K, Xiao A, Wu E H, Guo J Y, Xu C J and Wang Y H. 2021. Transformer in transformer//Proceedings of the 35th International Conference on Neural Information Processing Systems. [s.l.]: Curran Associates Inc.: 1217
  7. 7.
    He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE: 770-778
  8. 8.
    Kirillov A, Wu Y X, He K M and Girshick R. 2020. PointRend: image segmentation as rendering//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE: 9796-9805
  9. 9.
    Li A J, Jiao L C, Zhu H, Li L L and Liu F. 2022. Multitask semantic boundary awareness network for remote sensing image segmentation. IEEE Transactions on Geoscience and Remote Sensing, 60: 5400314
  10. 10.
    Liu Z, Lin Y T, Cao Y, Hu H, Wei Y X, Zhang Z, Lin S and Guo B N. 2021. Swin transformer: hierarchical vision transformer using shifted windows//2021 IEEE/CVF International Conference on Computer Vision. Montreal: IEEE: 9992-10002
  11. 11.
    Luo X, Tong X H and Pan H Y. 2021. Integrating multiresolution and multitemporal sentinel-2 imagery for land-cover mapping in the Xiongan New Area, China. IEEE Transactions on Geoscience and Remote Sensing, 59(2): 1029-1040
  12. 12.
    Luo Z, Li M and Zhang D Z. 2022. Building detection based on a boundary-regulated network and watershed segmentation. National Remote Sensing Bulletin, 26(7): 1459-1468
  13. 13.
    Milletari F, Navab N and Ahmadi S A. 2016. V-Net: fully convolutional neural networks for volumetric medical image segmentation//2016 Fourth International Conference on 3D Vision. Stanford: IEEE: 565-571
  14. 14.
    Min L, Gao K, Li W, Wang H, Li T, Wu Q and Jiao J C. 2020. A review of the optical remote sensing image segmentation technology. Spacecraft Recovery and Remote Sensing, 41(6): 1-13
  15. 15.
    Rottensteiner F, Sohn G, Jung J, Gerke M, Baillard C, Benitez S and Breitkopf U. 2012. The ISPRS benchmark on urban object classification and 3D building reconstruction. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, I-3: 293-298
  16. 16.
    Ronneberger O, Fischer P and Brox T. 2015. U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich : Springer International Publishing: 234-241
  17. 17.
    Shelhamer E, Long J and Darrell T. 2017. Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4): 640-651
  18. 18.
    Simonyan K and Zisserman A. 2015. Very deep convolutional networks for large-scale image recognition//Proceedings of the 3rd International Conference on Learning Representations. San Diego: [s.n.]
  19. 19.
    Srinivas A, Lin T Y, Parmar N, Shlens J, Abbeel P and Vaswani A. 2021. Bottleneck transformers for visual recognition//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE: 16514-16524
  20. 20.
    Sun X, Shi A J, Huang H and Mayer H. 2020. BAS4Net: boundary-aware semi-supervised semantic segmentation network for very high resolution remote sensing images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13: 5398-5413
  21. 21.
    Sun X D, Xia M and Dai T F. 2022. Controllable fused semantic segmentation with adaptive edge loss for remote sensing parsing. Remote Sensing, 14(1): 207
  22. 22.
    Takikawa T, Acuna D, Jampani V and Fidler S. 2020. Gated-SCNN: gated shape CNNs for semantic segmentation//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE: 5228-5237
  23. 23.
    Tan M X and Le Q V. 2019. EfficientNet: rethinking model scaling for convolutional neural networks//Proceedings of the 36th International Conference on Machine Learning. Long Beach: PMLR: 6105-6114
  24. 24.
    Wang B B, Yang H S, Wang J F and Xu C H. 2015. Urban land information extraction based on high-resolution remote sensing images. Geomatics and Spatial Information Technology, 38(4): 65-67
  25. 25.
    Wu Q Q, Wang S, Wang B and Wu Y L. 2022. Road extraction method of high-resolution remote sensing image on the basis of the spatial information perception semantic segmentation model. National Remote Sensing Bulletin, 26(9): 1872-1885
  26. 26.
    Xie E Z, Wang W H, Yu Z D, Anandkumar A, Alvarez J M and Luo P. 2021. SegFormer: simple and efficient design for semantic segmentation with transformers//Proceedings of the 35th International Conference on Neural Information Processing Systems. [s.l.]: Curran Associates Inc.: 924
  27. 27.
    Zeng W X, Ma Y, Ding Y, Zhang S Q and Li W G. 2021. Survey of image semantic segmentation algorithms based on deep learning. Modern Computer, 27(21): 115-122
  28. 28.
    Zhao H S, Shi J P, Qi X J, Wang X G and Jia J Y. 2017. Pyramid scene parsing network//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 6230-6239
  29. 29.
    Zheng S X, Lu J C, Zhao H S, Zhu X T, Luo Z K, Wang Y B, Fu Y W, Feng J F, Xiang T, Torr P H S and Zhang L. 2021. Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE: 6877-6886
  30. 30.
    Zhou Z W, Siddiquee M M R, Tajbakhsh N and Liang J M. 2020. UNet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Transactions on Medical Imaging, 39(6): 1856-1867

قراءة النص الكامل

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website