Früher|Nächster

Weakly supervised scale adaptation data augmentation for scene classification of high-resolution remote sensing images

  • role： First author第一作者
  • Affiliation：

    School of Geography and Information Engineering, China University of Geosciences (Wuhan), Wuhan 430074, China

  • Email：limingwang@cug.edu.cn
  • Introduction：王梨名，研究方向为高分遥感影像解译。E-mail： limingwang@cug.edu.cn
WANG Liming1,  
  • Affiliation：

    School of Geography and Information Engineering, China University of Geosciences (Wuhan), Wuhan 430074, China

    National Engineering Research Center of Geographic Information System, Wuhan 430074, China

QI Kunlun12,  
  • role： Corresponding author通信作者
  • Affiliation：

    School of Geography and Information Engineering, China University of Geosciences (Wuhan), Wuhan 430074, China

    National Engineering Research Center of Geographic Information System, Wuhan 430074, China

  • Email：yangchao@cug.edu.cn
  • Introduction：杨超，研究方向为遥感影像解译和时空大数据。E-mail： yangchao@cug.edu.cn
YANG Chao12*,  
  • Affiliation：

    State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China

WU Huayi3

Resümee

Scene classification of remote sensing images aims to assign a meaningful label to a given image. In recent years, Convolutional Neural Networks (CNNs)-based methods make a breakthrough and substantially outperform traditional methods in scene classification tasks of remote sensing images. However, obtaining features under different scales in remote sensing images is difficult due to the fixed receptive field of CNNs. This complexity seriously affects the performance of CNNs in scene classification of remote sensing images. This study proposes a method to learn the optimal scales for different scene image instances in a weakly supervised manner.A Weakly Supervised Scale Adaptive Data Augmentation Network (WSADAN) is proposed to capture feature information at different scales of remote sensing scenes, and a scale generation module and a scale fusion module are designed to improve the robustness. The scale generation module learns the optimal scale parameters based on the CNN features of the original image. The scale fusion module filters the CNN features of images with original and optimal scales to remove the noise and then deeply fuses them to exploit the correlation between features at different scales. The deeply fused multi-scale features are input into a fully connected layer to predict categories of scene images.The effectiveness of the scale generation and scale fusion modules is verified by ablation experiments. The accuracy of WSADANSGM compared with the baseline improves by 0.94% and 0.89% for the 20% and 50% training data ratios of RSSCN7 dataset, 1.27% and 0.87% for the 20% and 50% training data ratios of AID dataset, and 1.09% and 0.71% for the 10% and 20% training data ratios of NWPU dataset, respectively. Compared with WSADANSGM, WSADANSGM+SFM improves by 1.65% and 1.32% for the RSSCN7 dataset at 20% and 50% training data ratios, 1.65% and 1.26% for the AID dataset at 20% and 50% training data ratios, and 1.75% and 1.42% for the NWPU dataset at 10% and 20% training data ratios, respectively. In the experiment for scene scale change analysis, the classification accuracy of our method is higher than the baseline at any scale of image, which proves that our method can learn certain image scale information and has strong scale adaptation ability. We use three datasets for remote sensing scene classification, namely, RSSCN7, AID, and NWPU, for the experiments. On the RSSCN7 dataset, the overall accuracies are 91.65% and 94.07% with the training ratios of 20% and 50% for WSADAN-VGG16. For WSADAN-ResNet50, the corresponding accuracies are 92.69% and 94.82%. On the AID dataset, the overall accuracies are 92.78% and 95.18% with the training ratios of 20% and 50% for WSADAN-VGG16. For WSADAN-ResNet50, the corresponding accuracies are 93.73% and 95.88%. On the NWPU dataset, the overall accuracies are 87.01% and 90.44% with the training ratios of 10% and 20% for WSADAN-VGG16. For WSADAN-ResNet50, the corresponding accuracies are 90.71% and 92.63%.The proposed method can learn CNN features at a wider range of scales without manual multi-scale selection for different datasets. The performance of the proposed method is better than that of traditional CNNs, especially for the scene categories containing objects with large-scale variations.

Schlüsselwort

remote sensing;scene classification;deep learning;convolutional neural networks;weakly supervision;multi-scale;data augmentation

References

  1. 1.
    Alhichri H, Alajlan N, Bazi Y and Rabczuk T. 2018. Multi-scale convolutional neural network for remote sensing scene classification//Proceedings of the 2018 IEEE International Conference on Electro/Information Technology (EIT). Rochester: IEEE: 1-5
  2. 2.
    Cao R, Fang L Y, Lu T and He N J. 2021. Self-attention-based deep feature fusion for remote sensing scene classification. IEEE Geoscience and Remote Sensing Letters, 18(1): 43-47
  3. 3.
    Chen L J, Yang W, Xu K and Xu T. 2011. Evaluation of local features for scene classification using VHR satellite images//Proceedings of 2011 Joint Urban Remote Sensing Event. Munich: IEEE: 385-388
  4. 4.
    Chen W T, Li X J, He H X and Wang L Z. 2018. Assessing different feature sets’ effects on land cover classification in complex surface-mined landscapes by ZiYuan-3 satellite imagery. Remote Sensing, 10(1): 23
  5. 5.
    Chen Y Y, Ming D P and Lv X W. 2019. Superpixel based land cover classification of VHR satellite image combining multi-scale CNN and scale parameter estimation. Earth Science Informatics, 12(3): 341-363
  6. 6.
    Cheng G, Guo L, Zhao T Y, Han J W, Li H H and Fang J. 2013. Automatic landslide detection from remote-sensing imagery using a scene classification method based on BoVW and pLSA. International Journal of Remote Sensing, 34(1): 45-59
  7. 7.
    Cheng G, Han J W and Lu X Q. 2017. Remote sensing image scene classification: benchmark and state of the art. Proceedings of the IEEE, 105(10): 1865-1883
  8. 8.
    Gao Y, Shi J, Li J and Wang R Y. 2020. Remote sensing scene classification with dual attention-aware network//Proceedings of the IEEE 5th International Conference on Image, Vision and Computing (ICIVC). Beijing: IEEE: 171-175
  9. 9.
    Gu Y T, Wang Y T and Li Y S. 2019. A survey on deep learning-driven remote sensing image scene understanding: scene classification, scene retrieval and scene-guided object detection. Applied Sciences, 9(10): 2110
  10. 10.
    Han X B, Zhong Y F, Cao L Q and Zhang L P. 2017. Pre-trained AlexNet architecture with pyramid pooling and supervision for high spatial resolution remote sensing image scene classification. Remote Sensing, 9(8): 848
  11. 11.
    He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE: 770-778
  12. 12.
    He N J, Fang L Y, Li S T, Plaza J and Plaza A. 2020. Skip-connected covariance network for remote sensing scene classification. IEEE Transactions on Neural Networks and Learning Systems, 31(5): 1461-1474
  13. 13.
    Kingma D P and Ba J. 2014. Adam: a method for stochastic optimization//Proceedings of the 3rd International Conference on Learning Representations. San Diego: [s.n.]
  14. 14.
    Längkvist M, Kiselev A, Alirezaie M and Loutfi A. 2016. Classification and segmentation of satellite orthoimagery using convolutional neural networks. Remote Sensing, 8(4): 329
  15. 15.
    Liu Q S, Hang R L, Song H H, Zhu F P, Plaza J and Plaza A. 2016. Adaptive deep pyramid matching for remote sensing scene classification. arXiv:1611.03589
  16. 16.
    Liu Y S, Liu Y B and Ding L W. 2018. Scene classification based on two-stage deep feature fusion. IEEE Geoscience and Remote Sensing Letters, 15(2): 183-186
  17. 17.
    Luo B, Jiang S J and Zhang L P. 2013. Indexing of remote sensing images with different resolutions by multiple features. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 6(4): 1899-1912
  18. 18.
    Lv Z H, Li X M, Zhang B Y, Wang W X, Zhu Y Y, Hu J X and Feng S Z. 2016. Managing big city information based on WebVRGIS. IEEE Access, 4: 407-415
  19. 19.
    Ma X Y, Wang L M, Qi K L and Zheng G Z. 2021. Remote sensing image scene classification method based on multi-scale cyclic attention network. Earth Science, 46(10): 3740-3752
  20. 20.
    Martha T R, Kerle N, Van Westen C J, Jetten V and Kumar K V. 2011. Segment optimization and data-driven thresholding for knowledge-based landslide detection by object-based image analysis. IEEE Transactions on Geoscience and Remote Sensing, 49(12): 4928-4943
  21. 21.
    Qi K L, Yang C, Guan Q F, Wu H Y and Gong J Y. 2017. A multiscale deeply described correlatons-based model for land-use scene classification. Remote Sensing, 9(9): 917
  22. 22.
    Qi K L, Yang C, Hu C L, Zhai H, Guan Q F and Shen S Y. 2021. A multi-level improved circle pooling for scene classification of high-resolution remote sensing imagery. Neurocomputing, 462: 506-522
  23. 23.
    Qian X L, Li J, Cheng G, Yao X W, Zhao S N, Chen Y B and Jiang L Y. 2018. Evaluation of the effect of feature extraction strategy on the performance of high-resolution remote sensing image scene classification. Journal of Remote Sensing, 22(5): 758-776
  24. 24.
    Shi H H, Xu Y N, Teng W X and Wang N. 2021. Scene classification of high-resolution remote sensing imagery based on deep transfer deformable convolutional neural networks. Acta Geodaetica et Cartographica Sinica. 50(05): 652-663
  25. 25.
    Simonyan K and Zisserman A. 2015. Very deep convolutional networks for large-scale image recognition//Proceedings of the 3rd International Conference on Learning Representations. San Diego: IEEE: 7-9
  26. 26.
    Sun X L, Zhu Q Q and Qin Q Q. 2021. A multi-level convolution pyramid semantic fusion framework for high-resolution remote sensing image scene classification and annotation. IEEE Access, 9: 18195-18208
  27. 27.
    Wu H, Liu B Z, Su W H, Zhang W C and Sun J G. 2016. Deep filter banks for land-use scene classification. IEEE Geoscience and Remote Sensing Letters, 13(12): 1895-1899
  28. 28.
    Xia G S, Hu J W, Hu F, Shi B G, Bai X, Zhong Y F, Zhang L P and Lu X Q. 2017. AID: a benchmark data set for performance evaluation of aerial scene classification. IEEE Transactions on Geoscience and Remote Sensing, 55(7): 3965-3981
  29. 29.
    Yang Y and Newsam S. 2008. Comparing SIFT descriptors and gabor texture features for classification of remote sensed imagery//Proceedings of the 15th IEEE International Conference on Image Processing. San Diego: IEEE: 1852-1855
  30. 30.
    Yang Y and Newsam S. 2010. Bag-of-visual-words and spatial extensions for land-use classification//Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems. San Jose: ACM: 270-279
  31. 31.
    Yao Y Q, Cheng G, Xie X X and Han J W. 2021. Optical remote sensing image object detection based on multi-resolution feature fusion. National Remote Sensing Bulletin, 25(5): 1124-1137
  32. 32.
    Yu D H, Zhang B M, Zhao C, Guo H T and Lu J. 2020. Scene classification of remote sensing image using ensemble convolutional neural network. Journal of Remote Sensing, 24(6): 717-727
  33. 33.
    Yu Y L, Li X Z and Liu F X. 2020. Attention GANs: unsupervised deep feature learning for aerial scene classification. IEEE Transactions on Geoscience and Remote Sensing, 58(1): 519-531
  34. 34.
    Zan L Y, Li B P, Lu K X, Chen Z C and Zhang B. 2021. Intelligent detection of chemical plant based on poly FPN neural network model. National Remote Sensing Bulletin
  35. 35.
    Zeng D, Chen S J, Chen B Y and Li S Y. 2018. Improving remote sensing scene classification by integrating global-context and local-object features. Remote Sensing, 10(5): 734
  36. 36.
    Zhang J, Zhang M, Shi L K, Yan W J and Pan B. 2019. A multi-scale approach for remote sensing scene classification based on feature maps selection and region representation. Remote Sensing, 11(21): 2504
  37. 37.
    Zhang S Y. 2020. High-Resolution Remote Sensing Image Land Cover Classification Based on Deep Learning and Multi-Scale and Multi-Feature Fusion. Hangzhou: Zhejiang University
  38. 38.
    Zhao L J, Tang P and Huo L Z. 2016. Feature significance-based multibag-of-visual-words model for remote sensing image scene classification. Journal of Applied Remote Sensing, 10(3): 035004
  39. 39.
    Zhao W Z and Du S H. 2016. Learning multiscale and deep representations for classifying remotely sensed imagery. ISPRS Journal of Photogrammetry and Remote Sensing, 113: 155-165
  40. 40.
    Zheng X T, Yuan Y and Lu X Q. 2019. A deep scene representation for aerial scene classification. IEEE Transactions on Geoscience and Remote Sensing, 57(7): 4799-4809
  41. 41.
    Zheng Y. 2015. Methodologies for cross-domain data fusion: an overview. IEEE Transactions on Big Data, 1(1): 16-34
  42. 42.
    Zhong Y F, Zhu Q Q and Zhang L P. 2015. Scene classification based on the multifeature fusion probabilistic topic model for high spatial resolution remote sensing imagery. IEEE Transactions on Geoscience and Remote Sensing, 53(11): 6207-6222
  43. 43.
    Zhu Q Q, Zhong Y F, Zhao B, Xia G S and Zhang L P. 2016. Bag-of-visual-words scene classifier with local and global features for high spatial resolution remote sensing imagery. IEEE Geoscience and Remote Sensing Letters, 13(6): 747-751
  44. 44.
    Zhu Q Q, Li Z, Zhang Y N, Li J L, Du Y Q, Guan Q F and Li D R. 2021. Global-Local-Aware conditional random fields based building extraction for high spatial resolution remote sensing images. National Remote Sensing Bulletin, 25(7): 1422-1433
  45. 45.
    Zou Q, Ni L H, Zhang T and Wang Q. 2015. Deep learning based feature selection for remote sensing scene classification. IEEE Geoscience and Remote Sensing Letters, 12(11): 2321-2325

Lesen Sie die ganze Passage

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website