سابِق|تال

MtSCCD： Land-use scene classification and change-detection dataset for deep learning

  • role： First author第一作者
  • Affiliation：

    School of Remote Sensing & Geomatics Engineering, Nanjing University of Information Science and Technology, Nanjing 210044, China

    State Key Laboratory of Remote Sensing Science, Beijing Normal University, Beijing 100875, China

  • Email：zhouwx@nuist.edu.cn
  • Introduction：周维勋，研究方向为遥感图像智能解译。E-mail： zhouwx@nuist.edu.cn
ZHOU Weixun12,  
  • Affiliation：

    School of Remote Sensing & Geomatics Engineering, Nanjing University of Information Science and Technology, Nanjing 210044, China

LIU Jinglei1,  
  • Affiliation：

    School of Remote Sensing & Geomatics Engineering, Nanjing University of Information Science and Technology, Nanjing 210044, China

PENG Daifeng1,  
  • Affiliation：

    School of Remote Sensing & Geomatics Engineering, Nanjing University of Information Science and Technology, Nanjing 210044, China

GUAN Haiyan1,  
  • Affiliation：

    State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China

SHAO Zhenfeng3

ملخص

Land-Use Scene Classification and change Detection (LUSCD) aim to recognize land-use types and monitor their changes by using Remote-Sensing (RS) images, which play an important role in urban planning and land-use optimization. In the era of RS big data, conventional hand-crafted feature-based methods are infeasible for LUSCD because the extracted features are not sufficiently discriminative for RS images with high complexity. As a novel data-driven paradigm for information extraction from RS images, deep learning provides a new solution for LUSCD. However, the existing publicly available datasets have limited samples and is thus unable to train a successful deep-learning model. Therefore, it has great significance in constructing an open and large-scale LUSCD benchmark.To advance the progress of LUSCD using deep-learning methods, this paper releases a large-scale scene classification and change-detection dataset termed Multi-temporal Scene Classification and Change Detection (MtSCCD). The RGB images in MtSCCD are cropped from large-size high-resolution RS images captured from the central areas of five China cities, namely, Hangzhou, Shanghai, Wuhan, Nanjing, and Hefei. The size of the cropped images is 300×300 pixels with the spatial resolution of around 1 m. MtSCCD has 10 land use classes, which are residential land, public service and commercial land, educational land, industrial land, transportation land, agricultural land, water body, green space, woodland, and woodland. Based on the cropped land-use images in MtSCCD, this paper constructs two sub-datasets termed MtSCCD_LUSC (MtSCCD Land Use Scene Classification) and MtSCCD_LUCD (MtSCCD Land Use Change Detection) for land-use scene classification (LUSC) and land-use change detection (LUCD), respectively. MtSCCD dataset has the following characteristics. (1) It is currently the largest publicly available LUSCD dataset, and both of the two sub-datasets (i.e., MtSCCD_LUSC and MtSCCD_LUCD) have 65548 images in total. (2) The images in MtSCCD are split into training set, validation set, and testing set according to the five cities. For example, images from three of the five cities are randomly split into training and validation set, whereas the rest remain to be the testing set. Therefore, MtSCCD has high extensibility, i.e., it can be easily extended to be a larger dataset. (3) For a deep-learning model, the training set and testing set are categorized from different cities, so it is beneficial to demonstrate the model’s generalization ability. (4) MtSCCD has high intra-class diversity, making it a challenging dataset.Based on MtSCCD_LUSC and MtSCCD_LUCD, this paper evaluates several deep-learning feature-based methods for LUSC and LUCD. Specifically, AlexNet, VGG networks (i.e., VGG16 and VGG19), GoogLeNet, and ResNet networks (i.e., ResNet18, ResNet50, and ResNet101) are selected to extract deep-learning features that are then fed into SVM for LUSC. We also evaluate DenseNet, EfficientNet, SENet, ViT, and SwinT for LUSC. Two kinds of LUCD approaches including conventional classification-based methods and current similarity-based methods have been evaluated. Experimental results show that the highest overall accuracy of MtSCCD_LUSC dataset is around 76%, indicating much room for improvement. Regarding LUCD, similarity-based methods particularly similarity learning-based ones outperform classification-based methods by a significant margin, providing a promising research direction for LUCD.This paper presents the currently largest scene classification and change-detection dataset MtSCCD based on high-resolution RS images of the central area of five China cities. MtSCCD contains two subsets MtSCCD_LUSC and MtSCCD_LUCD. Both had 10 land-use types and 65548 images in total. Based on the two sub-datasets, this paper evaluates the performance of several deep networks for scene classification and change detection, expecting to provide baseline results for related researchers. We hope that the MtSCCD dataset can promote this progress in land-use type recognition and monitoring.

مفهوم

land use;scene classification;change detection;dataset;information extraction;feature extraction;deep learning;convolutional neural network

References

  1. 1.
    Bai K, Mu X D, Chen X B, Zhu Y Q and You X A. 2022. Unsupervised remote sensing image scene classification based on semi-supervised learning. Acta Geodaetica et Cartographica Sinica, 51(5): 691-702
  2. 2.
    Cheng G, Han J W and Lu X Q. 2017. Remote sensing image scene classification: benchmark and state of the art. Proceedings of the IEEE, 105(10): 1865-1883
  3. 3.
    Cheng G, Wang G X and Han J W. 2022. ISNet: towards improving separability for remote sensing image change detection. IEEE Transactions on Geoscience and Remote Sensing, 60: 5623811
  4. 4.
    Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X H, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S, Uszkoreit J and Houlsby N. 2021. An image is worth 16x16 words: transformers for image recognition at scale. arXiv:2010.11929
  5. 5.
    Feng Q L, Chen B A, Li G Q, Yao X C, Gao B B and Zhang L C. 2022. A review for sample datasets of remote sensing imagery. National Remote Sensing Bulletin, 26(4): 589-605
  6. 6.
    He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE: 770-778
  7. 7.
    Hu J, Shen L and Sun G. 2018. Squeeze-and-excitation networks//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 7132-7141
  8. 8.
    Huang G, Liu Z, Van Der Maaten L and Weinberger K Q. 2017. Densely connected convolutional networks//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 2261-2269
  9. 9.
    Huang Y H and Zhou W X. 2022. Similarity method for high-resolution remote sensing scene change detection. Bulletin of Surveying and Mapping, (8): 48-53
  10. 10.
    Krizhevsky A, Sutskever I and Hinton G E. 2012. ImageNet classification with deep convolutional neural networks//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe: Curran Associates Inc.: 1097-1105
  11. 11.
    LeCun Y, Bengio Y and Hinton G. 2015. Deep learning. Nature, 521(7553): 436-444
  12. 12.
    Li H F, Dou X, Tao C, Wu Z X, Chen J, Peng J, Deng M and Zhao L. 2020. RSI-CB: a large-scale remote sensing image classification benchmark using crowdsourced data. Sensors, 20(6): 1594
  13. 13.
    Liu K, Zhou Z, Li S Y, Liu Y F, Wan X, Liu Z W, Tan H and Zhang W F. 2020. Scene classification dataset using the Tiangong-1 hyperspectral remote sensing imagery and its applications. Journal of Remote Sensing (Chinese), 24(9): 1077-1087
  14. 14.
    Liu Z, Lin Y T, Cao Y, Hu H, Wei Y X, Zhang Z, Lin S and Guo B N. 2021. Swin transformer: hierarchical vision transformer using shifted windows//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Montreal: IEEE: 9992-10002
  15. 15.
    Ma L, Liu Y, Zhang X L, Ye Y X, Yin G F and Johnson B A. 2019. Deep learning in remote sensing applications: a meta-analysis and review. ISPRS Journal of Photogrammetry and Remote Sensing, 152: 166-177
  16. 16.
    Qian X L, Li J, Cheng G, Yao X W, Zhao S N, Chen Y B and Jiang L Y. 2018. Evaluation of the effect of feature extraction strategy on the performance of high-resolution remote sensing image scene classification. Journal of Remote Sensing, 22(5): 758-776
  17. 17.
    Simonyan K and Zisserman A. 2015. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556
  18. 18.
    Sui H G, Feng W Q, Li W Z, Sun K M and Xu C. 2018. Review of change detection methods for multi-temporal remote sensing imagery. Geomatics and Information Science of Wuhan University, 43(12): 1885-1898
  19. 19.
    Szegedy C, Liu W, Jia Y Q, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V and Rabinovich A. 2015. Going deeper with convolutions//2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE: 1-9
  20. 20.
    Tan M X and Le Q V. 2019. EfficientNet: rethinking model scaling for convolutional neural networks//Proceedings of the 36th International Conference on Machine Learning. Long Beach: [s.n.]: 6105-6114
  21. 21.
    Wu C, Zhang L F and Zhang L P. 2016. A scene change detection framework for multi-temporal very high resolution remote sensing images. Signal Processing, 124: 184-197
  22. 22.
    Wu C, Zhang L P and Du B. 2017. Kernel slow feature analysis for scene change detection. IEEE Transactions on Geoscience and Remote Sensing, 55(4): 2367-2384
  23. 23.
    Wu F, Zhang H, Wang C, Li L, Li J J, Chen W R and Zhang B. 2022. SARBuD1.0: a SAR building dataset based on GF-3 FSII imageries for built-up area extraction with deep learning method. National Remote Sensing Bulletin, 26(4): 620-631
  24. 24.
    Xia G S, Hu J W, Hu F, Shi B G, Bai X, Zhong Y F, Zhang L P and Lu X Q. 2017. AID: a benchmark data set for performance evaluation of aerial scene classification. IEEE Transactions on Geoscience and Remote Sensing, 55(7): 3965-3981
  25. 25.
    Xia G S, Yang W, Delon J, Gousseau Y, Sun H and Maître H. 2010. Structural high-resolution satellite image indexing//ISPRS TC VII Symposium - 100 Years ISPRS. 298-303
  26. 26.
    Yang B S, Han X and Dong Z. 2021. Point cloud benchmark dataset WHU-TLS and WHU-MLS for deep learning. National Remote Sensing Bulletin, 25(1): 231-240
  27. 27.
    Yang Y and Newsam S. 2010. Bag-of-visual-words and spatial extensions for land-use classification//Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems. San Jose: ACM: 270-279
  28. 28.
    Yu W Q, Cheng G, Wang M J, Yao Y Q, Xie X X, Yao X W and Han J W. 2022. MAR20: a benchmark for military aircraft recognition in remote sensing images. National Remote Sensing Bulletin, 1-11
  29. 29.
    Yuan J W, Ru L X, Wang S G and Wu C. 2022. WH-MAVS: a novel dataset and deep learning benchmark for multiple land use and land cover applications. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 15: 1575-1590
  30. 30.
    Yuan J W, Wu C, Du B, Zhang L P and Wang S G. 2020. Analysis of landscape pattern on urban land use based on GF-5 hyperspectral data. Journal of Remote Sensing (Chinese), 24(4): 465-478
  31. 31.
    Zhang L P and Wu C. 2017. Advance and future development of change detection for multi-temporal remote sensing imagery. Acta Geodaetica et Cartographica Sinica, 46(10): 1447-1459
  32. 32.
    Zhang L P, Zhang L F and Du B. 2016. Deep learning for remote sensing data: a technical tutorial on the state of the art. IEEE Geoscience and Remote Sensing Magazine, 4(2): 22-40
  33. 33.
    Zhou W X, Newsam S, Li C M and Shao Z F. 2018. PatternNet: a benchmark dataset for performance evaluation of remote sensing image retrieval. ISPRS Journal of Photogrammetry and Remote Sensing, 145: 197-209
  34. 34.
    Zhu X X, Tuia D, Mou L C, Xia G S, Zhang L P, Xu F and Fraundorfer F. 2017. Deep learning in remote sensing: a comprehensive review and list of resources. IEEE Geoscience and Remote Sensing Magazine, 5(4): 8-36
  35. 35.
    Zou Q, Ni L H, Zhang T and Wang Q. 2015. Deep learning based feature selection for remote sensing scene classification. IEEE Geoscience and Remote Sensing Letters, 12(11): 2321-2325

قراءة النص الكامل

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website