Spectral-spatial attention bilateral network for hyperspectral image classification

YANG Xing; CHI Yue; ZHOU Yatong; WANG Yang

doi:10.11834/jrs.20210563

Models and Methods | Views : 0 下载量: 324 CSCD: 0 更多指标

R-PDF
PDF
Export
Share
Collection
Album

Spectral-spatial attention bilateral network for hyperspectral image classification
Vol. 27, Issue 11, Pages: 2565-2578(2023)
Published： 07 November 2023 ，
DOI： 10.11834/jrs.20210563

扫描看全文

杨星，池越，周亚同，王杨.2023.基于光谱—空间注意力双边网络的高光谱图像分类.遥感学报，27（11）： 2565-2578

Yang X， Chi Y， Zhou Y T and Wang Y. 2023. Spectral-spatial attention bilateral network for hyperspectral image classification. National Remote Sensing Bulletin， 27（11）：2565-2578
杨星，池越，周亚同，王杨.2023.基于光谱—空间注意力双边网络的高光谱图像分类.遥感学报，27（11）： 2565-2578 DOI： 10.11834/jrs.20210563.

Yang X， Chi Y， Zhou Y T and Wang Y. 2023. Spectral-spatial attention bilateral network for hyperspectral image classification. National Remote Sensing Bulletin， 27（11）：2565-2578 DOI： 10.11834/jrs.20210563.

摘要

在过去几年里，卷积神经网络已经在高光谱图像分类上取得良好的效果，然而高光谱图像的高维性和卷积神经网络对所有波段的平等处理，限制了这些方法性能。本文提出了一种端到端的光谱空间注意力双边网络SSABN（Spectral-Spatial Attention Bilateral Network），直接将原始图像3D块作为输入数据，而不需要进行预处理。首先，通过光谱空间注意力模块从原始数据中增强有用波段，抑制无效波段。然后，设计双边网络两条路径。其中，空间路径用于提取空间信息，上下文路径用于提供更大的感受野，并通过特征融合模块有效的结合特征。实验结果表明，SSABN在3个公开数据集上取得了更高的分类精度，同时有效的减少训练时间。

Abstract

HypeSspectral Image Classification (HSIC) is a pixel-level classification problem

and it involves classifying each pixel in the hyperspectral image and confirming the pixel category. However

discriminative features in HSIC task are difficult to acquire and learn

and the extraction of sufficient and effective features directly affects the classification results. In the past few years

Convolutional Neural Networks (CNNs) have achieved better results in HSIC

but the high dimensionality of hyperspectral images and the equal processing of all bands by CNNs have limited the performance of CNN. This study proposes an end-to-end Spectral-Spatial Attention Bilateral Network (SSABN) for HSIC. The network directly uses 3D blocks of the original image as input data without the complicated preprocessing. First

the original data are processed through the spectral-spatial attention module to enhance the useful bands or pixels for classification and suppress invalid information. Then

the spatial and context paths of the bilateral network are designed. The spatial path has three layers

and each layer is composed of convolution

batch normalization

and Relu activation function to extract spatial information. The context path is composed of three downsampling and attention refining modules. The downsampling is used to provide receptive field

and the attention refinement module is used to refine downsampling features. Finally

a feature fusion module is designed to fuse different levels of features through maximum pooling and average pooling for generating discriminative features. Compared with common CNN

SSABN can adaptively enhance effective information

extract more abstract discriminative features

and consume less training time. Experimental results show that SSABN has good fitting ability in different training sample ratios. In the results of ablation experiments

the accuracy of the spectral-spatial attention mechanism is 1%—2% higher than those of other mainstream attention mechanisms

and the feature fusion module can improve the discrimination of extracted features. In the experiments of three public datasets

the classification accuracy of SSABN is higher than 99%

and the training time is less than those of other methods. The classification performance of SSABN is better than those of other hyperspectral image classification algorithms

while reducing its training time can more effectively improve accuracy and efficiency.

关键词

遥感卷积神经网络深度学习特征融合Indian Pine数据Pavia University数据Salinas数据集

Keywords

remote sensingconvolutional neural networkdeep learningfeature fusionIndian Pine datasetPavia University datasetSalinas dataset

references

Cao Y, Ye Y T, Zhao H L, Jiang Y Z and Wang H. 2019. Remote sensing of total suspended matter concentration and turbidity in a macrophytic lake. Journal of Remote Sensing, 23(6): 1253-1268

曹引, 冶运涛, 赵红莉, 蒋云钟, 王浩. 2019. 草型湖泊总悬浮物浓度和浊度遥感监测. 遥感学报, 23(6): 1253-1268 [DOI: 10.11834/jrs.20198144http://dx.doi.org/10.11834/jrs.20198144]

Chen X, Lee H and Lee M. 2019. Feasibility of using hyperspectral remote sensing for environmental heavy metal monitoring. [DOI: 10.5194/isprs-archives-XLII-3-W7-1-2019http://dx.doi.org/10.5194/isprs-archives-XLII-3-W7-1-2019]

Chen Y S, Lin Z H, Zhao X, Wang G and Gu Y. 2014. Deep learning-based classification of hyperspectral data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(6): 2094-2107. [DOI: 10.1109/JSTARS.2014.2329330http://dx.doi.org/10.1109/JSTARS.2014.2329330]

Chen Y S, Zhao X and Jia X P. 2015. Spectral-spatial classification of hyperspectral data based on deep belief network. IEEE Journal of Selected Topics in Applied Earth Observations and remote Sensing, 8(6): 2381-2392. [DOI: 10.1109/JSTARS.2015.2388577http://dx.doi.org/10.1109/JSTARS.2015.2388577]

Das A, Saha I and Scherer R. 2020. GhoMR: multi-receptive lightweight residual modules for hyperspectral classification. Sensors, 20(23): 6823. [DOI: 10.3390/s20236823http://dx.doi.org/10.3390/s20236823]

Deng C, Xue Y M, Liu X L, Li C and Tao D C. 2019. Active transfer learning network: a unified deep joint spectral-spatial feature learning model for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 57(3): 1741-1754. [DOI: 10.1109/TGRS.2018.2868851http://dx.doi.org/10.1109/TGRS.2018.2868851]

Ghamisi P, Dalla Mura M and Benediktsson J A. 2015. A survey on spectral-spatial classification techniques based on attribute profiles. IEEE Transactions on Geoscience and Remote Sensing, 53(5): 2335-2353. [DOI: 10.1109/TGRS.2014.2358934http://dx.doi.org/10.1109/TGRS.2014.2358934]

Ham J, Chen Y C, Crawford M M and Ghosh J. 2005. Investigation of the random forest framework for classification of hyperspectral data. IEEE Transactions on Geoscience and Remote Sensing, 43(3): 492-501. [DOI: 10.1109/TGRS.2004.842481http://dx.doi.org/10.1109/TGRS.2004.842481]

Li J L, Sun D W and Cheng J H. 2016. Recent advances in nondestructive analytical techniques for determining the total soluble solids in fruits: a review. Comprehensive Reviews in Food Science and Food Safety, 15(5): 897-911. [DOI: 10.1111/1541-4337.12217http://dx.doi.org/10.1111/1541-4337.12217]

Li R, Zheng S Y, Duan C X, Yang Y and Wang X Q. 2020. Classification of hyperspectral image based on double-branch dual-attention mechanism network. Remote Sensing, 12(3): 582. [DOI: 10.3390/rs12030582http://dx.doi.org/10.3390/rs12030582]

Li T, Zhang J P and Zhang Y. 2015. Classification of hyperspectral image based on deep belief networks//2014 IEEE International Conference on Image Processing (ICIP). Paris: IEEE. [DOI: 10.1109/ICIP.2014.7026039http://dx.doi.org/10.1109/ICIP.2014.7026039]

Luo R B, Liao W Z, Philips W and Pi Y G. 2015. An improved semi-supervised local discriminant analysis for feature extraction of hyperspectral image. 2015 Joint Urban Remote Sensing Event (JURSE). Lausanne: IEEE. [DOI: 10.1109/JURSE.2015.7120508http://dx.doi.org/10.1109/JURSE.2015.7120508]

Luo W J, Li Y J, Urtasun R and Zemel R. 2017. Understanding the effective receptive field in deep convolutional neural networks//Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona, Spain: Curran Associates Inc.

Ma L, Crawford M M and Tian J W. 2010. Local manifold learning-based k-nearest-neighbor for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 48(11): 4099-4109. [DOI: 10.1109/TGRS.2010.2055876http://dx.doi.org/10.1109/TGRS.2010.2055876]

Mei S H, Ji J Y, Geng Y H, Zhang Z, Li X and Du Q. 2019a. Unsupervised spatial-spectral feature learning by 3D convolutional autoencoder for hyperspectral classification. IEEE Transactions on Geoscience and Remote Sensing, 57(9)6808-6820. [DOI: 10.1109/TGRS.2019.2908756http://dx.doi.org/10.1109/TGRS.2019.2908756]

Mei X G, Pan E T, Ma Y, Dai X B, Huang J, Fan F, Du Q L, Zheng H and Ma J Y. 2019b. Spectral-spatial attention networks for hyperspectral image classification. Remote Sensing, 11(8): 963. [DOI: 10.3390/rs11080963http://dx.doi.org/10.3390/rs11080963]

Melgani F and Bruzzone L. 2004. Classification of hyperspectral remote sensing images with support vector machines. IEEE Transactions on Geoscience and Remote Sensing, 42(8): 1778-1790. [DOI: 10.1109/TGRS.2004.831865http://dx.doi.org/10.1109/TGRS.2004.831865]

Mou L C, Ghamisi P and Zhu X X. 2017. Deep recurrent neural networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 55(7): 3639-3655. [DOI: 10.1109/TGRS.2016.2636241http://dx.doi.org/10.1109/TGRS.2016.2636241]

Murphy R J, Whelan B, Chlingaryan A and Sukkarieh S. 2018. Quantifying leaf-scale variations in water absorption in lettuce from hyperspectral imagery: a laboratory study with implications for measuring leaf water content in the context of precision agriculture. Precision Agriculture, 20(4): 767-787. [DOI: 10.1007/s11119-018-9610-5http://dx.doi.org/10.1007/s11119-018-9610-5]

Rodarmel C and Shan J. 2002. Principal component analysis for hyperspectral image classification. Surveying and Land Information Systems, 62(2): 115-123.

Roy S K, Dubey S R, Chatterjee S, Chaudhuri B B. 2020. FuSENet: fused squeeze-and-excitation network for spectral-spatial hyperspectral image classification. IET Image Process, 14(8): 1653-1661. [DOI: 10.1049/iet-ipr.2019.1462http://dx.doi.org/10.1049/iet-ipr.2019.1462]

Roy S K, Krishna G, Dubey S R and Chaudhuri B B. 2020. HybridSN: exploring 3-D-2-D CNN feature hierarchy for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 17(2): 277-281 [DOI: 10.1109/LGRS.2019.2918719http://dx.doi.org/10.1109/LGRS.2019.2918719]

Shimoni M, Haelterman R and Perneel C. 2019. Hypersectral imaging for military and security applications: combining myriad processing and sensing techniques. IEEE Geoscience and Remote Sensing Magazine, 7(2): 101-117. [DOI: 10.1109/MGRS.2019.2902525http://dx.doi.org/10.1109/MGRS.2019.2902525]

Sun W and Du Q. 2018. Graph-regularized fast and robust principal component analysis for hyperspectral band selection. IEEE Transactions on Geoscience and Remote Sensing, 56(6): 3185-3195. [DOI: 10.1109/TGRS.2018.2794443http://dx.doi.org/10.1109/TGRS.2018.2794443]

Sun W W and Du Q. 2019. Hyperspectral band selection: a review. IEEE Geoscience and Remote Sensing Magazine, 7(2): 118-139. [DOI: 10.1109/MGRS.2019.2911100http://dx.doi.org/10.1109/MGRS.2019.2911100]

Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser Ł and Polosukhin I. 2017. Attention is all you need//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach: Curran Associates Inc.

Wang J and Chang C I. 2006. Independent component analysis-based dimensionality reduction with applications in hyperspectral image analysis. IEEE Transactions on Geoscience and Remote Sensing, 44(6): 1586-1600. [DOI: 10.1109/TGRS.2005.863297http://dx.doi.org/10.1109/TGRS.2005.863297]

Wang W J, Dou S G, Jiang Z M and Sun L J. 2018. A fast dense spectral-spatial convolution network framework for hyperspectral images classification. Remote Sensing, 10(7): 1068. [DOI: 10.3390/rs10071068http://dx.doi.org/10.3390/rs10071068]

Zhang X R, Sun Y J, Jiang K, Li C, Jiao L C and Zhou H Y. 2018. Spatial sequential recurrent neural network for hyperspectral image classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(11): 4141-4155. [DOI: 10.1109/JSTARS.2018.2844873http://dx.doi.org/10.1109/JSTARS.2018.2844873]

Zhong Z L, Li J, Luo Z M and Chapman M. 2018. Spectral-spatial residual network for hyperspectral image classification: a 3-d deep learning framework. IEEE Transactions on Geoscience and Remote Sensing, 56(2): 847-858. [DOI: 10.1109/TGRS.2017.2755542http://dx.doi.org/10.1109/TGRS.2017.2755542]

Zhou Y C, Peng J T and Chen C L P. 2015. Extreme learning machine with composite kernels for hyperspectral image classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 8(6): 2351-2360. [DOI: 10.1109/JSTARS.2014.2359965http://dx.doi.org/10.1109/JSTARS.2014.2359965]

Zhu L, Chen Y S, Ghamisi P and Benediktsson J A. 2018. Generative adversarial networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 56(9): 5046-5063 [DOI: 10.1109/TGRS.2018.2805286http://dx.doi.org/10.1109/TGRS.2018.2805286]

Alert me when the article has been cited

提交

Stripe noise removal in high resolution satellite remote sensing images based on deep learning

Building extraction in pixel level from aerial imagery with a deep encoder-decoder network

MtSCCD： Land-use scene classification and change-detection dataset for deep learning

Optical-signal token guided change detection network for heterogeneous remote sensing image