Spectral-spatial attention bilateral network for hyperspectral image classification

YANG Xing; CHI Yue; ZHOU Yatong; WANG Yang

doi:10.11834/jrs.20210563

Models and Methods | Views : 0 下载量: 2563 CSCD: 1

R-PDF
PDF
Export
Share
Collection
Album

Spectral-spatial attention bilateral network for hyperspectral image classification
Vol. 27, Issue 11, Pages: 2565-2578(2023)
Received：06 January 2021，

Published：07 November 2023
DOI： 10.11834/jrs.20210563
稿件说明：

移动端阅览

杨星，池越，周亚同，王杨.2023.基于光谱—空间注意力双边网络的高光谱图像分类.遥感学报，27（11）： 2565-2578 DOI： 10.11834/jrs.20210563.

Yang X， Chi Y， Zhou Y T and Wang Y. 2023. Spectral-spatial attention bilateral network for hyperspectral image classification. National Remote Sensing Bulletin， 27（11）：2565-2578 DOI： 10.11834/jrs.20210563.

摘要

在过去几年里，卷积神经网络已经在高光谱图像分类上取得良好的效果，然而高光谱图像的高维性和卷积神经网络对所有波段的平等处理，限制了这些方法性能。本文提出了一种端到端的光谱空间注意力双边网络SSABN（Spectral-Spatial Attention Bilateral Network），直接将原始图像3D块作为输入数据，而不需要进行预处理。首先，通过光谱空间注意力模块从原始数据中增强有用波段，抑制无效波段。然后，设计双边网络两条路径。其中，空间路径用于提取空间信息，上下文路径用于提供更大的感受野，并通过特征融合模块有效的结合特征。实验结果表明，SSABN在3个公开数据集上取得了更高的分类精度，同时有效的减少训练时间。

Abstract

HypeSspectral Image Classification (HSIC) is a pixel-level classification problem

and it involves classifying each pixel in the hyperspectral image and confirming the pixel category. However

discriminative features in HSIC task are difficult to acquire and learn

and the extraction of sufficient and effective features directly affects the classification results. In the past few years

Convolutional Neural Networks (CNNs) have achieved better results in HSIC

but the high dimensionality of hyperspectral images and the equal processing of all bands by CNNs have limited the performance of CNN. This study proposes an end-to-end Spectral-Spatial Attention Bilateral Network (SSABN) for HSIC. The network directly uses 3D blocks of the original image as input data without the complicated preprocessing. First

the original data are processed through the spectral-spatial attention module to enhance the useful bands or pixels for classification and suppress invalid information. Then

the spatial and context paths of the bilateral network are designed. The spatial path has three layers

and each layer is composed of convolution

batch normalization

and Relu activation function to extract spatial information. The context path is composed of three downsampling and attention refining modules. The downsampling is used to provide receptive field

and the attention refinement module is used to refine downsampling features. Finally

a feature fusion module is designed to fuse different levels of features through maximum pooling and average pooling for generating discriminative features. Compared with common CNN

SSABN can adaptively enhance effective information

extract more abstract discriminative features

and consume less training time. Experimental results show that SSABN has good fitting ability in different training sample ratios. In the results of ablation experiments

the accuracy of the spectral-spatial attention mechanism is 1%—2% higher than those of other mainstream attention mechanisms

and the feature fusion module can improve the discrimination of extracted features. In the experiments of three public datasets

the classification accuracy of SSABN is higher than 99%

and the training time is less than those of other methods. The classification performance of SSABN is better than those of other hyperspectral image classification algorithms

while reducing its training time can more effectively improve accuracy and efficiency.

关键词

Keywords

references

Cao Y , Ye Y T , Zhao H L , Jiang Y Z and Wang H . 2019 . Remote sensing of total suspended matter concentration and turbidity in a macrophytic lake . Journal of Remote Sensing , 23 ( 6 ): 1253 - 1268

曹引 , 冶运涛 , 赵红莉 , 蒋云钟 , 王浩 . 2019 . 草型湖泊总悬浮物浓度和浊度遥感监测 . 遥感学报 , 23 ( 6 ): 1253 - 1268 [ DOI: 10.11834/jrs.20198144 http://dx.doi.org/10.11834/jrs.20198144 ]

Chen X , Lee H and Lee M . 2019 . Feasibility of using hyperspectral remote sensing for environmental heavy metal monitoring . [ DOI: 10.5194/isprs-archives-XLII-3-W7-1-2019 http://dx.doi.org/10.5194/isprs-archives-XLII-3-W7-1-2019 ]

Chen Y S , Lin Z H , Zhao X , Wang G and Gu Y . 2014 . Deep learning-based classification of hyperspectral data . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 7 ( 6 ): 2094 - 2107 . [ DOI: 10.1109/JSTARS.2014.2329330 http://dx.doi.org/10.1109/JSTARS.2014.2329330 ]

Chen Y S , Zhao X and Jia X P . 2015 . Spectral-spatial classification of hyperspectral data based on deep belief network . IEEE Journal of Selected Topics in Applied Earth Observations and remote Sensing , 8 ( 6 ): 2381 - 2392 . [ DOI: 10.1109/JSTARS.2015.2388577 http://dx.doi.org/10.1109/JSTARS.2015.2388577 ]

Das A , Saha I and Scherer R . 2020 . GhoMR: multi-receptive lightweight residual modules for hyperspectral classification . Sensors , 20 ( 23 ): 6823 . [ DOI: 10.3390/s20236823 http://dx.doi.org/10.3390/s20236823 ]

Deng C , Xue Y M , Liu X L , Li C and Tao D C . 2019 . Active transfer learning network: a unified deep joint spectral-spatial feature learning model for hyperspectral image classification . IEEE Transactions on Geoscience and Remote Sensing , 57 ( 3 ): 1741 - 1754 . [ DOI: 10.1109/TGRS.2018.2868851 http://dx.doi.org/10.1109/TGRS.2018.2868851 ]

Ghamisi P , Dalla Mura M and Benediktsson J A . 2015 . A survey on spectral-spatial classification techniques based on attribute profiles . IEEE Transactions on Geoscience and Remote Sensing , 53 ( 5 ): 2335 - 2353 . [ DOI: 10.1109/TGRS.2014.2358934 http://dx.doi.org/10.1109/TGRS.2014.2358934 ]

Ham J , Chen Y C , Crawford M M and Ghosh J . 2005 . Investigation of the random forest framework for classification of hyperspectral data . IEEE Transactions on Geoscience and Remote Sensing , 43 ( 3 ): 492 - 501 . [ DOI: 10.1109/TGRS.2004.842481 http://dx.doi.org/10.1109/TGRS.2004.842481 ]

Li J L , Sun D W and Cheng J H . 2016 . Recent advances in nondestructive analytical techniques for determining the total soluble solids in fruits: a review . Comprehensive Reviews in Food Science and Food Safety , 15 ( 5 ): 897 - 911 . [ DOI: 10.1111/1541-4337.12217 http://dx.doi.org/10.1111/1541-4337.12217 ]

Li R , Zheng S Y , Duan C X , Yang Y and Wang X Q . 2020 . Classification of hyperspectral image based on double-branch dual-attention mechanism network . Remote Sensing , 12 ( 3 ): 582 . [ DOI: 10.3390/rs12030582 http://dx.doi.org/10.3390/rs12030582 ]

Li T , Zhang J P and Zhang Y . 2015 . Classification of hyperspectral image based on deep belief networks // 2014 IEEE International Conference on Image Processing (ICIP) . Paris : IEEE . [ DOI: 10.1109/ICIP.2014.7026039 http://dx.doi.org/10.1109/ICIP.2014.7026039 ]

Luo R B , Liao W Z , Philips W and Pi Y G . 2015 . An improved semi-supervised local discriminant analysis for feature extraction of hyperspectral image. 2015 Joint Urban Remote Sensing Event (JURSE) . Lausanne : IEEE . [ DOI: 10.1109/JURSE.2015.7120508 http://dx.doi.org/10.1109/JURSE.2015.7120508 ]

Luo W J , Li Y J , Urtasun R and Zemel R . 2017 . Understanding the effective receptive field in deep convolutional neural networks // Proceedings of the 30th International Conference on Neural Information Processing Systems . Barcelona, Spain : Curran Associates Inc.

Ma L , Crawford M M and Tian J W . 2010 . Local manifold learning-based k -nearest-neighbor for hyperspectral image classification . IEEE Transactions on Geoscience and Remote Sensing , 48 ( 11 ): 4099 - 4109 . [ DOI: 10.1109/TGRS.2010.2055876 http://dx.doi.org/10.1109/TGRS.2010.2055876 ]

Mei S H , Ji J Y , Geng Y H , Zhang Z , Li X and Du Q . 2019a . Unsupervised spatial-spectral feature learning by 3D convolutional autoencoder for hyperspectral classification . IEEE Transactions on Geoscience and Remote Sensing , 57(9)6808 - 6820 . [ DOI: 10.1109/TGRS.2019.2908756 http://dx.doi.org/10.1109/TGRS.2019.2908756 ]

Mei X G , Pan E T , Ma Y , Dai X B , Huang J , Fan F , Du Q L , Zheng H and Ma J Y . 2019b . Spectral-spatial attention networks for hyperspectral image classification . Remote Sensing , 11 ( 8 ): 963 . [ DOI: 10.3390/rs11080963 http://dx.doi.org/10.3390/rs11080963 ]

Melgani F and Bruzzone L . 2004 . Classification of hyperspectral remote sensing images with support vector machines . IEEE Transactions on Geoscience and Remote Sensing , 42 ( 8 ): 1778 - 1790 . [ DOI: 10.1109/TGRS.2004.831865 http://dx.doi.org/10.1109/TGRS.2004.831865 ]

Mou L C , Ghamisi P and Zhu X X . 2017 . Deep recurrent neural networks for hyperspectral image classification . IEEE Transactions on Geoscience and Remote Sensing , 55 ( 7 ): 3639 - 3655 . [ DOI: 10.1109/TGRS.2016.2636241 http://dx.doi.org/10.1109/TGRS.2016.2636241 ]

Murphy R J , Whelan B , Chlingaryan A and Sukkarieh S . 2018 . Quantifying leaf-scale variations in water absorption in lettuce from hyperspectral imagery: a laboratory study with implications for measuring leaf water content in the context of precision agriculture . Precision Agriculture , 20 ( 4 ): 767 - 787 . [ DOI: 10.1007/s11119-018-9610-5 http://dx.doi.org/10.1007/s11119-018-9610-5 ]

Rodarmel C and Shan J . 2002 . Principal component analysis for hyperspectral image classification . Surveying and Land Information Systems , 62 ( 2 ): 115 - 123 .

Roy S K , Dubey S R , Chatterjee S , Chaudhuri B B . 2020 . FuSENet: fused squeeze-and-excitation network for spectral-spatial hyperspectral image classification . IET Image Process , 14 ( 8 ): 1653 - 1661 . [ DOI: 10.1049/iet-ipr.2019.1462 http://dx.doi.org/10.1049/iet-ipr.2019.1462 ]

Roy S K , Krishna G , Dubey S R and Chaudhuri B B . 2020 . HybridSN: exploring 3-D-2-D CNN feature hierarchy for hyperspectral image classification . IEEE Geoscience and Remote Sensing Letters , 17 ( 2 ): 277 - 281 [ DOI: 10.1109/LGRS.2019.2918719 http://dx.doi.org/10.1109/LGRS.2019.2918719 ]

Shimoni M , Haelterman R and Perneel C . 2019 . Hypersectral imaging for military and security applications: combining myriad processing and sensing techniques . IEEE Geoscience and Remote Sensing Magazine , 7 ( 2 ): 101 - 117 . [ DOI: 10.1109/MGRS.2019.2902525 http://dx.doi.org/10.1109/MGRS.2019.2902525 ]

Sun W and Du Q . 2018 . Graph-regularized fast and robust principal component analysis for hyperspectral band selection . IEEE Transactions on Geoscience and Remote Sensing , 56 ( 6 ): 3185 - 3195 . [ DOI: 10.1109/TGRS.2018.2794443 http://dx.doi.org/10.1109/TGRS.2018.2794443 ]

Sun W W and Du Q . 2019 . Hyperspectral band selection: a review . IEEE Geoscience and Remote Sensing Magazine , 7 ( 2 ): 118 - 139 . [ DOI: 10.1109/MGRS.2019.2911100 http://dx.doi.org/10.1109/MGRS.2019.2911100 ]

Vaswani A , Shazeer N , Parmar N , Uszkoreit J , Jones L , Gomez A N , Kaiser Ł and Polosukhin I . 2017 . Attention is all you need // Proceedings of the 31st International Conference on Neural Information Processing Systems . Long Beach : Curran Associates Inc.

Wang J and Chang C I . 2006 . Independent component analysis-based dimensionality reduction with applications in hyperspectral image analysis . IEEE Transactions on Geoscience and Remote Sensing , 44 ( 6 ): 1586 - 1600 . [ DOI: 10.1109/TGRS.2005.863297 http://dx.doi.org/10.1109/TGRS.2005.863297 ]

Wang W J , Dou S G , Jiang Z M and Sun L J . 2018 . A fast dense spectral-spatial convolution network framework for hyperspectral images classification . Remote Sensing , 10 ( 7 ): 1068 . [ DOI: 10.3390/rs10071068 http://dx.doi.org/10.3390/rs10071068 ]

Zhang X R , Sun Y J , Jiang K , Li C , Jiao L C and Zhou H Y . 2018 . Spatial sequential recurrent neural network for hyperspectral image classification . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 11 ( 11 ): 4141 - 4155 . [ DOI: 10.1109/JSTARS.2018.2844873 http://dx.doi.org/10.1109/JSTARS.2018.2844873 ]

Zhong Z L , Li J , Luo Z M and Chapman M . 2018 . Spectral-spatial residual network for hyperspectral image classification: a 3-d deep learning framework . IEEE Transactions on Geoscience and Remote Sensing , 56 ( 2 ): 847 - 858 . [ DOI: 10.1109/TGRS.2017.2755542 http://dx.doi.org/10.1109/TGRS.2017.2755542 ]

Zhou Y C , Peng J T and Chen C L P . 2015 . Extreme learning machine with composite kernels for hyperspectral image classification . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 8 ( 6 ): 2351 - 2360 . [ DOI: 10.1109/JSTARS.2014.2359965 http://dx.doi.org/10.1109/JSTARS.2014.2359965 ]

Zhu L , Chen Y S , Ghamisi P and Benediktsson J A . 2018 . Generative adversarial networks for hyperspectral image classification . IEEE Transactions on Geoscience and Remote Sensing , 56 ( 9 ): 5046 - 5063 [ DOI: 10.1109/TGRS.2018.2805286 http://dx.doi.org/10.1109/TGRS.2018.2805286 ]

Alert me when the article has been cited

提交

Multimodal object detection method using adaptive dusion of infrared and visible features

Stripe noise removal in high resolution satellite remote sensing images based on deep learning

Building extraction in pixel level from aerial imagery with a deep encoder-decoder network