光谱可形变卷积驱动的高光谱图像分类

薛朝辉; 李博

doi:10.11834/jrs.20221798

遥感智能解译 | 浏览量 : 0 下载量: 1187 CSCD: 1

R-PDF
PDF
导出
分享
收藏
专辑

光谱可形变卷积驱动的高光谱图像分类
Hyperspectral image classification based on spectral deformable convolution neural network
2022年26卷第10期页码：2014-2028
收稿：2021-12-07，

纸质出版：2022-10-07
DOI： 10.11834/jrs.20221798
稿件说明：

移动端阅览

薛朝辉，李博.2022.光谱可形变卷积驱动的高光谱图像分类.遥感学报，26（10）： 2014-2028 DOI： 10.11834/jrs.20221798.

Xue Z H and Li B. 2022. Hyperspectral image classification based on spectral deformable convolution neural network. National Remote Sensing Bulletin， 26（10）：2014-2028 DOI： 10.11834/jrs.20221798.

摘要

基于卷积神经网络的高光谱图像分类是当前的研究热点，先后发展了空洞卷积、可形变卷积等先进模型。然而，现有可形变卷积只在空间维偏移，忽略了高光谱图像光谱之间的差异信息。为此，本文将可形变卷积从空间维扩展到光谱维，设计了光谱可形变卷积，提出了光谱可形变卷积网络SDCNN（Spectral Deformable Convolutional Neural Network）。首先，利用全连接层学习光谱可形变卷积的偏移量，采用线性差值对图像光谱维进行特征校准；其次，采用多层1×1卷积进行光谱维特征聚合；最后，使用三维卷积层提取光谱—空间联合特征。不同于空间可形变卷积，光谱可形变卷积只在光谱维上进行偏移，可以为不同类别选择更合适的特征波段，提升模型的判别性。在国际通用测试数据Indian Pines、University of Pavia以及University of Houston上进行了实验，结果表明：本文提出的SDCNN方法优于其他深度学习方法，在相同样本条件下取得了更高的分类精度，总体精度达到了98.86%（Indian Pines，10%/类）、99.81%（University of Pavia，5%/类）以及97.41%（University of Houston，50个/类），验证了该方法的有效性。

Abstract

Currently

convolutional neural network is a research hot spot in hyperspectral image classification. Some advanced models such as dilated convolution and deformable convolution have been developed successfully. However

the existing deformable convolution modules only shift in spatial domain and ignore the spectral difference information. Therefore

this paper proposed a novel deformable convolution

which extends spatial deformable to the spectral domain. In addition

a spectral deformable convolutional neural network (SDCNN) is proposed for hyperspectral image classification. In view of the existing problems in the application of spatial deformable convolution in HSI classification

this paper extends the deformable convolution to the spectral dimension

and proposes the spectral deformable convolution. Given that different feature categories may have different classification effects

more appropriate classification bands can be selected for different class by the learnt shifts in spectral domain. Therefore

the spectral feature extraction can focus on more effective band

thereby promoting more discriminative features. In this way

only one direction of spectral dimension needs to be learned

which reduces computational complexity

that is

only half of the occupying time than that of spatial deformable convolution. First

a full connection layer is used to learn the offset of the spectral deformable convolution

and a linear difference is used to perform feature correction in spectral domain. Second

a multilayer 1×1 convolution is used for spectral feature aggregation. Finally

a 3D convolution layer is used to extract spectral–spatial features. Experiments were conducted on three international popular datasets including Indian Pines

University of Pavia

and University of Houston. The experimental results demonstrate that the SDCNN is superior to other deep learning methods. SDCNN yields the highest classification accuracy with an overall accuracy of 98.86% (Indian pines

10%/class)

99.81% (University of Pavia

5%/class)

and 97.41% (University of Houston

50/class)

which can well verify the effectiveness of the proposed model. First

through comprehensive experiments

SDCNN yields the highest accuracy compared with other existing models on three international popular datasets

which can prove the validity of the SDCNN method. Second

the effectiveness of the spectral deformable convolution module is proven by comparing the traditional dilated convolution module and the spatial deformable convolution module. SDCNN generalizes better with limited training samples

especially when the number of labeled samples is small

The stability of classification performance of SDCNN method is proved;. Finally

In the experiment of separating samples

SDCNN method achieves the best classification effect compared with other methods

and has obvious accuracy advantages compared with spatial deformable convolution method

which proves that SDCNN method has less dependence on spatial features and stronger feature extraction ability.

关键词

Keywords

references

Belgiu M and Drăguţ L . 2016 . Random forest in remote sensing: a review of applications and future directions . ISPRS Journal of Photogrammetry and Remote Sensing , 114 : 24 - 31 [ DOI: 10.1016/j.isprsjprs.2016.01.011 http://dx.doi.org/10.1016/j.isprsjprs.2016.01.011 ]

Camps-Valls G , Gomez-Chova L , Munoz-Mari J , Vila-Frances J and Calpe-Maravilla J . 2006 . Composite kernels for hyperspectral image classification . IEEE Geoscience and Remote Sensing Letters , 3 ( 1 ): 93 - 97 [ DOI: 10.1109/LGRS.2005.857031 http://dx.doi.org/10.1109/LGRS.2005.857031 ]

Chen X , Li Z , Jiang Jet al . 2021 . Adaptive Effective Receptive Field Convolution for Semantic Segmentation of VHR Remote Sensing Images . IEEE Transactions on Geoscience and Remote Sensing , 59 ( 4 ): 3532 - 3546 [ DOI: 10.1109/TGRS.2020.3009143 http://dx.doi.org/10.1109/TGRS.2020.3009143 ]

Chen Y , Nasrabadi N M and Tran T D . 2011 . Hyperspectral image classification using dictionary-based sparse representation . IEEE Transactions on Geoscience and Remote Sensing , 49 ( 10 ): 3973 - 3985 [ DOI: 10.1109/TGRS.2011.2129595 http://dx.doi.org/10.1109/TGRS.2011.2129595 ]

Chen Y S , Lin Z H , Zhao X , Wang G and Gu Y F . 2014 . Deep learning-based classification of hyperspectral data . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 7 ( 6 ): 2094 - 2107 [ DOI: 10.1109/JSTARS.2014.2329330 http://dx.doi.org/10.1109/JSTARS.2014.2329330 ]

Chen Y S , Jiang H L , Li C Y , Jia X P and Ghamisi P . 2016 . Deep feature extraction and classification of hyperspectral images based on convolutional neural networks . IEEE Transactions on Geoscience and Remote Sensing , 54 ( 10 ): 6232 - 6251 [ DOI: 10.1109/TGRS.2016.2584107 http://dx.doi.org/10.1109/TGRS.2016.2584107 ]

Dai J F , Qi H Z , Xiong Y W , Li Y , Zhang G D , Hu H and Wei Y C . 2017 . Deformable convolutional networks // 2017 IEEE International Conference on Computer Vision (ICCV) . Venice : IEEE: 764 - 773 [ DOI: 10.1109/ICCV.2017.89 http://dx.doi.org/10.1109/ICCV.2017.89 ]

Du P J , Xia J S , Xue Z H , Tan K , Su H J and Bao R . 2016 . Review of hyperspectral remote sensing image classification . Journal of Remote Sensing , 20 ( 2 ): 236 - 256

杜培军 , 夏俊士 , 薛朝辉 , 谭琨 , 苏红军 , 鲍蕊 . 2016 . 高光谱遥感影像分类研究进展 . 遥感学报 , 20 ( 2 ): 236 - 256 [ DOI: 10.11834/jrs.20165022 http://dx.doi.org/10.11834/jrs.20165022 ]

Fauvel M , Tarabalka Y , Benediktsson J A , Chanussot J and Tilton J C . 2013 . Advances in spectral-spatial classification of hyperspectral images . Proceedings of the IEEE , 101 ( 3 ): 652 - 675 [ DOI: 10.1109/JPROC.2012.2197589 http://dx.doi.org/10.1109/JPROC.2012.2197589 ]

Feng J , Wu X D , Shang R H , Sui C , Li J , Jiao L C and Zhang X R . 2021 . Attention multibranch convolutional neural network for hyperspectral image classification based on adaptive region search . IEEE Transactions on Geoscience and Remote Sensing , 59 ( 6 ): 5054 - 5070 [ DOI: 10.1109/TGRS.2020.3011943 http://dx.doi.org/10.1109/TGRS.2020.3011943 ]

Hinton G E and Salakhutdinov R R . 2006 . Reducing the dimensionality of data with Neural Networks . Science , 313 ( 5786 ): 504 - 507 [ DOI: 10.1126/science.1127647 http://dx.doi.org/10.1126/science.1127647 ]

Huang H and Qu H P . 2014 . Hyperspectral remote sensing image classification based on SSDE . Optics and Precision Engineering , 22 ( 2 ): 434 - 442

黄鸿 , 曲焕鹏 . 2014 . 基于半监督稀疏鉴别嵌入的高光谱遥感影像分类 . 光学精密工程 , 22 ( 2 ): 434 - 442 [ DOI: 10.3788/OPE.20142202.0434 http://dx.doi.org/10.3788/OPE.20142202.0434 ]

Hu W , Huang Y Y , Wei L , Zhang F and Li H C . 2015 . Deep convolutional neural networks for hyperspectral image classification . Journal of Sensors , 2015 : 258619 [ DOI: 10.1155/2015/258619 http://dx.doi.org/10.1155/2015/258619 ]

Li C M , Qiu Z L , Cao X Y , Chen Z H , Gao H M and Hua Z J . 2021 . Hybrid dilated convolution with multi-scale residual fusion network for hyperspectral image classification . Micromachines , 12 ( 5 ): 545 [ DOI: 10.3390/mi12050545 http://dx.doi.org/10.3390/mi12050545 ]

Li J , Bioucas-Dias J M and Plaza A . 2010 . Semisupervised hyperspectral image segmentation using multinomial logistic regression with active learning . IEEE Transactions on Geoscience and Remote Sensing , 48 ( 11 ): 4085 - 4098 [ DOI: 10.1109/TGRS.2010.2060550 http://dx.doi.org/10.1109/TGRS.2010.2060550 ]

Li W , Du Q and Xiong M M . 2015 . Kernel collaborative representation with Tikhonov regularization for hyperspectral image classification . IEEE Geoscience and Remote Sensing Letters , 12 ( 1 ): 48 - 52 [ DOI: 10.1109/LGRS.2014.2325978 http://dx.doi.org/10.1109/LGRS.2014.2325978 ]

Li Z K , Wang T N , Li W , Du Q , Wang C Y , Liu C W and Shi X B . 2020 . Deep multilayer fusion dense network for hyperspectral image classification . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 13 : 1258 - 1270 [ DOI: 10.1109/JSTARS.2020.2982614 http://dx.doi.org/10.1109/JSTARS.2020.2982614 ]

Liu G C , Lin Z C , Yan S C , Sun J , Yu Y and Ma Y . 2013 . Robust recovery of subspace structures by low-rank representation . IEEE Transactions on Pattern Analysis and Machine Intelligence , 35 ( 1 ): 171 - 184 [ DOI: 10.1109/TPAMI.2012.88 http://dx.doi.org/10.1109/TPAMI.2012.88 ]

Lowe A , Harrison N and French A P . 2017 . Hyperspectral image analysis techniques for the detection and classification of the early onset of plant disease and stress . Plant Methods , 13 ( 1 ): 1 - 12 [ DOI: 10.1186/s13007-017-0233-z http://dx.doi.org/10.1186/s13007-017-0233-z ]

Lu B , Dao P D , Liu J G , He Y H and Shang J L . 2020 . Recent advances of hyperspectral imaging technology and applications in agriculture . Remote Sensing , 12 ( 16 ): 1 - 44 [ DOI: 10.3390/rs12162659 http://dx.doi.org/10.3390/rs12162659 ]

Melgani F and Bruzzone L . 2004 . Classification of hyperspectral remote sensing images with support vector machines . IEEE Transactions on Geoscience and Remote Sensing , 42 ( 8 ): 1778 - 1790 [ DOI: 10.1109/TGRS.2004.831865 http://dx.doi.org/10.1109/TGRS.2004.831865 ]

Mou L C , Ghamisi P and Zhu X X . 2017 . Deep recurrent neural networks for hyperspectral image classification . IEEE Transactions on Geoscience and Remote Sensing , 55 ( 7 ): 3639 - 3655 [ DOI: 10.1109/TGRS.2016.2636241 http://dx.doi.org/10.1109/TGRS.2016.2636241 ]

Mou L C and Zhu X X . 2020 . Learning to pay attention on spectral domain: a spectral attention module-based convolutional network for hyperspectral image classification . IEEE Transactions on Geoscience and Remote Sensing , 58 ( 1 ): 110 - 122 [ DOI: 10.1109/TGRS.2019.2933609 http://dx.doi.org/10.1109/TGRS.2019.2933609 ]

Paoletti M E , Haut J M , Plaza J and Plaza A . 2019 . Deep learning classifiers for hyperspectral imaging: a review . ISPRS Journal of Photogrammetry and Remote Sensing , 158 : 279 - 317 [ DOI: 10.1016/j.isprsjprs.2019.09.006 http://dx.doi.org/10.1016/j.isprsjprs.2019.09.006 ]

Pooja K , Nidamanuri R R and Mishra D . 2019 . Multi-scale dilated residual convolutional neural network for hyperspectral image classification // 2019 10th Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing (WHISPERS) . Amsterdam : IEEE: 1 - 5 [ DOI: 10.1109/WHISPERS.2019.8921284 http://dx.doi.org/10.1109/WHISPERS.2019.8921284 ]

Tong Q X , Zhang B and Zhang L F . 2016 . Current progress of hyperspectral remote sensing in China . Journal of Remote Sensing , 20 ( 5 ): 689 - 707

童庆禧 , 张兵 , 张立福 . 2016 . 中国高光谱遥感的前沿进展 . 遥感学报 , 20 ( 5 ): 689 - 707 [ DOI: 10.11834/jrs.20166264 http://dx.doi.org/10.11834/jrs.20166264 ]

van der Meer F D , van der Werff H M A , van Ruitenbeek F J A , Hecker C A , Bakker W H , Noomen M F , Van Der meijde M , Carranza E J M , Smeth J B D and Woldai T . 2012 . Multi- and hyperspectral geologic remote sensing: a review . International Journal of Applied Earth Observation and Geoinformation , 14 ( 1 ): 112 - 128 [ DOI: 10.1016/j.jag.2011.08.002 http://dx.doi.org/10.1016/j.jag.2011.08.002 ]

Wang P Q , Chen P F , Yuan Y , Liu D , Huang Z H , Hou X D and Cottrell G . 2018 . Understanding convolution for semantic segmentation // 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) . Lake Tahoe, NV : IEEE: 1451 - 1460 [ DOI: 10.1109/WACV.2018.00163 http://dx.doi.org/10.1109/WACV.2018.00163 ]

Wei X P , Yu X C , Zhang P Q , Zhi L and Yang F . 2020 . CNN with local binary patterns for hyperspectral images classification . Journal of Remote Sensing , 24 ( 8 ): 1000 - 1009

魏祥坡 , 余旭初 , 张鹏强 , 职露 , 杨帆 . 2020 . 联合局部二值模式的CNN高光谱图像分类 . 遥感学报 , 24 ( 8 ): 1000 - 1009 [ DOI: 10.11834/jrs.20208333 http://dx.doi.org/10.11834/jrs.20208333 ]

Xu Y H , Du B and Zhang L P . 2020 . Beyond the patchwise classification: spectral-spatial fully convolutional networks for hyperspectral image classification . IEEE Transactions on Big Data , 6 ( 3 ): 492 - 506 [ DOI: 10.1109/TBDATA.2019.2923243 http://dx.doi.org/10.1109/TBDATA.2019.2923243 ]

Xu Y H , Zhang L P , Du B and Zhang F . 2018 . Spectral-spatial unified networks for hyperspectral image classification . IEEE Transactions on Geoscience and Remote Sensing , 56 ( 10 ): 5893 - 5909 [ DOI: 10.1109/TGRS.2018.2827407 http://dx.doi.org/10.1109/TGRS.2018.2827407 ]

Yu C Y , Han R , Song M P , Liu C Y and Chang C I . 2020 . A simplified 2D-3D CNN architecture for hyperspectral image classification based on spatial–spectral fusion . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 13 : 2485 - 2501 [ DOI: 10.1109/JSTARS.2020.2983224 http://dx.doi.org/10.1109/JSTARS.2020.2983224 ]

Zhang L P , Zhang L F and Du B . 2016 . Deep Learning for Remote Sensing Data: A Technical Tutorial on the State of the Art . IEEE Geoscience and Remote Sensing , 4 ( 2 ): 22 - 40 [ DOI: 10.1109/MGRS.2016.2540798 http://dx.doi.org/10.1109/MGRS.2016.2540798 ]

Zhang M M , Li W and Du Q . 2018 . Diverse region-based CNN for hyperspectral image classification . IEEE Transactions on Image Processing , 27 ( 6 ): 2623 - 2634 [ DOI: 10.1109/TIP.2018.2809606 http://dx.doi.org/10.1109/TIP.2018.2809606 ]

Zhao F , Zhang J J , Meng Z and Liu H Q . 2021 . Densely connected pyramidal dilated convolutional network for hyperspectral image classification . Remote Sensing , 13 ( 17 ): 3396 [ DOI: 10.3390/rs13173396 http://dx.doi.org/10.3390/rs13173396 ]

Zhao W Z and Du S H . 2016 . Spectral-spatial feature extraction for hyperspectral image classification: a dimension reduction and deep learning approach . IEEE Transactions on Geoscience and Remote Sensing , 54 ( 8 ): 4544 - 4554 [ DOI: 10.1109/TGRS.2016.2543748 http://dx.doi.org/10.1109/TGRS.2016.2543748 ]

Zhong Z L , Li J , Luo Z M and Chapman M . 2018 . Spectral-spatial residual network for hyperspectral image classification: a 3-D deep learning framework . IEEE Transactions on Geoscience and Remote Sensing , 56 ( 2 ): 847 - 858 [ DOI: 10.1109/TGRS.2017.2755542 http://dx.doi.org/10.1109/TGRS.2017.2755542 ]

Zhong Z L , Li J , Ma L F , Jiang H and Zhao H . 2017 . Deep residual networks for hyperspectral image classification // 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS) . Fort Worth, TX : IEEE: 1824 - 1827 [ DOI: 10.1109/IGARSS.2017.8127330 http://dx.doi.org/10.1109/IGARSS.2017.8127330 ]

Zhu J , Fang L Y and Ghamisi P . 2018 . Deformable convolutional neural networks for hyperspectral image classification . IEEE Geoscience and Remote Sensing Letters , 15 ( 8 ): 1254 - 1258 [ DOI: 10.1109/LGRS.2018.2830403 http://dx.doi.org/10.1109/LGRS.2018.2830403 ]

文章被引用时，请邮件提醒。

提交

可见光和红外特征自适应融合的多模态目标检测方法

MtSCCD：面向深度学习的土地利用场景分类与变化检测数据集

考虑光谱信息和超像素分割的高光谱解混网络

基于光谱—空间注意力双边网络的高光谱图像分类