联合分离卷积与密集连接轻量级神经网络的高光谱图像分类

宋廷强; 宗达; 刘童心; 范海生; 黄腾杰; 蒋晓旭; 王浩宇

doi:10.11834/jrs.20210313

遥感智能解译 | 浏览量 : 0 下载量: 995 CSCD: 1

R-PDF
PDF
导出
分享
收藏
专辑

联合分离卷积与密集连接轻量级神经网络的高光谱图像分类
Hyperspectral image classification of the deep neural network based on 3D convolution and dense connection
2022年26卷第11期页码：2317-2328
收稿：2020-08-11，

纸质出版：2022-11-07
DOI： 10.11834/jrs.20210313
稿件说明：

移动端阅览

宋廷强，宗达，刘童心，范海生，黄腾杰，蒋晓旭，王浩宇.2022.联合分离卷积与密集连接轻量级神经网络的高光谱图像分类.遥感学报，26（11）： 2317-2328 DOI： 10.11834/jrs.20210313.

Song T Q，Zong D，Liu T X，Fan H S，Huang T J，Jiang X X and Wang H Y. 2022. Hyperspectral image classification of the deep neural network based on 3D convolution and dense connection. National Remote Sensing Bulletin， 26（11）：2317-2328 DOI： 10.11834/jrs.20210313.

摘要

针对高光谱遥感图像空间分辨率低，标注训练样本困难的问题，本文提出一种基于分离卷积（Separable convolution）与密集连接（Dense connection）的轻量级神经网络SDLN模型。该模型基于DenseNet的思想，同时采用计算量更少的分离卷积代替3D卷积，根据算法提取的光谱信息和空间信息，结合目标及周围像素信息推断其中心像素内容，实现对单像素的分类。基于IP、PU和KSC这3个广泛使用的高光谱数据集进行实验，按照分层抽样的方法，每个类别选取少量样本作为训练集，分类精度分别达到了97.4%、97.6%、99.2%，与SSRN、SVM-RBF、MDGCN、DBDA及pResNet多种先进分类算法对比，分类精度提高且时间成本降低。

Abstract

With the progress of deep learning

researchers are increasingly paying attention to its application in hyperspectral image classification. Many experiments are conducted to achieve a trade-off between accuracy and efficiency to improve the feature extraction performance of neural networks toward small training sample sets.

This work has proposed a high-speed and high-precision neural network structure based on spatial spectral information. A cascaded neural network for spectral spatial information extraction is constructed by combining the idea of DenseNet and adopting dilated convolutions instead of 3D convolutions as the main calculation method. The whole network structure is divided into four components: spectral information extraction

spectral compression

fusion of spatial and spectral information

and voting solution.

Three convolutional layers are built in the spectral information extraction component. In each layer

1×1×7 convolution kernels are used to extract spectral information and maintain the independence of spatial information. The number of kernels is set to 60. In light of the DenseNet idea

the network outputs of the first and second layers are dimensionally split in spectrum and inputted into the third layer. The outputs of the first

second

and third layers are also dimensionally split and inputted into the spectral compression component.

In the spectral compression component

a 1×1×7 convolution kernel is used with a step size set to three. The spectral dimension is compressed

and the number of parameters of the deeper network is lessened by reducing the size of the feature map.

In the spatial and spectral information fusion component

the goal is to fuse spatial information for the first time with 3×3 receptive fields and integrate the spectral information of the data. Separable convolutions are adopted instead of traditional 3D convolutions

and the 3×3×K convolution kernel is decomposed into a 3×3×1 convolution and a 1×1×K convolution. The value of K is equal to the spectral dimension of the input feature map. Then

40 9×9×1 feature maps are outputted.

Voting means that if the output of most pixels is the same value

then the average value of all values will also be pulled near this certain value. In the voting solution component using parameter-free global average pooling

the 9×9×1 feature maps are voted to obtain 1×1×1 output values. These 40 output values are spliced into the fully connected layer

and the classification results our outputted through Softmax.

A series of experiments were carried out on the Indian Pains and Pavia University and Kennedy Space Center datasets. In the IP data set

the average accuracy reaches 95.0%

the overall accuracy 97.4%

and Kappa 0.97 by training with 5% data sets. In the UP data set

and Kappa reach 97.6%

97.1%

and 0.97

respectively

by training with a 0.5% data set. The overall accuracy in the KSC data set can reach 99.2%. The network has been proven to strong feature extraction and classification ability.

This method effectively improves the classification accuracy of hyperspectral images in the case of small sample sets and studies the effect of training and input data sizes on the classification accuracy. The classification accuracy of the network is improved with the increase in the training or input data. However

redundant information generated by a large amount of training data and excessive input data does not help improve the classification performance.

关键词

Keywords

references

Archibald R and Fann G . 2007 . Feature selection and classification of hyperspectral images with support vector machines . IEEE Geoscience and Remote Sensing Letters , 4 ( 4 ): 674 - 677 [ DOI: 10.1109/LGRS.2007.905116 http://dx.doi.org/10.1109/LGRS.2007.905116 ]

Ball J E and Wei P . 2018 . Deep learning hyperspectral image classification using multiple class-based denoising autoencoders , mixed pixel training augmentation, and morphological operations// IGARSS 2018 — 2018 IEEE International Geoscience and Remote Sensing Symposium. Valencia : IEEE: 6903 - 6906 [ DOI: 10.1109/IGARSS.2018.8519368 http://dx.doi.org/10.1109/IGARSS.2018.8519368 ]

Chang C I , Zhao X L , Althouse M L G and Pan J J . 1998 . Least squares subspace projection approach to mixed pixel classification for hyperspectral images . IEEE Transactions on Geoscience and Remote Sensing , 36 ( 3 ): 898 - 912 [ DOI: 10.1109/36.673681 http://dx.doi.org/10.1109/36.673681 ]

Cui B G , Ma X D and Xie X Y , 2017 . Hyperspectral image de-noising and classification with small training samples . Journal of Remote Sensing , 21 ( 5 ): 728 - 738

崔宾阁 , 马秀丹 , 谢小云 . 2017 . 小样本的高光谱图像降噪与分类 . 遥感学报 , 21 ( 5 ): 728 - 738 [ DOI： 10.11834/jrs.20176239 http://dx.doi.org/10.11834/jrs.20176239 ]

Fu W , Li S T and Fang L Y . 2015 . Spectral-spatial hyperspectral image classification via superpixel merging and sparse representation // 2015 IEEE International Geoscience and Remote Sensing Symposium . Milan : IEEE: 4971 - 4974 [ DOI: 10.1109/IGARSS.2015.7326948 http://dx.doi.org/10.1109/IGARSS.2015.7326948 ]

Gao Q S , Lim S and Jia X P . 2018 . Hyperspectral image classification using convolutional neural networks and multiple feature learning . Remote Sensing , 10 ( 2 ): 299 [ DOI: 10.3390/rs10020299 http://dx.doi.org/10.3390/rs10020299 ]

He M Y , Li B and Chen H H . 2017 . Multi-scale 3D deep convolutional neural network for hyperspectral image classification // 2017 IEEE International Conference on Image Processing (ICIP) . Beijing : IEEE: 3904 - 3908 [ DOI: 10.1109/ICIP.2017.8297014 http://dx.doi.org/10.1109/ICIP.2017.8297014 ]

Huang G , Liu Z , Laurens V and Weinberger K . Q . 2017 . Densely connected convolutional networks// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 1063 - 6919 [ DOI: 10.1109/CVPR.2017.243 http://dx.doi.org/10.1109/CVPR.2017.243 ]

Howard A G , Zhu M L , Chen B , Kalenichenko D , Wang W J , Weyand T , Andreetto M and Adam H . 2017 . MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications . ArXiv Preprint ArXiv : 1704 . 04861 [ DOI： 10.48550/arXiv.1704.04861 http://dx.doi.org/10.48550/arXiv.1704.04861 ]

Ioffe S and Szegedy C . 2015 . Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift . ArXiv Preprint ArXiv : 1502 . 03167 [ DOI： 10.48550/arXiv.1502.03167 http://dx.doi.org/10.48550/arXiv.1502.03167 ]

Jiao L C , Liang M M , Chen H , Yang S Y , Liu H Y and Cao X H . 2017 . Deep fully convolutional network-based spatial distribution prediction for hyperspectral image classification . IEEE Transactions on Geoscience and Remote Sensing , 55 ( 10 ): 5585 - 5599 [ DOI: 10.1109/TGRS.2017.2710079 http://dx.doi.org/10.1109/TGRS.2017.2710079 ]

Joelsson S R , Benediktsson J A and Sveinsson J R . 2005 . Random forest classifiers for hyperspectral data // Proceedings of IEEE International Geoscience and Remote Sensing Symposium , 2005(IGARSS ' 05 ) [ DOI: 10.1109/IGARSS.2005.1526129 http://dx.doi.org/10.1109/IGARSS.2005.1526129 ]

Kirsch M , Lorenz S , Zimmermann R , Tusa L , Möckel R , Hödl P , Booysen R , Khodadadzadeh M and Gloaguen R . 2018 . Integration of terrestrial and drone-borne hyperspectral and photogrammetric sensing methods for exploration mapping and mining monitoring . Remote Sensing , 10 ( 9 ): 1366 - 1366 [ DOI: 10.3390/rs10091366 http://dx.doi.org/10.3390/rs10091366 ]

Krizhevsky A , Sutskever I and Hinton G E . 2012 . ImageNet classification with deep convolutional neural networks // Proceedings of the 25th International Conference on Neural Information Processing Systems . Lake Tahoe, Nevada : Curran Associates Inc.: 1097 - 1105 [ DOI: 10.1145/3065386 http://dx.doi.org/10.1145/3065386 ]

Li F , Lu H C and Zhang P P . 2019a . An innovative multi-kernel learning algorithm for hyperspectral classification . Computers and Electrical Engineering , 79 : 106456 - 106464 [ DOI: 10.1016/j.compeleceng.2019.106456 http://dx.doi.org/10.1016/j.compeleceng.2019.106456 ]

Li R , Zheng S Y , Duan C X , Yang Y and Wang X Q . 2020 . Classification of hyperspectral image based on double-branch dual-attention mechanism network . Remote Sensing , 12 ( 3 ): 582 [ DOI: 10.3390/rs12030582 http://dx.doi.org/10.3390/rs12030582 ]

Li W , Chen C , Su H J and Du Q . 2015 . Local binary patterns and extreme learning machine for hyperspectral imagery classification . IEEE Transactions on Geoscience and Remote Sensing , 53 ( 7 ): 3681 - 3693 [ DOI: 10.1109/TGRS.2014.2381602 http://dx.doi.org/10.1109/TGRS.2014.2381602 ]

Li X , Chen S , Hu X L and Yang J . 2019b . Understanding the disharmony between dropout and batch normalization by variance shift // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Long Beach, CA : IEEE: 2677 - 2685 [ DOI: 10.1109/CVPR.2019.00279 http://dx.doi.org/10.1109/CVPR.2019.00279 ]

Liu X F , Sun Q Q , Meng Y , Fu M and Bourennane S . 2018 . Hyperspectral image classification based on parameter-optimized 3D-CNNs combined with transfer learning and virtual samples . Remote Sensing , 10 ( 9 ): 1425 - 1425 [ DOI: 10.3390/rs10091425 http://dx.doi.org/10.3390/rs10091425 ]

Makantasis K , Karantzalos K , Doulamis A and Doulamis N . 2015 . Deep supervised learning for hyperspectral data classification through convolutional neural networks // 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS) . Milan : IEEE: 4959 - 4962 [ DOI: 10.1109/IGARSS.2015.7326945 http://dx.doi.org/10.1109/IGARSS.2015.7326945 ]

Mei S H , Ji J Y , Hou J H , Li X and Du Q . 2017 . Learning sensor-specific spatial-spectral features of hyperspectral images via convolutional neural networks . IEEE Transactions on Geoscience and Remote Sensing , 55 ( 8 ): 4520 - 4533 [ DOI: 10.1109/TGRS.2017.2693346 http://dx.doi.org/10.1109/TGRS.2017.2693346 ]

Melgani F and Bruzzone L . 2004 . Classification of hyperspectral remote sensing images with support vector machines . IEEE Transactions on Geoscience and Remote Sensing , 42 ( 8 ): 1778 - 1790 [ DOI: 10.1109/TGRS.2004.831865 http://dx.doi.org/10.1109/TGRS.2004.831865 ]

Paoletti M E , Haut J M , Fernandez-Beltran R , Plaza J , Plaza A J and Pla F . 2019 . Deep pyramidal residual networks for spectral-spatial hyperspectral image classification . IEEE Transactions on Geoscience and Remote Sensing , 57 ( 2 ): 740 - 754 [ DOI: 10.1109/TGRS.2018.2860125 http://dx.doi.org/10.1109/TGRS.2018.2860125 ]

Prey L , von Bloh M and Schmidhalter U . 2018 . Evaluating RGB imaging and multispectral active and hyperspectral passive sensing for assessing early plant vigor in winter wheat . Sensors , 18 ( 9 ): 2931 [ DOI: 10.3390/s18092931 http://dx.doi.org/10.3390/s18092931 ]

Rao M , Tang P and Zhang Z . 2020 . A developed siamese CNN with 3D adaptive spatial-spectral pyramid pooling for hyperspectral image classification . Remote Sensing , 12 ( 12 ): 1964 [ DOI： 10.3390/rs12121964 http://dx.doi.org/10.3390/rs12121964 ]

Redmon J and Farhadi A . 2018 . YOLOv3: An Incremental Improvement . ArXiv E-prints [ DOI： 10.48550/arXiv.1804.02767 http://dx.doi.org/10.48550/arXiv.1804.02767 ]

Santurkar S , Tsipras D , Ilyas A , Madry A . 2018 . How does batch normalization help optimization? // NeurIPS 2018 [ DOI: 10.48550/arXiv.1805.11604 http://dx.doi.org/10.48550/arXiv.1805.11604 ]

Simonyan K and Zisserman A . 2014 . Very Deep Convolutional Networks for Large-scale Image Recognition // 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) [ DOI： 10.48550/arXiv.1409.1556 http://dx.doi.org/10.48550/arXiv.1409.1556 ]

Srivastava N , Hinton G , Krizhevsky A , Sutskever I and Salakhutdinov R . 2014 . Dropout: a simple way to prevent neural networks from overfitting . The Journal of Machine Learning Research , 15 ( 1 ): 1929 - 1958 [ DOI： 10.1109/cisp.2015.7407967 http://dx.doi.org/10.1109/cisp.2015.7407967 ]

Sun W W , Zhang D F , Yang G and Li W Y . 2018 . Band selection for hyperspectral imagery based on weighted probabilistic archetypal analysis . Journal of Remote Sensing , 22 ( 1 ): 110 - 118

孙伟伟 , 张殿发 , 杨刚 , 李巍岳 . 2018 . 加权概率原型分析的高光谱影像波段选择 . 遥感学报 , 22 ( 1 ): 110 - 118 [ DOI: 10.11834/jrs.20186446 http://dx.doi.org/10.11834/jrs.20186446 ]

Szegedy C , Liu W , Jia Y Q , Sermanet P , Reed S , Anguelov D , Erhan D , Vanhoucke V and Rabinovich A . 2015 . Going deeper with convolutions // 2015 IEEE Conference on Computer Vision and Pattern Recognition . Boston, MA : IEEE: 1 - 9 [ DOI: 10.1109/CVPR.2015.7298594 http://dx.doi.org/10.1109/CVPR.2015.7298594 ]

Wan S , Gong C , Zhong P , Du B , Zhang L F and Yang J . 2020 . Multiscale dynamic graph convolutional network for hyperspectral image classification . IEEE Transactions on Geoscience and Remote Sensing , 58 ( 5 ): 3162 - 3177 [ DOI: 10.1109/TGRS.2019.2949180 http://dx.doi.org/10.1109/TGRS.2019.2949180 ]

Wang L Q , Zhao X and Qin Z C . 2021 . Hyperspectral image classification with spectral-spatial consistency regularization . Journal of China Academy of Electronics and Information Technology , 16 ( 8 ): 789 - 796

王雷全 , 赵欣 , 秦智超 . 2021 . 基于光谱—空间一致性正则化的高光谱图像分类 . 中国电子科学研究院学报 , 16 ( 8 ): 789 - 796 [ DOI： 10.3969/j.issn.1673-5692.2021.08.007 http://dx.doi.org/10.3969/j.issn.1673-5692.2021.08.007 ]

Zhang K , Hei B Q , Zhou Z and Li S Y . 2018 . CNN with coefficient of variation-based dimensionality reduction for hyperspectral remote sensing images classification . Journal of Remote Sensing , 22 ( 1 ): 87 - 96

张康 , 黑保琴 , 周壮 , 李盛阳 . 2018 . 变异系数降维的CNN高光谱遥感图像分类 . 遥感学报 , 22 ( 1 ): 87 - 96 [ DOI： 10.11834/jrs.20187075 http://dx.doi.org/10.11834/jrs.20187075 ]

Zheng Q , Huang W J , Cui X M , Dong Y Y , Shi Y , Ma H Q and Liu L Y . 2018 . Identification of wheat yellow rust using optimal three-band spectral indices in different growth stages . Sensors , 19 ( 1 ): 35 [ DOI: 10.3390/s19010035 http://dx.doi.org/10.3390/s19010035 ]

Zhong Z L , Li J , Luo Z M and Chapman M . 2018 . Spectral-spatial residual network for hyperspectral image classification: a 3-D deep learning framework . IEEE Transactions on Geoscience and Remote Sensing , 56 ( 2 ): 847 - 858 [ DOI: 10.1109/TGRS.2017.2755542 http://dx.doi.org/10.1109/TGRS.2017.2755542 ]

Zhou Y J and Tian Q J . 2008 . Image quality evaluation of EO-1 hyperion sensor . Geo-information Science , 10 ( 5 ): 678 - 683

周雨霁 , 田庆久 . 2008, EO-1 Hyperion高光谱数据的质量评价 . 地球信息科学 , 10 ( 5 ): 678 - 683 [ DOI: 10.3969/j.issn.1560-8999.2008.05.020 http://dx.doi.org/10.3969/j.issn.1560-8999.2008.05.020 ]

文章被引用时，请邮件提醒。

提交

光谱可形变卷积驱动的高光谱图像分类

岩溶湿地无人机LiDAR点云精细3D制图及植被空间分异研究

LAE-Transformer：融合局部感知增强的机载LiDAR点云分割网络

融合自适应增强与动态筛选机制的多尺度InSAR相位滤波方法