CNN with local binary patterns for hyperspectral images classification

Xiangpo WEI; Xuchu YU; Pengqiang ZHANG; Lu ZHI; Fan YANG

doi:10.11834/jrs.20208333

Doctor's Voice | Views : 0 下载量: 1461 CSCD: 10

PDF
Export
Share
Collection
Album

CNN with local binary patterns for hyperspectral images classification
Vol. 24, Issue 8, Pages: 1000-1009(2020)
Received：14 August 2018，

Published：07 August 2020
DOI： 10.11834/jrs.20208333
稿件说明：

移动端阅览

魏祥坡,余旭初,张鹏强,职露,杨帆.2020.联合局部二值模式的CNN高光谱图像分类.遥感学报,24(8): 1000-1009 DOI： 10.11834/jrs.20208333.

Wei X P,Yu X C,Zhang P Q,Zhi L and Yang F. 2020. CNN with local binary patterns for hyperspectral images classification. Journal of Remote Sensing(Chinese)，24(8): 1000-1009[DOI：10.11834/jrs.20208333] DOI：

摘要

卷积神经网络CNN（Convolutional Neural Networks）具有强大的特征提取能力，应用于高光谱图像特征提取取得了良好的效果，双通道CNN模型能够分别提取高光谱图像的光谱特征和空间特征，并实现了特征的决策级融合。局部二值模式LBP（Local Binary Patterns）是一种简单但有效的空间特征描述算子，能够减轻CNN特征提取的压力并提高分类精度。为了充分利用CNN的特征提取能力及LBP特征的判别能力，提出一种双通道CNN和LBP相结合的高光谱图像分类方法，首先，采用1维CNN（1D-CNN）模型处理原始高光谱数据提取深层光谱特征，同时采用另一个1D-CNN模型处理LBP特征数据进一步提取深层空间特征，然后，将两个CNN模型的全连接层进行连接，实现深层光谱特征和空间特征的融合，并将融合特征输入到分类层中完成分类。实验结果表明，该方法在Indian Pines数据、Pavia University数据及Salinas数据上能够分别取得98.54%、99.73%、99.56%的分类精度，甚至在有限数量的训练样本条件下也能取得较好的分类效果。

Abstract

The classification of hyperspectral image remains a challenging task because of the complexity of spectral and spatial structures

high dimensionality

and strong correlation between adjacent bands. The combination of spatial and spectral information can provide significant advantage in terms of reducing the uncertainty of the samples because the same object has different spectrums and objects with the same spectrum in a hyperspectral image. The Local Binary Pattern (LBP) has also been introduced for spatial-domain feature extraction and classification of hyperspectral images as a simple but powerful texture descriptor. More recently

deep learning has been proven to be a preferable way to extract nonlinear high-level features because of its hierarchical learning framework. The combination of LBP features and the CNNs can lessen the workload of CNNs because of the discrimination capacity of LBP features. In this paper

a novel classification method combining DC–CNN and LBP features

called LBP Dual-Channel CNN (LBP–DC–CNN)

is proposed.

In LBP–DC–CNN

original hyperspectral data and LBP features are processed in a DC–CNN framework. On the one hand

original hyperspectral data is fed into a 1D–CNN model to extract original spectral features. On the other hand

LBP features are fed into an identical1D–CNN model to extract hierarchical spatial features further. Next

the fully connected layers of the two 1D–CNN models in the DC–CNN framework is concatenated into a fused layer

thus completing the fusion of spectral features and spatial features. Finally

the fused layer is fed into a softmax layer to conduct classification.

(1) The OAs of LBP–DC–CNN are better than those of LBP–CNN and DC–CNN

which validate the feature extraction capacity of the CNNs and the advantage of LBP features. LBP–DC–CNN provides better accuracy than that of DC–CNN

which is an advantage of LBP features compared with the spatial features extracted by 2D–CNN model. In addition

the accuracy of LBP–DC–CNN is better than that of LBP–CNN

which validates the reasonability and discriminative power of the dual-channel CNN framework.

(2) The OA of LBP–DC–CNN is apparently superior to those of compared methods

which makes DC–CNN and LBP features advantageous. For the Indian Pines data

LBP–DC–CNN (i.e.

98.54 %) yields approximately 2% higher accuracy than the DC–CNN (i.e.

96.68%)and approximately 4% higher accuracy than the LBP–CNN (i.e.

94.74 %). For the University of Pavia data

LBP–DC–CNN (i.e.

99.73 %) yields approximately 1 % higher accuracy than the DC–CNN (i.e.

98.74 %) and approximately 4 % higher accuracy than the LBP-CNN (i.e.

95.92 %). For the Salinas data

LBP–DC–CNN (i.e.

99.56 %) yields approximately 2 % higher accuracy than the DC–CNN (i.e.

97.33 %) and approximately 5 % higher accuracy than the LBP–CNN (i.e.

94.52 %).

(3) LBP–DC–CNN can improve the class-specific accuracy of some ground materials

such as Corn-notill and Soybean-mintill in the Indian Pines data

Asphalt and Bricks in the University of Pavia data

and Grapes_untrained and Vinyard_untrained in the Salinas data. LBP features aremore discriminative than spatial features extracted by 2D–CNN.

Result

Experiments were conducted on the Indian Pines dataset

Pavia University dataset

and Salinas dataset to verify the performance of LBP–DC–CNN compared with conventional methods. The results are as follows:

关键词

Keywords

references

Bioucas-Dias J M , Plaza A , Camps-Valls G , Scheunders P , Nasrabadi N and Chanussot J . 2013 . Hyperspectral remote sensing data analysis and future challenges . IEEE Geoscience and Remote Sensing Magazine , 1 ( 2 ): 6 - 36 [ DOI: 10.1109/MGRS.2013.2244672 http://dx.doi.org/10.1109/MGRS.2013.2244672 ]

Chen Y S , Jiang H L , Li C Y , Jia X P and Ghamisi P . 2016 . Deep feature extraction and classification of hyperspectral images based on convolutional neural networks . IEEE Transactions on Geoscience and Remote Sensing , 54 ( 10 ): 6232 - 6251 [ DOI: 10.1109/TGRS.2016.2584107 http://dx.doi.org/10.1109/TGRS.2016.2584107 ]

Chen Y S , Lin Z H , Zhao X , Wang G and Gu Y F . 2014 . Deep learning-based classification of hyperspectral data . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 7 ( 6 ): 2094 - 2107 [ DOI: 10.1109/JSTARS.2014.2329330 http://dx.doi.org/10.1109/JSTARS.2014.2329330 ]

Chen Y S , Zhao X and Jia X P . 2015 . Spectral–spatial classification of hyperspectral data based on deep belief network . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 8 ( 6 ): 2381 - 2392 [ DOI: 10.1109/JSTARS.2015.2388577 http://dx.doi.org/10.1109/JSTARS.2015.2388577 ]

Chen Y S , Zhu L , Ghamisi P , Jia X P , Li G Y and Tang L . 2017 . Hyperspectral images classification with Gabor filtering and convolutional neural network . IEEE Geoscience and Remote Sensing Letters , 14 ( 12 ): 2355 - 2359 [ DOI: 10.1109/LGRS.2017.2764915 http://dx.doi.org/10.1109/LGRS.2017.2764915 ]

Hu W , Huang Y Y , Wei L , Zhang F and Li H C . 2015 . Deep convolutional neural networks for hyperspectral image classification . Journal of Sensors , 2015 ( 2015 ): 258619 [ DOI: 10.1155/2015/258619 http://dx.doi.org/10.1155/2015/258619 ]

Ji Z and Nie L H . 2016 . Texture image classification with noise-tolerant local binary pattern . Journal of Computer Research and Development , 53 ( 5 ): 1128 - 1135

冀中 , 聂林红 . 2016 . 基于抗噪声局部二值模式的纹理图像分类 . 计算机研究与发展 , 53 ( 5 ): 1128 - 1135 [ DOI: 10.7544/issn1000-1239.2016.20148320 http://dx.doi.org/10.7544/issn1000-1239.2016.20148320 ]

Jia S , Deng B , Zhu J S , Jia X P and Li Q Q . 2018 . Local binary pattern-based hyperspectral image classification with superpixel guidance . IEEE Transactions on Geoscience and Remote Sensing , 56 ( 2 ): 749 - 759 [ DOI: 10.1109/TGRS.2017.2754511 http://dx.doi.org/10.1109/TGRS.2017.2754511 ]

Kingma D P and Ba J . 2014 . Adam: a method for stochastic optimization . arXiv preprint arXiv : 1412 . 6980

Krizhevsky A , Sutskever I and Hinton G E . 2012 . ImageNet classification with deep convolutional neural networks // Proceedings of the 25th International Conference on Neural Information Processing Systems . Lake Tahoe, Nevada : Curran Associates Inc .: 1097 - 1105

Li J , Marpu P R , Plaza A , Bioucas-Dias J M and Benediktsson J A . 2013 . Generalized composite kernel framework for hyperspectral image classification . IEEE Transactions on Geoscience and Remote Sensing , 51 ( 9 ): 4816 - 4829 [ DOI: 10.1109/TGRS.2012.2230268 http://dx.doi.org/10.1109/TGRS.2012.2230268 ]

Li W , Wu G D , Zhang F and Du Q . 2017 . Hyperspectral image classification using deep pixel-pair features . IEEE Transactions on Geoscience and Remote Sensing , 55 ( 2 ): 844 - 853 [ DOI: 10.1109/TGRS.2016.2616355 http://dx.doi.org/10.1109/TGRS.2016.2616355 ]

Moser G and Serpico S B . 2013 . Combining support vector machines and Markov random fields in an integrated framework for contextual image classification . IEEE Transactions on Geoscience and Remote Sensing , 51 ( 5 ): 2734 - 2752 [ DOI: 10.1109/TGRS.2012.2211882 http://dx.doi.org/10.1109/TGRS.2012.2211882 ]

Pesaresi M and Benediktsson J A . 2001 . A new approach for the morphological segmentation of high-resolution satellite imagery . IEEE Transactions on Geoscience and Remote Sensing , 39 ( 2 ): 309 - 320 [ DOI: 10.1109/36.905239 http://dx.doi.org/10.1109/36.905239 ].

Plaza A , Plaza J and Martín G . 2009 . Incorporation of spatial constraints into spectral mixture analysis of remotely sensed hyperspectral data // Proceedings of 2009 IEEE International Workshop on Machine Learning for Signal Processing . Grenoble : IEEE : 1 - 6 [ DOI: 10.1109/MLSP.2009.5306202 http://dx.doi.org/10.1109/MLSP.2009.5306202 ]

Shu L , McIsaac K and Osinski G R . 2018 . Learning spatial-spectral features for hyperspectral image classification . IEEE Transactions on Geoscience and Remote Sensing , 56 ( 9 ): 5138 - 5147 [ DOI: 10.1109/TGRS.2018.2809912 http://dx.doi.org/10.1109/TGRS.2018.2809912 ]

Simonyan K and Zisserman A . 2014 . Very deep convolutional networks for large-scale image recognition . arXiv preprint arXiv : 1409 . 1556

Ye Z and Bai L . 2017 . Hyperspectral image classification based on principal component analysis and local binary patterns . Laser and Optoelectronics Progress , 54 ( 11 ): 111006.1 - 1110 06.10

叶珍 , 白璘 . 2017 . 基于主成分分析与局部二值模式的高光谱图像分类 . 激光与光电子学进展 , 54 ( 11 ): 111006.1 - 111006.10 [ DOI: 10.3788/LOP54.111006 http://dx.doi.org/10.3788/LOP54.111006 ]

Ye Z , Fowler J E and Bai L . 2017 . Spatial-spectral hyperspectral classification using local binary patterns and Markov random fields . Journal of Applied Remote Sensing , 11 ( 3 ): 035002 [ DOI: 10.1117/1.JRS.11.035002 http://dx.doi.org/10.1117/1.JRS.11.035002 ]

Zhang H K , Li Y , Zhang Y Z and Shen Q . 2017 . Spectral-spatial classification of hyperspectral imagery using a dual-channel convolutional neural network . Remote Sensing Letters , 8 ( 5 ): 438 - 447 [ DOI: 10.1080/2150704X.2017.1280200 http://dx.doi.org/10.1080/2150704X.2017.1280200 ]

Zhang W and Wang W W . 2015 . Face recognition based on local binary pattern and deep learning . Journal of Computer Applications , 35 ( 5 ): 1474 - 1478

张雯 , 王文伟 . 2015 . 基于局部二值模式和深度学习的人脸识别 . 计算机应用 , 35 ( 5 ): 1474 - 1478 [ DOI: 10.11772/j.issn.1001-9081.2015.05.1474 http://dx.doi.org/10.11772/j.issn.1001-9081.2015.05.1474 ]

Zhi L , Yu X C and Fu Q Y . 2018 . Hyperspectral imagery spatial-spectral classification combining local binary patterns . Journal of Geomatics Science and Technology , 35 ( 1 ): 65 - 69 , 76 .

职露 , 余旭初付琼莹 . 2018 . 联合局部二值模式的高光谱影像空—谱分类方法 . 测绘科学技术学报 , 35 ( 1 ): 65 - 69,76 [ DOI: 10.3969/j.issn.1673-6338.2018.01.013 http://dx.doi.org/10.3969/j.issn.1673-6338.2018.01.013 ]

Alert me when the article has been cited

提交

Class-independent domain adaptation for hyperspectral image classification

Hyperspectral remote sensing image classification based on multidirectional adaptive aware network

生成对抗网络的无监督高光谱解混

A modified correlation alignment algorithm for the domain adaptation of GF-5 hyperspectral image

Research progress of generative adversarial networks in remote sensing image fusion