结合协方差池化与跨尺度特征提取的高光谱分类
Hyperspectral classification algorithm based on covariance pooling and cross scale feature extraction
- 2024年28卷第1期 页码：203-218
纸质出版日期： 2024-01-07
DOI： 10.11834/jrs.20242326
扫 描 看 全 文
浏览全部资源
扫码关注微信
摘要
深度卷积神经网络在高光谱图像分类任务上取得了优越性能。但是，主流深度学习算法通常采用一阶池化运算，容易忽略光谱之间的相关性，因而难以获取高阶统计判别特征。另外，这类算法往往难以选择最优的窗口大小去捕获不同感受野信息。针对上述问题，本文提出了一种结合协方差池化和跨尺度特征提取的高光谱影像分类方法。该方法设计了跨尺度自适应特征提取模块，能够自动提取多尺度特征，获取不同视野的互补信息，避免了尺度选择问题；进一步利用平均池化和快速协方差池化的联合池化操作，得到一阶统计量和结合空间光谱信息的二阶统计量；最终，将一阶和二阶池化特征进行融合用于分类。在3个公开高光谱数据集Indian Pines、Houston和Pavia University上分别随机选取5%、5%和1%标记样本进行训练，本文算法得到的总体分类精度分别达到97.63%、98.48%和98.21%，分类性能优于主流深度学习方法。
Abstract
The deep convolution neural network algorithm has achieved excellent performance in hyperspectral image classification. However
these deep learning algorithms generally use first-order pooling operation
which ignores the correlation between different spectral bands. Thus
obtaining high-order statistical discriminant features is difficult. In addition
using these algorithms to choose the optimal window size and capture different receptive field information is complicated. This paper proposes a hyperspectral classification method combining covariance pooling and cross-scale feature extraction to solve the aforementioned problems. This method aims to automatically extract the complementary and discriminative information of different scales and exploit the first- and second-order pooling features to improve the classification performance.
A covariance pooling and cross-scale feature extraction method is proposed for hyperspectral image classification. In this method
a cross-scale adaptive feature extraction module is designed. This module can automatically combine multiscale feature information and obtain complementary information of different visual fields
avoiding the scale selection problem. Furthermore
the first- and second-order statistics combined with spatial-spectral information are obtained using the joint pooling operation of average and fast covariance pooling. Finally
the first- and second-order pooled features are fused for classification.
A total of 5%
5%
and 1% labeled samples were randomly selected from three public hyperspectral datasets
namely
Indian pines
Houston University
and Pavia University
respectively. The overall classification accuracy of the proposed algorithm reached 97.63%
98.48%
and 98.21%
and the classification performance was better than the state-of-the-art deep learning methods.
Cross-scale feature extraction considers the complementary spatial-spectral information between different scales to obtain additional adaptive feature information. Combining fast covariance and average pooling
the discriminant features are obtained by pooling feature fusion to obtain superior classification results.
关键词
高光谱图像分类协方差池化多尺度特征融合卷积神经网络
Keywords
hyperspectral image classificationcovariance poolingmultiscalefeature fusionconvolution neural networks
references
Benediktsson J A, Palmason J A and Sveinsson J R. 2005. Classification of hyperspectral data from urban areas based on extended morphological profiles. IEEE Transactions on Geoscience and Remote Sensing, 43(3): 480-491 [DOI: 10.1109/TGRS.2004.842478http://dx.doi.org/10.1109/TGRS.2004.842478]
Camps-Valls G, Gomez-Chova L, Munoz-Mari J, Vila-Frances J and Calpe-Maravilla J. 2006. Composite kernels for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 3(1): 93-97 [DOI: 10.1109/LGRS.2005.857031http://dx.doi.org/10.1109/LGRS.2005.857031]
Carreira J, Caseiro R, Batista J and Sminchisescu C. 2015. Free-form region description with second-order pooling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(6): 1177-1189 [DOI: 10.1109/TPAMI.2014.2361137http://dx.doi.org/10.1109/TPAMI.2014.2361137]
Chen Y, Nasrabadi N M and Tran T D. 2011. Hyperspectral image classification using dictionary-based sparse representation. IEEE Transactions on Geoscience and Remote Sensing, 49(10): 3973-3985 [DOI: 10.1109/TGRS.2011.2129595http://dx.doi.org/10.1109/TGRS.2011.2129595]
Chen Y S, Jiang H L, Li C Y, Jia X P and Ghamisi P. 2016. Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 54(10): 6232-6251 [DOI: 10.1109/TGRS.2016.2584107http://dx.doi.org/10.1109/TGRS.2016.2584107]
Chen Y S, Lin Z H, Zhao X, Wang G and Gu Y F. 2014. Deep learning-based classification of hyperspectral data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(6): 2094-2107 [DOI: 10.1109/JSTARS.2014.2329330http://dx.doi.org/10.1109/JSTARS.2014.2329330]
Demarez V. 1999. Seasonal variation of leaf chlorophyll content of a temperate forest. Inversion of the PROSPECT model. International Journal of Remote Sensing, 20(5): 879-894 [DOI: 10.1080/014311699212975http://dx.doi.org/10.1080/014311699212975]
Du P J, Xia J S, Xue Z H, Tan K, Su H J and Bao R. 2016. Review of hyperspectral remote sensing image classification. Journal of Remote Sensing, 20(2): 236-256
杜培军, 夏俊士, 薛朝辉, 谭琨, 苏红军, 鲍蕊. 2016. 高光谱遥感影像分类研究进展. 遥感学报, 20(2): 236-256 [DOI: 10.11834/jrs.20165022http://dx.doi.org/10.11834/jrs.20165022]
Gong Z Q, Zhong P, Yu Y, Hu W D and Li S T. 2019. A CNN with multiscale convolution and diversified metric for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 57(6): 3599-3618 [DOI: 10.1109/TGRS.2018.2886022http://dx.doi.org/10.1109/TGRS.2018.2886022]
Ham J, Chen Y C, Crawford M M and Ghosh J. 2005. Investigation of the random forest framework for classification of hyperspectral data. IEEE Transactions on Geoscience and Remote Sensing, 43(3): 492-501 [DOI: 10.1109/TGRS.2004.842481http://dx.doi.org/10.1109/TGRS.2004.842481]
He N J, Fang L Y, Li S T, Plaza A and Plaza J. 2018. Remote sensing scene classification using multilayer stacked covariance pooling. IEEE Trans actions on Geoscience and Remote Sensing, 56(12): 6899-6910 [DOI: 10.1109/TGRS.2018.2845668http://dx.doi.org/10.1109/TGRS.2018.2845668]
He N J, Paoletti M E, Haut J M, Fang L Y, Li S T, Plaza A and Plaza J. 2019. Feature extraction with multiscale covariance maps for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 57(2): 755-769 [DOI: 10.1109/TGRS.2018.2860464http://dx.doi.org/10.1109/TGRS.2018.2860464]
Ionescu C, Vantzos O and Sminchisescu C. 2015. Matrix backpropagation for deep networks with structured layers//2015 IEEE International Conference on Computer Vision. Santiago: IEEE: 2965-2973 [DOI: 10.1109/ICCV.2015.339http://dx.doi.org/10.1109/ICCV.2015.339]
Jia S, Shen L L and Li Q Q. 2015. Gabor feature-based collaborative representation for hyperspectral imagery classification. IEEE Transactions on Geoscience and Remote Sensing, 53(2): 1118-1129 [DOI: 10.1109/TGRS.2014.2334608http://dx.doi.org/10.1109/TGRS.2014.2334608]
Lee H and Kwon H. 2017. Going deeper with contextual CNN for hyperspectral image classification. IEEE Transactions on Image Processing, 26(10): 4843-4855 [DOI: 10.1109/TIP.2017.2725580http://dx.doi.org/10.1109/TIP.2017.2725580]
Li P H, Xie J T, Wang Q L and Gao Z L. 2018. Towards faster training of global covariance pooling networks by iterative matrix square root normalization//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 947-955 [DOI: 10.1109/CVPR.2018.00105http://dx.doi.org/10.1109/CVPR.2018.00105]
Li P H, Xie J T, Wang Q L and Zuo W M. 2017a. Is second-order information helpful for large-scale visual recognition?//2017 IEEE International Conference on Computer Vision. Venice: IEEE: 2089-2097 [DOI: 10.1109/ICCV.2017.228http://dx.doi.org/10.1109/ICCV.2017.228]
Li S M, Zhu X Y and Bao J. 2019. Hierarchical multi-scale convolutional neural networks for hyperspectral image classification. Sensors, 19(7): 1714 [DOI: 10.3390/s19071714http://dx.doi.org/10.3390/s19071714]
Li T, Zhang J P and Zhang Y. 2014. Classification of hyperspectral image based on deep belief networks//2014 IEEE International Conference on Image Processing. Paris: IEEE: 5132-5136 [DOI: 10.1109/ICIP.2014.7026039http://dx.doi.org/10.1109/ICIP.2014.7026039]
Li W, Chen C, Su H J and Du Q. 2015. Local binary patterns and extreme learning machine for hyperspectral imagery classification. IEEE Transactions on Geoscience and Remote Sensing, 53(7): 3681-3693 [DOI: 10.1109/TGRS.2014.2381602http://dx.doi.org/10.1109/TGRS.2014.2381602]
Li Y, Zhang H K and Shen Q. 2017b. Spectral–spatial classification of hyperspectral imagery with 3D convolutional neural network. Remote Sensing, 9(1): 67 [DOI: 10.3390/rs9010067http://dx.doi.org/10.3390/rs9010067]
Lin T Y, RoyChowdhury A and Maji S. 2015. Bilinear CNN models for fine-grained visual recognition//2015 IEEE International Conference on Computer Vision. Santiago: IEEE: 1449-1457 [DOI: 10.1109/ICCV.2015.170http://dx.doi.org/10.1109/ICCV.2015.170]
Ma L, Crawford M M and Tian J W. 2010. Local manifold learning based K-nearest-neighbor for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 48(11): 4099-4109 [DOI: 10.1109/TGRS.2010.2055876http://dx.doi.org/10.1109/TGRS.2010.2055876]
Makantasis K, Karantzalos K, Doulamis A and Doulamis N. 2015. Deep supervised learning for hyperspectral data classification through convolutional neural networks. IEEE International Geoscience and Remote Sensing Symposium. Milan: IEEE: 4959-4962 [DOI: 10.1109/IGARSS.2015.7326945http://dx.doi.org/10.1109/IGARSS.2015.7326945]
Melgani F and Bruzzone L. 2004. Classification of hyperspectral remote sensing images with support vector machines. IEEE Transactions on Geoscience and Remote Sensing, 42(8): 1778-1790 [DOI: 10.1109/TGRS.2004.831865http://dx.doi.org/10.1109/TGRS.2004.831865]
Mura M D, Benediktsson J A, Waske B and Bruzzone L. 2010. Extended profiles with morphological attribute filters for the analysis of hyperspectral data. International Journal of Remote Sensing, 31(22): 5975-5991 [DOI: 10.1080/01431161.2010.512425http://dx.doi.org/10.1080/01431161.2010.512425]
Murphy R J, Whelan B, Chlingaryan A and Sukkarieh S. 2019. Quantifying leaf-scale variations in water absorption in lettuce from hyperspectral imagery: a laboratory study with implications for measuring leaf water content in the context of precision agriculture. Precision Agriculture, 20(4): 767-787 [DOI: 10.1007/s11119-018-9610-5http://dx.doi.org/10.1007/s11119-018-9610-5]
Peng J T, Chen H, Zhou Y C and Li L Q. 2017. Ideal regularized composite kernel for hyperspectral image classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 10(4): 1563-1574 [DOI: 10.1109/JSTARS.2016.2621416http://dx.doi.org/10.1109/JSTARS.2016.2621416]
Peng J T, Sun W W and Du Q. 2019. Self-paced joint sparse representation for the classification of hyperspectral images. IEEE Transactions on Geoscience and Remote Sensing, 57(2): 1183-1194 [DOI: 10.1109/TGRS.2018.2865102http://dx.doi.org/10.1109/TGRS.2018.2865102]
Peng J T, Sun W W, Wei T H and Fan W Q. 2020. A modified correlation alignment algorithm for the domain adaptation of GF-5 hyperspectral image. Journal of Remote Sensing (Chinese), 24(4): 417-426
彭江涛, 孙伟伟, 魏天慧, 范文琦. 2020. 高分五号高光谱影像的关联对齐域适应与分类. 遥感学报, 24(4): 417-426 [DOI: 10.11834/jrs.20209212http://dx.doi.org/10.11834/jrs.20209212]
Roy S K, Krishna G, Dubey S R and Chaudhuri B B. 2020. HybridSN: exploring 3-D–2-D CNN feature hierarchy for hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters, 17(2): 277-281 [DOI: 10.1109/LGRS.2019.2918719http://dx.doi.org/10.1109/LGRS.2019.2918719]
Roy S K, Manna S, Song T C and Bruzzone L. 2021. Attention-based adaptive spectral-spatial kernel ResNet for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 59(9): 7831-7843 [DOI: 10.1109/TGRS.2020.3043267http://dx.doi.org/10.1109/TGRS.2020.3043267]
Shimoni M, Haelterman R and Perneel C. 2019. Hypersectral imaging for military and security applications: combining myriad processing and sensing techniques. IEEE Geoscience and Remote Sensing Magazine, 7(2): 101-117 [DOI: 10.1109/MGRS.2019.2902525http://dx.doi.org/10.1109/MGRS.2019.2902525]
Song W W, Li S T, Fang L Y and Lu T. 2018. Hyperspectral image classification with deep feature fusion network. IEEE Transactions on Geoscience and Remote Sensing, 56(6): 3173-3184 [DOI: 10.1109/TGRS.2018.2794326http://dx.doi.org/10.1109/TGRS.2018.2794326]
Sun W W, Zhang D F, Yang G and Li W Y. 2018. Band selection for hyperspectral imagery based on weighted probabilistic archetypal analysis. Journal of Remote Sensing, 22(1): 110-118
孙伟伟, 张殿发, 杨刚, 李巍岳. 2018. 加权概率原型分析的高光谱影像波段选择. 遥感学报, 22(1): 110-118 [DOI: 10.11834/jrs.20186446http://dx.doi.org/10.11834/jrs.20186446]
Tao C, Pan H B, Li Y S and Zou Z R. 2015. Unsupervised spectral-spatial feature learning with stacked sparse autoencoder for hyperspectral imagery classification. IEEE Geoscience and Remote Sensing Letters, 12(12): 2438-2442 [DOI: 10.1109/LGRS.2015.2482520http://dx.doi.org/10.1109/LGRS.2015.2482520]
Tong Q X, Zhang B and Zhang L F. 2016. Current progress of hyperspectral remote sensing in China. Journal of Remote Sensing, 20(5): 689-707
童庆禧, 张兵, 张立福. 2016. 中国高光谱遥感的前沿进展. 遥感学报, 20(5): 689-707 [DOI: 10.11834/jrs.20166264http://dx.doi.org/10.11834/jrs.20166264]
Tuzel O, Porikli F, Meer P. 2006. Region covariance: A fast descriptor for detection and classification. ECCV European Conference on Computer Vision. 3952: 589-600. [DOI: 10.1007/11744047_45http://dx.doi.org/10.1007/11744047_45]
Xie J, He N J, Fang L Y and Ghamisi P. 2021. Multiscale Densely-Connected Fusion Networks for Hyperspectral Images Classification. IEEE Transactions on Circuits and Systems for Video Technology, 31(1): 246-259 [DOI: 10.1109/TCSVT.2020.2975566http://dx.doi.org/10.1109/TCSVT.2020.2975566]
Xue Z H, Zhang M X, Liu Y F and Du P J. 2021. Attention-based second-order pooling network for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 59(11): 9600-9615 [DOI: 10.1109/TGRS.2020.3048128http://dx.doi.org/10.1109/TGRS.2020.3048128]
Zhang C J, Li G D, Du S H, Tan W Z and Gao F. 2019. Three-dimensional densely connected convolutional network for hyperspectral remote sensing image classification. Journal of Applied Remote Sensing, 13(1): 016519 [DOI: 10.1117/1.JRS.13.016519http://dx.doi.org/10.1117/1.JRS.13.016519]
Zhang H K, Li Y, Zhang Y Z and Shen Q. 2017. Spectral-spatial classification of hyperspectral imagery using a dual-channel convolutional neural network. Remote Sensing Letters, 8(5): 438-447 [DOI: 10.1080/2150704X.2017.1280200http://dx.doi.org/10.1080/2150704X.2017.1280200]
Zhao W Z and Du S H. 2016. Spectral-spatial feature extraction for hyperspectral image classification: a dimension reduction and deep learning approach. IEEE Transactions on Geoscience and Remote Sensing, 54(8): 4544-4554 [DOI: 10.1109/TGRS.2016.2543748http://dx.doi.org/10.1109/TGRS.2016.2543748]
Zheng J W, Feng Y C, Bai C and Zhang J L. 2021. Hyperspectral image classification using mixed convolutions and covariance pooling. IEEE Transactions on Geoscience and Remote Sensing, 59(1): 522-534 [DOI: 10.1109/TGRS.2020.2995575http://dx.doi.org/10.1109/TGRS.2020.2995575]
Zhong Z L, Li J, Luo Z M and Chapman M. 2018. Spectral-spatial residual network for hyperspectral image classification: a 3-D deep learning framework. IEEE Transactions on Geoscience and Remote Sensing, 56(2): 847-858 [DOI: 10.1109/TGRS.2017.2755542http://dx.doi.org/10.1109/TGRS.2017.2755542]
文章被引用时，请邮件提醒。
提交
相关作者
暂无数据
相关机构
南京航空航天大学 自动化学院
南京航空航天大学 计算机科学与技术学院
中国矿业大学 信息与控制工程学院
湖南师范大学 地理空间大数据挖掘与应用湖南省重点实验室
湖南师范大学 地理科学学院
选择翻译语言
- 地址：北京市海淀区北四环西路19号 《遥感学报》编辑部 邮编：100190
- 联系电话：010-58887052 Email：nrsb@aircas.ac.cn
- 技术支持由北京北大方正电子有限公司提供 京ICP备20021838号-9
京公网安备11010802024621 - 本系统建议在Chrome、 IE9+ 以上版本浏览器阅读本站内容，360浏览器请切换至极速模式
- Cookies帮助我们提供服务并提供个性化体验。使用本网站，即表示您同意我们使用Cookies
0
回到顶部