سابِق|تال

Detail focused fusion of hyperspectral and multispectral images

  • role： First author第一作者
  • Affiliation：

    School of Computer Science and Information Engineering, Hefei University of Technology, Hefei 230601, China

    Anhui Province Key Laboratory of Industry Safety and Emergency Technology, Hefei 230000, China

  • Email：fangshuai@163.com
  • Introduction：方帅，研究方向遥感图像处理、计算机视觉。E-mail：fangshuai@163.com
FANG Shuai12,  
  • Affiliation：

    School of Computer Science and Information Engineering, Hefei University of Technology, Hefei 230601, China

YAN Mingchang1,  
  • role： Corresponding author通信作者
  • Affiliation：

    School of Computer Science and Information Engineering, Hefei University of Technology, Hefei 230601, China

  • Email：zhangjing@hfut.edu.cn
  • Introduction：张晶，研究方向为人工智能、数据挖掘。E-mail：zhangjing@hfut.edu.cn
ZHANG Jing1*,  
  • Affiliation：

    Department of Automation, University of Science and Technology of China, Hefei 230072, China

CAO Yang3

ملخص

Hyperspectral image (HSI) and multispectral image (MSI) are two types of images widely used in the field of remote sensing. These images are useful in certain applications, such as environmental monitoring, target detection, and mineral exploration. HSI contains a large amount of spectral information. Photons are typically collected in a larger spatial area on the sensor to ensure a sufficiently high signal-to-noise ratio (SNR). Accordingly, the HSI spatial resolution is much lower compared with MSI. This low spatial resolution greatly affects the practicality of HSI. Accordingly, fusing a low-spatial resolution HSI (LR-HSI) with a high-spatial resolution MSI (HR-MSI) in the same scene to obtain a high-resolution HSI (HR-HSI) is a method for solving such problems, which resolves the contradiction that the spatial resolution and the spectral resolution cannot simultaneously maintain a high level. From the analysis of fusion effect, the spatial and spectral reconstruction errors of the existing algorithms are mainly reflected in the edge and detail areas.The method proposed in this work was a fusion algorithm for dictionary construction and image reconstruction based on detail attention. In terms of maintaining spectral characteristics, the spectral distribution in the detail area is complex and diverse because of the proximity effect of the image. This work proposes to perform dictionary learning on the image and detail layers. The detail perception error terms and a constraint of edge adaptive directional total variation are proposed for spatial characteristic enhancement, which is combined with a local low rank constraint in the same fusion framework to estimate the sparse coefficient.Experiments were conducted on two datasets, namely, Pavia University and Indian Pine, to verify the effectiveness of the proposed method. The quantitative evaluation metrics contain peak SNR, relative dimensionless global error in synthesis, spectral angle map, and universal image quality index. Based on the experimental comparison, the fusion result of the algorithm proposed in this work is significantly improved compared with those of the other algorithms in terms of spatial and spectral characteristics.This work uses dictionary learning to propose a fusion algorithm for dictionary construction and image reconstruction with attention to details through the analysis of the existing hyperspectral and multispectral image fusion algorithms. A hierarchical dictionary learning algorithm is proposed to address the problem of large reconstruction error in the detail part of the existing algorithms. The detail perception error term and the direction adaptive full variational regularization term are used to improve the spectral dictionary solution and coefficient estimation, respectively. The result of the fusion is the error in the spectral characteristics and spatial texture of the detail, which achieves an accurate representation of the edge detail.

مفهوم

remote sensing;hyperspectral image;image fusion;Dictionary Learning;Edge Adaptive Directional Total Variation;Local Low Rank

References

  1. 1.
    Afonso M V, Bioucas-Dias J M and Figueiredo M A T. 2011. An augmented Lagrangian approach to the constrained optimization formulation of imaging inverse problems. IEEE Transactions on Image Processing, 20(3): 681-695
  2. 2.
    Alparone L, Wald L, Chanussot J, Thomas C, Gamba P and Bruce L M. 2007. Comparison of pansharpening algorithms: outcome of the 2006 GRS-S data-fusion contest. IEEE Transactions on Geoscience and Remote Sensing, 45(10): 3012-3021
  3. 3.
    Bayram İ and Kamasak M E. 2012. A directional total variation//2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO). Bucharest: IEEE: 265-269
  4. 4.
    Dian R W, Fang L Y and Li S T. 2017. Hyperspectral image super-resolution via non-local sparse tensor factorization//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 3862-3871
  5. 5.
    Dian R W, Li S T, Fang L Y and Bioucas-Dias J. 2018. Hyperspectral image super-resolution via local low-rank and sparse representations//2018 IEEE International Geoscience and Remote Sensing Symposium. Valencia: IEEE: 4003-4006
  6. 6.
    Dian R W, Li S T, Fang L Y, Lu T and Bioucas-Dias J M. 2020. Nonlocal sparse tensor factorization for semiblind hyperspectral and multispectral image fusion. IEEE Transactions on Cybernetics, 50(10): 4469-4480
  7. 7.
    Dong W S, Fu F Z, Shi G M, Cao X, Wu J J, Li G Y and Li X. 2016. Hyperspectral image super-resolution via non-negative structured sparse representation. IEEE Transactions on Image Processing, 25(5): 2337-2352
  8. 8.
    Dong W S, Li X, Zhang L and Shi G M. 2011. Sparsity-based image denoising via dictionary learning and structural clustering//Proceedings of 2011 IEEE Conference on Computer Vision and Pattern Recognition. Colorado Springs: IEEE: 457-464
  9. 9.
    Han X H, Shi B X and Zheng Y Q. 2018. Self-similarity constrained sparse representation for hyperspectral image super-resolution. IEEE Transactions on Image Processing, 27(11): 5625-5637
  10. 10.
    Hardie R C, Eismann M T and Wilson G L. 2004. MAP estimation for hyperspectral image resolution enhancement using an auxiliary sensor. IEEE Transactions on Image Processing, 13(9): 1174-1184
  11. 11.
    Kanatsoulis C I, Fu X, Sidiropoulos N D and Ma W K. 2018. Hyperspectral super-resolution: a coupled tensor factorization approach. IEEE Transactions on Signal Processing, 66(24): 6503-6517
  12. 12.
    Li S T，Li C Y and Kang X D. 2021. Development status and future prospects of multi-source remote sensing image fusion. National Remote Sensing Bulletin， 25（1）：148-166
  13. 13.
    Li X H, Shen H F, Zhang L P, Zhang H Y, Yuan Q Q and Yang G. 2014. Recovering quantitative remote sensing products contaminated by thick clouds and shadows using multitemporal dictionary learning. IEEE Transactions on Geoscience and Remote Sensing, 52(11): 7086-7098
  14. 14.
    Lin C H, Ma F, Chi C Y and Hsieh C H. 2018. A convex optimization-based coupled nonnegative matrix factorization algorithm for hyperspectral and multispectral data fusion. IEEE Transactions on Geoscience and Remote Sensing, 56(3): 1652-1667
  15. 15.
    Lorente D, Aleixos N, Gómez-Sanchis J, Cubero S, García-Navarrete O L and Blasco J. 2012. Recent advances and applications of hyperspectral imaging for fruit and vegetable quality assessment. Food and Bioprocess Technology, 5(4): 1121-1142
  16. 16.
    Meng X C, Sun W W, Ren K, Yang G, Shao F and Fu R D. 2020. Spatial-spectral fusion of GF-5/GF-1 remote sensing images based on multiresolution analysis. Journal of Remote Sensing. 24(4): 379-387
  17. 17.
    Palsson F, Sveinsson J R and Ulfarsson M O. 2017. Multispectral and hyperspectral image fusion using a 3-D-convolutional neural network. IEEE Geoscience and Remote Sensing Letters, 14(5): 639-643
  18. 18.
    Simões M, Bioucas-Dias J, Almeida L B and Chanussot J. 2015. A convex formulation for hyperspectral image superresolution via subspace-based regularization. IEEE Transactions on Geoscience and Remote Sensing, 53(6): 3373-3388
  19. 19.
    Veganzones M A, Simões M, Licciardi G, Yokoya N, Bioucas-Dias J M and Chanussot J. 2016. Hyperspectral super-resolution of locally low rank images from complementary multisource data. IEEE Transactions on Image Processing, 25(1): 274-288
  20. 20.
    Wald L. 2000. Quality of high resolution synthesised images: is there a simple criterion?//Proceedings of the 3rd Conference “Fusion of Earth Data: Merging Point Measurements, Raster Maps and Remotely Sensed Images”. Sophia Antipolis: HAL: 99-103
  21. 21.
    Wang X Y, Zhong Y F, Zhang L P and Xu Y Y. 2017. Spatial group sparsity regularized nonnegative matrix factorization for hyperspectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 55(11): 6287-6304
  22. 22.
    Wang Z and Bovik A C. 2002. A universal image quality index. IEEE Signal Processing Letters, 9(3): 81-84
  23. 23.
    Wei Q, Dobigeon N and Tourneret J Y. 2014. Bayesian fusion of hyperspectral and multispectral images//Proceedings of 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Florence: IEEE: 3176-3180 [DOI: 10.1109/ICASSP.2014.6854186]
  24. 24.
    Wu R Y, Ma W K, Fu X and Li Q. 2020. Hyperspectral super-resolution via global-local low-rank matrix estimation. IEEE Transactions on Geoscience and Remote Sensing, 58(10): 7125-7140
  25. 25.
    Xu G H, Liu Q H, Chen L F and Liu L Y. 2016. Remote sensing for China’s sustainable development: opportunities and challenges. Journal of Remote Sensing, 20(5): 679-688
  26. 26.
    Yang J X, Zhao Y Q and Chan J C W. 2018. Hyperspectral and multispectral image fusion via deep two-branches convolutional neural network. Remote Sensing, 10(5): 800
  27. 27.
    Yokoya N, Grohnfeldt C and Chanussot J. 2017. Hyperspectral and multispectral data fusion: a comparative review of the recent literature. IEEE Geoscience and Remote Sensing Magazine, 5(2): 29-56
  28. 28.
    Yokoya N, Yairi T and Iwasaki A. 2012. Coupled nonnegative matrix factorization unmixing for hyperspectral and multispectral data fusion. IEEE Transactions on Geoscience and Remote Sensing, 50(2): 528-537
  29. 29.
    Zhang B. 2016. Advancement of hyperspectral image processing and information extraction. Journal of Remote Sensing, 20(5): 1062-1090
  30. 30.
    Zhang H and Wang Y Q. 2013. Edge adaptive directional total variation. The Journal of Engineering, 2013(11): 61-62

قراءة النص الكامل

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website