سابِق|تال

Spatial fusion enhancement of thermal infrared images based on multi-resolution analysis and low-rank guided filter

  • role： First author第一作者
  • Affiliation：

    School of Electronics and Information Engineering, Harbin Institute of Technology, Harbin 150001, China

  • Email：hit_mxy@hit.edu.cn
  • Introduction：苗馨远，1992年生，男，博士研究生，研究方向为定量遥感、多源图像融合、热红外图像温度发射率分离等。E-mail： hit_mxy@hit.edu.cn
MIAO Xinyuan,  
  • role： Corresponding author通信作者
  • Affiliation：

    School of Electronics and Information Engineering, Harbin Institute of Technology, Harbin 150001, China

  • Email：zhye@hit.edu.cn
  • Introduction：张晔，1960年生，男，教授，研究方向为遥感高光谱图像分析与处理、图像/视频压缩及传输、空间多源信息协同处理及应用等。E-mail： zhye@hit.edu.cn
ZHANG Ye*,  
  • Affiliation：

    School of Electronics and Information Engineering, Harbin Institute of Technology, Harbin 150001, China

ZHANG Junping

ملخص

Owing to the special imaging principle in the long-wave infrared region, thermal infrared remote sensing images contain the temperature and emissivity features of targets. However, the low spatial resolution of thermal infrared images limits its wide application. With the development of remote sensing technology, multisource remote sensing images in the same region can provide complete information of a target for researchers. On the basis of the high spatial resolution of visible band images, thermal infrared image fusion enhancement and subpixel feature extraction have a high application value.Therefore, a new method named subpixel temperature retrieval of thermal infrared images based on multiresolution superpixel low-rank representation and residual correlation is proposed in this paper. The method achieves two goals by fusing visible and thermal infrared images in a super-resolution way: (1) enhancement of spatial characteristics for thermal infrared images based on adaptive fusion and (2) estimation of subpixel temperature and super resolution for thermal infrared images.The main processing and advantages of the algorithm are listed as follows: (1) For superpixel segmentation and low-rank restoration at multiresolution, superpixel blocks, instead of traditional blocks, are used as low-rank restoration units to enhance the stability of species in each unit and suppress structural noise. (2) Through constructing a guided linear filter, the high-spatial-resolution feature of the visible image can be transferred to the thermal infrared image while keeping the spectral information of the thermal infrared image unchanged. (3) For the estimation of subpixel temperature and super resolution of thermal infrared images, the correlation between the residuals of VIS and fusion images is established at the low-resolution layer and applied to the high-resolution layer to preserve image details.To validate the effectiveness of the proposed method, the visible and thermal infrared data in the 2014 IGARSS data fusion contest are used for experiments. The algorithm is evaluated in three aspects: (1) the improvement of spatial characteristics for thermal infrared images through adaptive low-rank representation, such as noise suppression, intraclass smoothing, and edge enhancement; (2) the spectral information protection of thermal infrared images in homogeneous and heterogenous regions; (3) the super-resolution effect and the accuracy of subpixel temperature retrieval. Compared with the traditional supervised graph-based feature fusion method, the proposed method has the best edge-sharpening, noise suppression, and spatial smoothing effects. It can protect the spectral information of thermal infrared images for different region types. The super-resolution image obtained by the proposed algorithm achieves high-temperature retrieval accuracy, and the overall root-mean-square error is less than 1 K. The average classification accuracy is improved by more than 20%.

مفهوم

subpixel temperature retrieval;visible and thermal infrared image fusion;guided filter;multi-resolution self-adaption low-rank representation;super-pixel segmentation

References

  1. 1.
    Cai R R and Wang B. 2017. Hyperspectral imagery restoration based on low-rank representation. Journal of Fudan University (Natural Science), 56(3): 346-358
  2. 2.
    Candes E J and Donoho D L. 2000. Curvelets, multiresolution representation, and scaling laws//Proceedings SPIE 4119, Wavelet Applications in Signal and Image Processing VIII. San Diego: SPIE: 4119 [DOI: 10.1117/12.408568]
  3. 3.
    Chen J S, Li Z Q and Huang B. 2017. Linear spectral clustering superpixel. IEEE Transactions on Image Processing, 26(7): 3317-3330
  4. 4.
    Da Cunha A L, Zhou J and Do M N. 2006. The nonsubsampled contourlet transform: theory, design, and applications. IEEE Transactions on Image Processing, 15(10): 3089-3101
  5. 5.
    Ding M, Wei L and Wang B F. 2013. Research on fusion method for infrared and visible images via compressive sensing. Infrared Physics and Technology, 57: 56-67
  6. 6.
    Do M N and Vetterli M. 2005. The contourlet transform: an efficient directional multiresolution image representation. IEEE Transactions on Image Processing, 14(12): 2091-2106
  7. 7.
    Fasbender D, Radoux J and Bogaert P. 2008. Bayesian data fusion for adaptable image pansharpening. IEEE Transactions on Geoscience and Remote Sensing, 46(6): 1847-1857
  8. 8.
    Foster L A, Brock B W, Cutler M E J and Diotri F. 2012. A physically based method for estimating supraglacial debris thickness from thermal band remote-sensing data. Journal of Glaciology, 58(210): 677-691
  9. 9.
    Garcia F, Mirbach B, Ottersten B, Grandidier F and Cuesta Á. 2010. Pixel weighted average strategy for depth sensor data fusion//2010 IEEE International Conference on Image Processing. Hong Kong, China: IEEE: 2805-2808 [
  10. 10.
    Gillespie A, Rokugawa S, Matsunaga T, Cothern J S, Hook S and Kahle A B. 1998. A temperature and emissivity separation algorithm for Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) images. IEEE Transactions on Geoscience and Remote Sensing, 36(4): 1113-1126
  11. 11.
    He K M, Sun J and Tang X O. 2010. Guided image filtering//Proceeding of 11th European Conference on Computer Vision. Heraklion: Springer: 1-14
  12. 12.
    Jiao L C and Tan S. 2003. Development and prospect of image multiscale geometric analysis. Acta Electronica Sinica, 31(12A): 1975-1981 (焦李成, 谭山. 2003. 图像的多尺度几何分析: 回顾和展望. 电子学报, 31(12A): 1975-1981) [DOI: 10.3321/j.issn:0372-2112.2003.z1.007]
  13. 13.
    Kang X D, Li S T and Benediktsson J A. 2014. Spectral–spatial hyperspectral image classification with edge-preserving filtering. IEEE Transactions on Geoscience and Remote Sensing, 52(5): 2666-2677
  14. 14.
    Kustas W P, Norman J M, Anderson M C and French A N. 2003. Estimating subpixel surface temperatures and energy fluxes from the vegetation index-radiometric temperature relationship. Remote Sensing of Environment, 85(4): 429-440
  15. 15.
    Li H, Manjunath B S and Mitra S K. 1995. Multisensor image fusion using the wavelet transform. Graphical Models and Image Processing, 57(3): 235-245
  16. 16.
    Li X and Qin S Y. 2011. Efficient fusion for infrared and visible images based on compressive sensing principle. IET Image Processing, 5(2): 141-147
  17. 17.
    Liao W Z, Huang X, Van Coillie F, Gautama S, Pižurica A, Philips W, Liu H, Zhu T T, Shimoni M, Moser G and Tuia D. 2015a. Processing of multiresolution thermal hyperspectral and digital color data: outcome of the 2014 IEEE GRSS data fusion contest. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 8(6): 2984-2996
  18. 18.
    Liao W Z, Huang X, Van Coillie F, Thoonen G, Pižurica A, Scheunders P and Philips W. 2015b. Two-stage fusion of thermal hyperspectral and visible RGB image by PCA and Guided filter//2015 7th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS). Tokyo: IEEE: 1-4
  19. 19.
    Li Z L, Duan S B, Tang B H, Wu Y, Ren H Z and Yan G j. 2016. Review of methods for land surface temperature derived from thermal infrared remotely sensed data. National Remote Sensing Bulletin, 20(005):899-920
  20. 20.
    Manolakis D, Golowich S and DiPietro R S. 2014. Long-wave infrared hyperspectral remote sensing of chemical clouds: a focus on signal processing approaches. IEEE Signal Processing Magazine, 31(4): 120-141
  21. 21.
    Manolakis D, Pieper M, Truslow E, Lockwood R, Weisner A, Jacobson J and Cooley T. 2019. Longwave infrared hyperspectral imaging: principles, progress, and challenges. IEEE Geoscience and Remote Sensing Magazine, 7(2): 72-100
  22. 22.
    Mao K B, Qin Z H and Xu B. 2005. Method for land surface temperature retrieval from ASTER data. Journal of Institute of Surveying and Mapping, 22(1): 40-42
  23. 23.
    Pieri D and Abrams M. 2004. ASTER watches the world's volcanoes: a new paradigm for volcanological observations from orbit. Journal of Volcanology and Geothermal Research, 135(1/2): 13-28
  24. 24.
    Qin Z H, Zhang M H, Karnieli A and Berliner P. 2001. Mono-window algorithm for retrieving land surface temperature from landsat TM6 data. Acta Geographica Sinica, 56(4): 456-466
  25. 25.
    Sun L, Jeon B, Zheng Y H, Xu Y and Wu Z B. 2017. Homogeneous region based low rank representation in hidden field for hyperspectral classification//2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). Fort Worth: IEEE: 4758-4761
  26. 26.
    Van Der Meer F D, Van Der Werff H M A, Van Ruitenbeek F J A, Hecker C A, Bakker W H, Noomen M F, Van Der meijde M, Carranza E J M, De Smeth J B and Woldai T. 2012. Multi- and hyperspectral geologic remote sensing: a review. International Journal of Applied Earth Observation and Geoinformation, 14(1): 112-128
  27. 27.
    Wang C C, Yang G J, Ma Z and Xing Z R. 2009. Fusion of VNIR and thermal infrared remote sensing data based on GA-SOFM neural network. Geo-Spatial Information Science, 12(4): 271-280
  28. 28.
    Wang J D, Li X W, Sun X M and Liu Q. 2000. Inversion of temperature of non-isothermal pixel components using thermal radiation directional model. Chinese Science E Series, 30(S1): 54-60
  29. 29.
    Wang J N, Jiao L C, Liu H Y, Yang S Y and Liu F. 2015. Hyperspectral image classification by spatial–spectral derivative-aided kernel joint sparse representation. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 8(6): 2485-2500
  30. 30.
    Wang Z, Bovik A C, Sheikh H R and Simoncelli E P. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4): 600-612
  31. 31.
    Weng Q H, Rajasekar U and Hu X F. 2011. Modeling urban heat islands and their relationship with impervious surface and vegetation abundance by using ASTER images. IEEE Transactions on Geoscience and Remote Sensing, 49(10): 4080-4089
  32. 32.
    Wright J, Ganesh A, Rao S, Peng Y and Ma Y. 2009. Robust principal component analysis: exact recovery of corrupted low-rank matrices via convex optimization. Neuro Computing, 87(4): 3-56
  33. 33.
    Wu Y Y, Jin C, Pang L W, Zhao J L, Song Y, Hu T G, Zhao D R and Xu J F. 2019. Distribution characteristics of surface thermal environment in Zhejiang province based on thermal infrared remote sensing. National Remote Sensing Bulletin, 234
  34. 34.
    Xu Y, Wu Z B, Wei Z H, Mura M D, Chanussot J and Bertozzi A. 2016. GAS plume detection in hyperspectral video sequence using low rank representation//2016 IEEE International Conference on Image Processing (ICIP). Phoenix: IEEE: 2221-2225
  35. 35.
    Zhou T Y and Tao D C. 2011. Godec: Randomized low-rank and sparse matrix decomposition in noisy case//Proceedings of the 28th International Conference on International Conference on Machine Learning. Washington: Omnipress: 33-40

قراءة النص الكامل

The above content is generated by Large Model Translation. The translated content is for reference only. We do not assume any commercial or legal responsibilty for any consequences arising from the use of our website