A MVG-based non-reference quality evaluation method for Pan/MS Fusion

Kedi BAO; Xiangchao MENG; Feng SHAO; Mengmeng YE; Kangjun JIN; Zhenyu PENG

doi:10.11834/jrs.20210191

Technologies and Methodologies | Views : 0 下载量: 1380 CSCD: 0

PDF
Export
Share
Collection
Album

A MVG-based non-reference quality evaluation method for Pan/MS Fusion
Vol. 26, Issue 3, Pages: 568-578(2022)
Received：12 June 2020，

Published：07 March 2022
DOI： 10.11834/jrs.20210191
稿件说明：

移动端阅览

包科迪，孟祥超，邵枫，叶蒙梦，金康俊，彭振宇.2022.全色/多光谱融合影像的MVG无参考质量评价.遥感学报，26（3）： 568-578 DOI： 10.11834/jrs.20210191.

Bao K D，Meng X C，Shao F，Ye M M，Jin K J and Peng Z Y. 2022. A MVG-based non-reference quality evaluation method for Pan/MS Fusion. National Remote Sensing Bulletin， 26（3）：568-578 DOI： 10.11834/jrs.20210191.

摘要

同时具有高空间分辨率和高光谱保真度的全色/多光谱融合影像具有广泛的应用前景。然而，一方面全色/多光谱融合方法通常会产生不同的空间畸变或光谱失真；另一方面，受卫星成像系统限制，无法获取真实高空间分辨率多光谱参考影像，因此，如何在无真实参考影像情况下有效评估融合影像质量具有重要意义。为此，本文提出了一种基于多元高斯模型（MVG）的全色/多光谱融合影像无参考质量评价方法，本方法基于全色/多光谱遥感影像融合大型数据库，构建了包含多种卫星类型和专题地物的基准数据集、测试数据集、验证数据集，用于MVG模型设计、测试、验证；在此基础上，充分考虑融合影像的空间畸变和光谱失真，针对性提取影像的空间和光谱敏感特征，通过大量空、谱特征的MVG模型训练拟合，建立基准MVG评价模型，此外，在基准MVG训练中，选择高方差子影像进行模型训练，以增强模型的稳健性；最终，以基准MVG模型与融合影像MVG模型之间的相对距离计算融合影像质量。实验结果表明，本研究方法相较于传统无参考质量评价方法具有一定优越性，有利于全色/多光谱融合影像质量评价的应用研究。

Abstract

Pansharpening aims to sharpen a Low-spatial-Resolution (LR) multispectral (MS) image using a High-spatial-Resolution (HR) panchromatic (PAN) image to obtain a HR MS image. The pansharpened image with high spatial resolution and spectral fidelity had a wide application prospect. However

on the one hand

pansharpening methods usually produce different spatial or spectral distortions; on the other hand

the real HR MS reference image for the quality evaluation of the fused image cannot be obtained due to the limitation of satellite imaging systems. Therefore

the effective evaluation the quality of the fusion image without the real reference image is of great significance.

In this paper

we propose a new non-reference quality evaluation method for pansharpened image on the basis of the Multivariate Gaussian Model (MVG). On the basis of a large-scale pansharpening database

this study constructed benchmark

test

and verification data sets

including various satellite sensors and thematic types. Then

a novel benchmark MVG evaluation model was constructed on the basis of the benchmark data set. In the proposed method

the images were first divided into sub-image blocks

and the spatial and spectral sensitive features of each sub-image block were first extracted. Then

many spatial and spectral characteristics were trained to establish the benchmark MVG evaluation model. In addition

in the benchmark MVG training

the sub-image with a high variance was selected for model training to enhance the robustness of the benchmark MVG model. Then

the testing MVG evaluation model for the fused image was established by comprehensively considering the spatial and spectral distortions. Finally

the relative distance between the benchmark and testing MVG models of the fused image was used to calculate the fused image quality.

Experimental results show that the proposed method has better performance than the traditional non-reference quality evaluation methods in most satellite- and thematic-based data sets. The proposed method is based on Wald’s protocol and cannot be extended well to the full resolution evaluation of the fused image. Therefore

in future work

we will continue to study the full resolution evaluation method for pansharpening

which is more challenging and meaningful.

关键词

Keywords

references

Aiazzi B , Alparone L , Baronti S and Garzelli A . 2002 . Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis . IEEE Transactions on Geoscience and Remote Sensing , 40 ( 10 ): 2300 - 2312 [ DOI: 10.1109/TGRS.2002.803623 http://dx.doi.org/10.1109/TGRS.2002.803623 ]

Aiazzi B , Alparone L , Baronti S , Garzelli A and Selva M . 2006 . MTF-tailored multiscale fusion of high-resolution MS and pan imagery . Photogrammetric Engineering and Remote Sensing , 72 ( 5 ): 591 - 596 [ DOI: 10.14358/PERS.72.5.591 http://dx.doi.org/10.14358/PERS.72.5.591 ]

Alparone L , Aiazzi B , Baronti S , Garzelli A , Nencini F and Selva M . 2008 . Multispectral and panchromatic data fusion assessment without reference . Photogrammetric Engineering and Remote Sensing , 74 ( 2 ): 193 - 200 [ DOI: 10.14358/PERS.74.2.193 http://dx.doi.org/10.14358/PERS.74.2.193 ]

Alparone L , Baronti S , Garzelli A and Nencini F . 2004 . A global quality measurement of pan-sharpened multispectral imagery . IEEE Geoscience and Remote Sensing Letters , 1 ( 4 ): 313 - 317 [ DOI: 10.1109/LGRS.2004.836784 http://dx.doi.org/10.1109/LGRS.2004.836784 ]

Bishop C M . 2006 . Pattern Recognition and Machine Learning . New York : Springer

Canny J . 1986 . A computational approach to edge detection . IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI- 8 ( 6 ): 679 - 698 [ DOI: 10.1109/TPAMI.1986.4767851 http://dx.doi.org/10.1109/TPAMI.1986.4767851 ]

Chavez P S , Sides S C and Anderson J A . 1991 . Comparison of three different methods to merge multiresolution and multispectral data: Landsat TM and SPOT panchromatic . Photogrammetric Engineering and Remote Sensing , 57 ( 3 ): 295 - 303

Fukunaga K . 1972 . Introduction to Statistical Pattern Recognition. New York, NY, USA: Academic Press

Garzelli A and Nencini F . 2009 . Hypercomplex quality assessment of multi/hyperspectral images . IEEE Geoscience and Remote Sensing Letters , 6 ( 4 ): 662 - 665 [ DOI: 10.1109/LGRS.2009.2022650 http://dx.doi.org/10.1109/LGRS.2009.2022650 ]

Garzelli A , Nencini F and Capobianco L . 2008 . Optimal MMSE pan sharpening of very high resolution multispectral images . IEEE Transactions on Geoscience and Remote Sensing , 46 ( 1 ): 228 - 236 [ DOI: 10.1109/TGRS.2007.907604 http://dx.doi.org/10.1109/TGRS.2007.907604 ]

Han P , Gong J Y , Li Z L and Cheng L . 2008 . Comparing the effect of aggregation methods for remote sensing image . Journal of Remote Sensing , 12 ( 6 ): 964 - 971

韩鹏 , 龚健雅 , 李志林 , 程亮 . 2008 . 遥感影像空间尺度上推方法的评价 . 遥感学报 , 12 ( 6 ): 964 - 971

Kwan C , Budavari B , Bovik A C and Marchisio G . 2017 . Blind quality assessment of fused WorldView-3 images by using the combinations of pansharpening and hypersharpening paradigms . IEEE Geoscience and Remote Sensing Letters , 14 ( 10 ): 1835 - 1839 [ DOI: 10.1109/LGRS.2017.2737820 http://dx.doi.org/10.1109/LGRS.2017.2737820 ]

Laben C A and Brower B V . 2000 . Process for enhancing the spatial resolution of multispectral imagery using pan-sharpening. U.S ., No . 6011875

Meng X C , Shen H F , Yuan Q Q , Li H F , Zhang L P and Sun W W . 2019 . Pansharpening for cloud-contaminated very high-resolution remote sensing images . IEEE Transactions on Geoscience and Remote Sensing , 57 ( 5 ): 2840 - 2854 [ DOI: 10.1109/TGRS.2018.2878007 http://dx.doi.org/10.1109/TGRS.2018.2878007 ]

Meng X C , Xiong Y M , Shao F , Shen H F , Sun W W , Yang G , Yuan Q Q , Fu R D and Zhang H Y . 2021 . A large-scale benchmark data set for evaluating pansharpening performance: overview and implementation . IEEE Geoscience and Remote Sensing Magazine , 9 ( 1 ): 18 - 52 [ DOI: 10.1109/MGRS.2020.2976696 http://dx.doi.org/10.1109/MGRS.2020.2976696 ]

Ojala T , Pietikainen M and Maenpaa T . 2002 . Multiresolution gray-scale and rotation invariant texture classification with local binary patterns . IEEE Transactions on Pattern Analysis and Machine Intelligence , 24 ( 7 ): 971 - 987 [ DOI: 10.1109/TPAMI.2002.1017623 http://dx.doi.org/10.1109/TPAMI.2002.1017623 ]

Ranchin T and Wald L . 2000 . Fusion of high spatial and spectral resolution images: the ARSIS concept and its implementation . Photogrammetric Engineering and Remote Sensing , 66 ( 1 ): 49 - 61

Sales M H R , Souza C M and Kyriakidis P C . 2013 . Fusion of MODIS images using Kriging with external drift . IEEE Transactions on Geoscience and Remote Sensing , 51 ( 4 ): 2250 - 2259 [ DOI: 10.1109/TGRS.2012.2208467 http://dx.doi.org/10.1109/TGRS.2012.2208467 ]

Sheikh H R , Sabir M F and Bovik A C . 2006 . A statistical evaluation of recent full reference image quality assessment algorithms . IEEE Transactions on Image Processing , 15 ( 11 ): 3440 - 3451 [ DOI: 10.1109/TIP.2006.881959 http://dx.doi.org/10.1109/TIP.2006.881959 ]

Tu T M , Su S C , Shyu H C and Huang P S . 2001 . A new look at IHS-like image fusion methods . Information Fusion , 2 ( 3 ): 177 - 186 [ DOI: 10.1016/S1566-2535(01)00036-7 http://dx.doi.org/10.1016/S1566-2535(01)00036-7 ]

Vivone G , Alparone L , Chanussot J , Mura M D , Garzelli A , Licciardi G A , Restaino R and Wald L . 2015 . A critical comparison among pansharpening algorithms . IEEE Transactions on Geoscience and Remote Sensing , 53 ( 5 ): 2565 - 2586 [ DOI: 10.1109/TGRS.2014.2361734 http://dx.doi.org/10.1109/TGRS.2014.2361734 ]

Vivone G , Restaino R and Chanussot J . 2018 . A Bayesian procedure for full-resolution quality assessment of pansharpened products . IEEE Transactions on Geoscience and Remote Sensing , 56 ( 8 ): 4820 - 4834 [ DOI: 10.1109/TGRS.2018.2839564 http://dx.doi.org/10.1109/TGRS.2018.2839564 ]

Wald L . 2000 . Quality of high resolution synthesised images: is there a simple criterion? // Proceedings of the Third Conference . Nice, France : SEE/URISCA: 99 - 103

Wald L , Ranchin T and Mangolini M . 1997 . Fusion of satellite images of different spatial resolutions: assessing the quality of resulting images . Photogrammetric Engineering and Remote Sensing , 63 ( 6 ): 691 - 699

Wang Z and Bovik A C . 2002 . A universal image quality index . IEEE Signal Processing Letters , 9 ( 3 ): 81 - 84 [ DOI: 10.1109/97.995823 http://dx.doi.org/10.1109/97.995823 ]

Wang Z , Bovik A C , Sheikh H R and Simoncelli E P . 2004 . Image quality assessment: from error visibility to structural similarity . IEEE Transactions on Image Processing , 13 ( 4 ): 600 - 612 [ DOI: 10.1109/TIP.2003.819861 http://dx.doi.org/10.1109/TIP.2003.819861 ]

Xiong Y M , Shao F , Meng X C , Jiang Q P , Sun W W , Fu R D and Ho Y S . 2020 . A large-scale remote sensing database for subjective and objective quality assessment of pansharpened images . Journal of Visual Communication and Image Representation , 73 : 102947 [ DOI: 10.1016/j.jvcir.2020.102947 http://dx.doi.org/10.1016/j.jvcir.2020.102947 ]

Yang J X , Zhao Y Q , Yi C and Chan J C W . 2017 . No-reference hyperspectral image quality assessment via quality-sensitive features learning . Remote Sensing , 9 ( 4 ): 305 [ DOI: 10.3390/rs9040305 http://dx.doi.org/10.3390/rs9040305 ]

Yuhas R H , Goetz A F H and Boardman J W . 1992 . Discrimination among semi-arid landscape endmembers using the spectral angle mapper (SAM) algorithm //Summaries of the Third Annual JPL Airborne Geoscience Workshop. [s.l.]: JPL: 147 - 149

Zhang L F , Peng M Y , Sun X J , Cen Y and Tong Q X . 2019 . Progress and bibliometric analysis of remote sensing data fusion methods ( 1992-2018 ). Journal of Remote Sensing , 23 ( 4 ): 603 - 619

张立福 , 彭明媛 , 孙雪剑 , 岑奕 , 童庆禧 . 2019 . 遥感数据融合研究进展与文献定量分析(1992-2018) . 遥感学报 , 23 ( 4 ): 603- 619 [ DOI: 10.11834/jrs.20199073 http://dx.doi.org/10.11834/jrs.20199073 ]

Zhang L P and Shen H F . 2016 . Progress and future of remote sensing data fusion . Journal of Remote Sensing , 20 ( 5 ): 1050 - 1061

张良培 , 沈焕锋 . 2016 . 遥感数据融合的进展与前瞻 . 遥感学报 , 20 ( 5 ): 1050- 1061 [ DOI: 10.11834/jrs.20166243 http://dx.doi.org/10.11834/jrs.20166243 ]

Zheng J , Wang J and Song S M . 2019 . Advantages of Canny double threshold operator in edge detection . Geospatial Information , 17 ( 11 ): 128 - 130

郑健 , 王继 , 宋世铭 . 2019 . Canny双阈值算子在边缘提取中的优势 . 地理空间信息 , 17 ( 11 ): 128- 130 [ DOI: 10.3969/j.issn.1672-4623.2019.11.036 http://dx.doi.org/10.3969/j.issn.1672-4623.2019.11.036 ]

Zhou B Z , Shao F , Meng X C , Fu R D and Ho Y S . 2019 . No-reference quality assessment for pansharpened images via opinion-unaware learning . IEEE Access , 7 : 40388 - 40401 [ DOI: 10.1109/ACCESS.2019.2905615 http://dx.doi.org/10.1109/ACCESS.2019.2905615 ]

Alert me when the article has been cited

提交

Change detection method based on adaptive nonlocal pattern consistency for multimodal remote sensing images

SPIN-2 Panchromatic and SPOT-4 Multi-Spectral Image Fusion Based on Support Vector Machine

Landslide susceptibility assessment of the Luding 9.5 earthquake area by fusing LT-1 ascending and descending ground deformation observations

Small wetland extraction based on Sentinel-1 time-series data and MultiRocket-RF model