Cloud detection for FY-3D MERSI Ⅱ images combine radiative transfer simulation and shallow neural network

JIN Shikuan; MA Yingying; GONG Wei; YE Zhiwei; XIA Xiaoyu

doi:10.11834/jrs.20210406

Chinese Satellites | Views : 0 下载量: 1227 CSCD: 1

R-PDF
PDF
Export
Share
Collection
Album

Cloud detection for FY-3D MERSI Ⅱ images combine radiative transfer simulation and shallow neural network
Vol. 26, Issue 11, Pages: 2136-2146(2022)
Received：20 September 2020，

Published：07 November 2022
DOI： 10.11834/jrs.20210406
稿件说明：

移动端阅览

金适宽，马盈盈，龚威，叶志伟，夏小鱼.2022.结合辐射传输模拟与浅层神经网络的FY-3D MERSI影像云识别.遥感学报，26（11）： 2136-2146 DOI： 10.11834/jrs.20210406.

Jin S K，Ma Y Y，Gong W，Ye Z W and Xia X Y. 2022. Cloud detection for FY-3D MERSI Ⅱ images combine radiative transfer simulation and shallow neural network. National Remote Sensing Bulletin， 26（11）：2136-2146 DOI： 10.11834/jrs.20210406.

摘要

本文结合辐射传输模型和机器学习提出了一种从FY-3D卫星MERSI Ⅱ传感器光学影像中识别云像元的方法CRMC（Combine Reflectance simulation and Machine learning for Cloud detection）。该方法通过设置变化的地物和大气内在光学特性IOPs（Inherent Optical Properties），达到考虑多种下垫面的二项反射特征和不同大气条件下气溶胶和云参数的目的。CRMC方法主要包含3个步骤：（1）通过聚类分析从MODIS二项反射参数产品中分离出11种典型下垫面地表反射参数；（2）将随机设置的气溶胶和云参数以及地表反射率参数（即IOPs）输入SBDART辐射传输模型，得到模拟的反射率值数据集，并以此训练浅层神经网络模型；（3）利用浅层神经网络模型逐像元预测云概率，并根据实际需要确定区分云像元和非云像元的云概率阈值。通过与CALIPSO垂直特性掩膜产品（VFM）逐像元对比验证发现，CRMC方法总正确率为79.6%，且在陆地和海面上分别为78.5%和81.2%。通过与MODIS云掩膜产品横向对比（MYD35）发现，当云阈值设定为0.2时，CRMC方法在陆地，主要是阔叶林、农田、城市和裸土等地表上的云识别效果较好，但在海面的云识别效果仍待进一步提高。

Abstract

Cloud pixel detection is a crucial pre-process in numerous remote sensing applications

such as the aerosol parameter retrieval

land use change

anomaly detection/classification

crop monitoring

and marine ecological survey. On the one hand

cloud pixel misidentification (mistaken as surface or aerosol) in optical images has substantial negative effects on the above-mentioned traditional applications

due to the significant influence of cloud layers on shortwave radiation. On the other hand

traditional threshold methods are largely constrained in applicability by the high spatiotemporal heterogeneity of aerosols and clouds as well as the diversification of satellite sensors and spectral channels. Therefore

a reliable cloud detection method with wide applicability is in demand

especially for studies over multiple surfaces (i.e.

land

ocean

and cryosphere). Thus

this study proposed an innovative method based on the radiative transfer simulation and machine learning

namely CRMC (Combine Reflectance simulation and Machine learning for Cloud detection)

to detect cloud pixels in optical images produced from the MERSI Ⅱ sensor onboard the FY-3D satellite. The biggest advantage of this method is its compatibility with different sensors

generating cloud pixel samples through the physical process simulation

and applying the machine learning technique to learn sample features

thereby excluding the influence of anthropogenic factors. Specifically

to address the mismatch between the MODIS cloud detection algorithm and MERSI-based aerosol inversion

the CRMC method sets different Inherent Optical Properties (IOPs) of surface and atmospheric objects

considering binomial reflection characteristics of underlying surfaces and various parameters of aerosols and clouds. In addition

the method outputs cloud probabilities in pixels and allows custom thresholds to control the strictness level of cloud pixel detection. The CRMC method mainly includes three steps: (1) Defining 11 typical underlying reflectance parameters from MODIS binomial reflection products using a cluster analysis approach; (2) Inputting the typical underlying reflectance and aerosol and cloud parameters with random inherent optical properties into the SBDART radiation transmission model to obtain a simulated reflectance dataset for training the shallow neural network; (3) Calculating the cloud probability of the target image with the trained shallow neural network and selecting a suitable threshold according to the actual need to complete the cloud detection. Compared with the CALIPSO Vertical Feature Mask (VFM)

results of the CRMC method show a maximum total accuracy of 79.6% (78.5% and 81.2% on land and sea

respectively). Under the condition of cloud probability threshold=0.2 (hit rates for cloud and cloud-free pixels are the same in these cases)

the CRMC outperforms MODIS cloud mask products (MYD35) over land

especially on broad-leaved forest

farmland

urban and bare soil. However

the accuracy of the CRMC over sea is lower than that of MYD35. In the view of the surface uniformity

the cloud detection over sea can enrich the brightness temperature information to optimize the corresponding performance. To sum up

while greatly improving the applicability to different optical sensors by not relying on special spectral range

the CRMC method can achieve a good cloud pixel identification effect for FY-3D MERSI Ⅱ images

with a similar hit rate compared with MODIS products. Also

the CRMC has certain anti-disturbance ability to haze.

关键词

Keywords

references

Ackerman S A , Strabala K I , Menzel W P , Frey R A , Moeller C C and Gumley L E . 1998 . Discriminating clear sky from clouds with MODIS . Journal of Geophysical Research : Atmospheres , 103 ( D24 ): 32141 - 32157 [ DOI: 10.1029/1998JD200032 http://dx.doi.org/10.1029/1998JD200032 ]

Chen X D , Zhang X , Liu L Y and Wang X F . 2019 . Enhanced multi-temporal cloud detection algorithm for optical remote sensing images . Journal of Remote Sensing , 23 ( 2 ): 280 - 290

陈曦东 , 张肖 , 刘良云 , 汪晓帆 . 2019 . 增强型多时相云检测 . 遥感学报 , 23 ( 2 ): 280- 290 [ 10.11834/jrs.20198017 http://dx.doi.org/10.11834/jrs.20198017 ]

Gao B C , Goetz A F H and Wiscombe W J . 1993 . Cirrus cloud detection from Airborne Imaging Spectrometer data using the 1.38 μm water vapor band . Geophysical Research Letters , 20 ( 4 ): 301 - 304 [ DOI: 10.1029/93GL00106 http://dx.doi.org/10.1029/93GL00106 ]

Hughes M J and Hayes D J . 2014 . Automated detection of cloud and cloud shadow in single-date landsat imagery using neural networks and spatial post-processing . Remote Sensing , 6 ( 6 ): 4907 - 4926 [ DOI: 10.3390/rs6064907 http://dx.doi.org/10.3390/rs6064907 ]

Irish R R , Barker J L , Goward S N and Arvidson T . 2006 . Characterization of the Landsat-7 ETM+ automated cloud-cover assessment (ACCA) algorithm . Photogrammetric Engineering and Remote Sensing , 72 ( 10 ): 1179 - 1188 [ DOI: 10.14358/PERS.72.10.1179 http://dx.doi.org/10.14358/PERS.72.10.1179 ]

Jiao Z T , Li X W , Wang J D and Zhang H . 2011 . Assessment of MODIS BRDF shape indicators . Journal of Remote Sensing , 15 ( 3 ): 432 - 456

焦子锑 , 李小文 , 王锦地 , 张虎 . 2011 . 评估MODIS的BRDF角度指数产品 . 遥感学报 , 15 ( 3 ): 432 - 456 [ DOI: 10.11834/jrs.20110073 http://dx.doi.org/10.11834/jrs.20110073 ]

Jin S K , Zhang M , Ma Y Y , Gong W , Chen C , Yang L K , Hu X Q , Liu B M , Chen N , Du B and Shi Y F . 2021 . Adapting the dark target algorithm to advanced MERSI sensor on the FengYun-3-D satellite: retrieval and validation of aerosol optical depth over land . IEEE Transactions on Geoscience and Remote Sensing , 59 ( 10 ): 8781 - 8797 [ DOI: 10.1109/TGRS.2020.3021021 http://dx.doi.org/10.1109/TGRS.2020.3021021 ]

Kotchenova S Y , Vermote E F , Levy R and Lyapustin A . 2008 . Radiative transfer codes for atmospheric correction and aerosol retrieval: intercomparison study . Applied Optics , 47 ( 13 ): 2215 - 2226 . [10.1364/AO.47.002215]

Li X H , Shen H F , Zhang L P , Zhang H Y , Yuan Q Q and Yang G . 2014 . Recovering quantitative remote sensing products contaminated by thick clouds and shadows using multitemporal dictionary learning . IEEE Transactions on Geoscience and Remote Sensing , 52 ( 11 ): 7086 - 7098 [ DOI: 10.1109/TGRS.2014.2307354 http://dx.doi.org/10.1109/TGRS.2014.2307354 ]

Li Z W , Shen H F , Cheng Q , Liu Y H , You S C and He Z Y . 2019 . Deep learning based cloud detection for medium and high resolution remote sensing images of different sensors . ISPRS Journal of Photogrammetry and Remote Sensing , 150 : 197 - 212 [ DOI: 10.1016/j.isprsjprs.2019.02.017 http://dx.doi.org/10.1016/j.isprsjprs.2019.02.017 ]

Lin C H , Lin B Y , Lee K Y and Chen Y C . 2015 . Radiometric normalization and cloud detection of optical satellite images using invariant pixels . ISPRS Journal of Photogrammetry and Remote Sensing , 106 : 107 - 117 [ DOI: 10.1016/j.isprsjprs.2015.05.003 http://dx.doi.org/10.1016/j.isprsjprs.2015.05.003 ]

Liu C L and Wu B F . 2004 . Application of cloud detection algorithm for the AVHRR data . Journal of Remote Sensing , 8 ( 6 ): 677 - 687

刘成林 , 吴炳方 . 2004 . NOAA AVHRR云标识技术的应用分析 . 遥感学报， 8 ( 6 ): 677 - 687 [ DOI: 10.11834/jrs.20040619 http://dx.doi.org/10.11834/jrs.20040619 ]

McClatchey R A , Fenn R W , Selby J E A , Volz F E and Garing J S . 1971 . Optical Properties of the Atmosphere. AFCRL-TR-71-0279, Environment Research Papers, No 354 , L . G . Hancom Fiel Bedford, Mass. USA

Meng F , Yang X M , Zhou C H and Li Z . 2017 . A sparse dictionary learning-based adaptive patch inpainting method for thick clouds removal from high-spatial resolution remote sensing imagery . Sensors (Basel) , 17 ( 9 ): 2130 [ DOI: 10.3390/s17092130 http://dx.doi.org/10.3390/s17092130 ]

Peng L K ， Liu L C ， Chen X H ， Chen J ， Cao X and Qiu Y A . 2021 . Generalization ability of cloud detection network for satellite imagery based on DeepLabv 3 +. National Remote Sensing Bulletin ， 25 （ 5 ）： 1169 - 1186

彭龙康，刘励聪，陈学泓，陈晋，曹鑫，邱岳安 . 2021 . 遥感影像云检测网络泛化性能研究：以DeepLabv3+为例 . 遥感学报， 25 （ 5 ）： 1169 - 1186 [ DOI: 10.11834/jrs.20210061 http://dx.doi.org/10.11834/jrs.20210061 ]

Ricchiazzi P , Yang S R , Gautier C and Sowle D . 1998 . SBDART: a research and teaching software tool for plane-parallel radiative transfer in the Earth's atmosphere . Bulletin of the American Meteorological Society , 79 ( 10 ): 2101 - 2114 [ DOI: 10.1175/1520-0477(1998)079<2101:SARATS>2.0.CO;2 http://dx.doi.org/10.1175/1520-0477(1998)079<2101:SARATS>2.0.CO;2 ]

Roujean J L , Leroy M and Deschamps P Y . 1992 . A bidirectional reflectance model of the Earth’s surface for the correction of remote sensing data . Journal of Geophysical Research , 97 ( D18 ): 20455 - 20468 [ DOI: 10.1029/92JD01411 http://dx.doi.org/10.1029/92JD01411 ]

Segal-Rozenhaimer M , Li A L , Das K and Chirayath V . 2020 . Cloud detection algorithm for multi-modal satellite imagery using convolutional neural-networks (CNN) . Remote Sensing of Environment , 237 : 111446 [ DOI: 10.1016/j.rse.2019.111446 http://dx.doi.org/10.1016/j.rse.2019.111446 ]

Stamnes K , Hamre B , Stamnes J J , Ryzhikov G , Biryulina M , Mahoney R , Hauss B and Sei A . 2011 . Modeling of radiation transport in coupled atmosphere-snow-ice-ocean systems . Journal of Quantitative Spectroscopy and Radiative Transfer , 112 ( 4 ): 714 - 726 [ DOI: 10.1016/j.jqsrt.2010.06.006 http://dx.doi.org/10.1016/j.jqsrt.2010.06.006 ]

Stowe L L , Davis P A and McClain E P . 1999 . Scientific basis and initial evaluation of the CLAVR-1 global clear cloud classification algorithm for the advanced very high resolution radiometer . Journal of Atmospheric and Oceanic Technology , 16 ( 6 ): 656 - 681 [ DOI: 10.1175/1520-0426(1999)016<0656:SBAIEO>2.0.CO;2 http://dx.doi.org/10.1175/1520-0426(1999)016<0656:SBAIEO>2.0.CO;2 ]

Sun L , Wei J , Wang J , Mi X T , Guo Y M , Lv Y , Yang Y K , Gan P , Zhou X Y , Jia C and Tian X P . 2016 . A Universal Dynamic Threshold Cloud Detection Algorithm (UDTCDA) supported by a prior surface reflectance database . Journal of Geophysical Research : Atmospheres , 121 ( 12 ): 7172 - 7196 [ DOI: 10.1002/2015JD024722 http://dx.doi.org/10.1002/2015JD024722 ]

Vermote E F , Tanre D , Deuze J L , Herman M and Morcette J J . 1997 . Second simulation of the satellite signal in the solar spectrum , 6 S: an overview. IEEE Transactions on Geoscience and Remote Sensing , 35 ( 3 ): 675 - 686 [ DOI: 10.1109/36.581987 http://dx.doi.org/10.1109/36.581987 ]

Wei J , Huang W , Li Z Q , Sun L , Zhu X L , Yuan Q Q , Liu L and Cribb M . 2020 . Cloud detection for Landsat imagery by combining the random forest and superpixels extracted via energy-driven sampling segmentation approaches . Remote Sensing of Environment , 248 : 112005 [ DOI: 10.1016/j.rse.2020.112005 http://dx.doi.org/10.1016/j.rse.2020.112005 ]

Wei L S , Shang H Z , Husi L T , Ma R , Hu D H , Zhao K F , Si F Q and Shi J C . 2021 . Cloud detection algorithm based on GF-5 DPC data . National Remote Sensing Bulletin , 25 ( 10 )： 2053 - 2066

伟乐斯，尚华哲，胡斯勒图，马润，胡大海，朝克夫，司福祺，施建成 . 2021 . GF-5 DPC数据的云检测方法研究 . 遥感学报， 25 ( 10 )： 2053 - 2066 [ DOI: 10.11834/jrs.20210226 http://dx.doi.org/10.11834/jrs.20210226 ]

Xu N , Niu X H , Hu X Q , Wang X H , Wu R H , Chen S S , Chen L , Sun L , Ding L , Yang Z D and Zhang P . 2018 . Prelaunch calibration and radiometric performance of the advanced MERSI Ⅱ on FengYun-3D . IEEE Transactions on Geoscience and Remote Sensing , 56 ( 8 ): 4866 - 4875 [ DOI: 10.1109/TGRS.2018.2841827 http://dx.doi.org/10.1109/TGRS.2018.2841827 ]

Yang Z D , Zhang P , Gu S Y , Hu X Q , Tang S H , Yang L K , Xu N , Zhen Z J , Wang L , Wu Q , Dou F L , Liu R X , Wu X , Zhu L , Zhang L Y , Wang S J , Sun Y Q and Bai W H . 2019 . Capability of Fengyun-3D satellite in earth system observation . Journal of Meteorological Research , 33 ( 6 ): 1113 - 1130 [ DOI: 10.1007/s13351-019-9063-4 http://dx.doi.org/10.1007/s13351-019-9063-4 ]

Yu C Y , Liang B , Bao C G , Li M , Hu Y Y , Lan D D , Xu Y , Lu H and Ma M H . 2013 . Study on eutrophication status and trend in Bohai Sea . Marine Environmental Science , 32 ( 2 ): 175 - 177

于春艳 , 梁斌 , 鲍晨光 , 李冕 , 胡莹莹 , 兰冬东 , 许妍 , 洛昊 , 马明辉 . 2013 . 渤海富营养化现状及趋势研究 . 海洋环境科学 , 32 ( 2 ): 175 - 177 [ DOI: CNKI:SUN:HYHJ.0.2013-02-004 http://dx.doi.org/CNKI:SUN:HYHJ.0.2013-02-004 ]

Luo J C , Zhou C H and Yang Y . 2001 . ANN Remote Sensing Classification Model and Its Integration Approach with Geo-knowledge . Journal of Remote Sensing , 5 ( 2 ): 122 - 129

骆剑承 , 周成虎 , 杨艳 . 2001 . 人工神经网络遥感影像分类模型及其与知识集成方法研究 . 遥感学报 , 5 ( 2 ): 122- 129 [ 10.11834/jrs.20010210 http://dx.doi.org/10.11834/jrs.20010210 ]

Zhang C , Li H F and Shen H F . 2020 . Thin cloud correction for GF-5 AHSI visible images by combining statistical information with scattering model . Journal of Remote Sensing , 24 ( 4 ): 368 - 378

张弛 , 李慧芳 , 沈焕锋 . 2020 . 联合统计信息与散射模型的GF-5 AHSI可见光影像薄云校正 . 遥感学报 , 24 ( 4 ): 368- 378 [ 10.11834/jrs.20209271 http://dx.doi.org/10.11834/jrs.20209271 ]

Zhou H M , Yang W X and Lu X . 1995 . Study on detecting cloud of NOAA satellite image . Journal of Remote Sensing , 10 ( 2 ): 137 - 142

周红妹 , 杨星卫 , 陆贤 . 1995 . NOAA气象卫星云检测方法的研究 . 环境遥感 , 10 ( 2 ): 137 - 142 [ DOI: 10.11834/jrs.1995019 http://dx.doi.org/10.11834/jrs.1995019 ]

Zhu Z and Woodcock C E . 2012 . Object-based cloud and cloud shadow detection in Landsat imagery . Remote Sensing of Environment , 118 : 83 - 94 [ DOI: 10.1016/j.rse.2011.10.028 http://dx.doi.org/10.1016/j.rse.2011.10.028 ]

Alert me when the article has been cited

提交

GF-7 satellite imagery DSM improvement by fusing laser altimetry data without ground control points in mining areas

A neural network-based model for compensation of inversion errors in satellite-borne GNSS-R sea surface heights

Near-surface soil freeze/thaw status datasets of China based on FY-3/MWRI data from 2010 to 2021

Identification and estimation of sea ice in Liaodong Bay using HY-1C/D satellite images

A neural network-based forward operator for assimilating the FY-4A visible reflectance