Development of high-resolution satellite ground processing system

Bingbing WANG; Wenyong YU; Xiaoxiang LONG; Jun LIN; Haibo WANG; Chaoyu SONG; Qi CHEN; Shule GE; Shuai LI

doi:10.11834/jrs.20210369

Technologies and Methodologies | Views : 0 下载量: 381 CSCD: 1 更多指标

PDF
Export
Share
Collection
Album

Development of high-resolution satellite ground processing system
Vol. 25, Issue 9, Pages: 1946-1963(2021)
Published： 07 September 2021 ，
DOI： 10.11834/jrs.20210369

扫描看全文

王冰冰，喻文勇，龙小祥，林军，王海波，宋超宇，陈琦，葛曙乐，李帅.2021.高分辨率卫星地面处理系统研制.遥感学报，25（9）： 1946-1963

Wang B B， Yu W Y， Long X X， Lin J， Wang H B，Song C Y， Chen Q， Ge S L and Li S. 2021. Development of high-resolution satellite ground processing system. National Remote Sensing Bulletin， 25（9）：1946-1963
王冰冰，喻文勇，龙小祥，林军，王海波，宋超宇，陈琦，葛曙乐，李帅.2021.高分辨率卫星地面处理系统研制.遥感学报，25（9）： 1946-1963 DOI： 10.11834/jrs.20210369.

Wang B B， Yu W Y， Long X X， Lin J， Wang H B，Song C Y， Chen Q， Ge S L and Li S. 2021. Development of high-resolution satellite ground processing system. National Remote Sensing Bulletin， 25（9）：1946-1963 DOI： 10.11834/jrs.20210369.

摘要

高分卫星地面处理系统是高分辨率对地观测系统重大专项的重要组成部分，实现高分一号到高分七号卫星各载荷数据录入、各级标准产品生产、存档、仿真、评价、分发服务、定标检校以及有效载荷管理等。地面处理系统采用了基于HADOOP平台的计算与存储统一的云计算地面处理架构，通过HDFS分布式文件系统将节点内部存储聚合为海量数据存储系统，通过YARN资源管理系统实现海量数据的统一调度和大规模并行处理，结合多光谱、高光谱、红外、微波、激光雷达和大气探测等多种卫星遥感数据高精度处理算法，构建多星通用大数据综合处理平台，大幅提升数据综合处理、数据存储与管理、分发服务等能力，为各类用户提供高精度、高质量的标准数据产品。

Abstract

The ground processing system of the high-resolution satellites is an important component of the high-resolution earth observation system. Following the principles of standardization

safety and reliability

maintainability

economy

and advanced technology

a remote sensing data comprehensive processing platform

which can realize the functions of data ingestion

multilevel standard data production

archiving

simulation

evaluation

distribution services

radiometric and geometric calibration

observation management of the payloads on GF-1 to GF-7 satellites

is designed and developed.

The ground processing system adopts the cloud computing ground processing architecture based on the HADOOP platform and integrates the internal storage of nodes into a massive data storage system through the HDFS distributed file system. It also realizes the united scheduling and large-scale parallel processing of massive data through the Yarn resource management system. Moreover

it studies and develops the high-precision processing algorithms of the multispectral

hyper-spectral

infrared

microwave

Lidar

and atmosphere detection payloads on the satellites.

The high-resolution satellite ground processing system builds a multisatellite comprehensive processing platform for massive satellite data. The capabilities of the ground processing system have been substantially improved in data integrated processing

data storage and management

and data distribution services; it provides standard data products to users with high accuracy and quality.

It has 400 trillion calculations per second and highly automated data processing capacity of more than 20 TB per day. It can process a wide range of satellite payload types for GF-1 to GF-7 satellites. In the normal mode

the data processing time of each orbit is not more than 2 hours

and the emergency processing time is not more than 45 minutes.The system has 25 PB scalable storage and management capabilities of massive remote sensing image data. The products are stored online and near line within the life cycle

and level 0 data are permanently archived.

For data distribution

the system can distribute more than four million scene data every year and supports 1000 external users to visit and retrieve the data on the service website at the same time. The average retrieval response time is less than 10 seconds. It also supports 200 external users to download data products online.

The system has the continuous operation ability of 7×24 hours. The operation stability is better than 99%. The main and backup switches of the key business equipment are completed within 10 minutes after the failure. The mean time to repair is 4 hours.

The high-resolution satellite ground processing system has realized satellite data ingestion

standard product processing

archiving

simulation

evaluation

distribution

calibration

payload management

and data sharing services. It has also realized the comprehensive data processing of a wide range of remote sensing data

which cover optical

submeter high spatial resolution

multimode radar

geosynchronous orbit

hyperspectral resolution

atmospheric

stereo mapping

and Lidar earth observation. A multisatellite and multiload remote sensing data product system have been established to provide multisource remote sensing data for land

ocean

agriculture

disaster reduction

forestry

water conservancy

environment

earthquake

meteorology

transportation

surveying

and mapping users. It has played an important role in promoting China’s economic construction

social development

scientific and technological progress

and serving China’s belt and road initiatives.

关键词

云计算高分卫星地面系统遥感数据处理

Keywords

cloud computinghigh-resolution satelliteground processing systemremote sensing data processing

references

Arvidson R, Billingsley F, Chase R, Chavez Jr P, Devirian M and Mosher F. 1986. Earth observing system, volume IIa: data and information system, report of the EOS data panel[EB/OL]. https://commons.erau.edu/publication/551/https://commons.erau.edu/publication/551/[2020-08-25]

Chen L Y, Liang X D and Ding C B. 2010. Non-uniform reconstruction method in SAR imaging. Journal of System Simulation, 22(5): 1242-1245

陈龙永, 梁兴东, 丁赤飚. 2010. 一种SAR成像中的非均匀采样重构方法. 系统仿真学报, 22(5): 1242-1245

Deng Y K, Jia X X, Feng J and Xu W. 2010. Sliding spotlight SAR data processing using the azimuth frequency de-ramping algorithm. Journal of Electronics and Information Technology, 32(11): 2655-2660

邓云凯, 贾小雪, 冯锦, 徐伟. 2010. 基于方位频率去斜的滑动聚束SAR成像算法. 电子与信息学报, 32(11): 2655-2660 [DOI: 10.3724/SP.J.1146.2009.01473http://dx.doi.org/10.3724/SP.J.1146.2009.01473]

Esfandiari M, Ramapriyan H, Behnke J and Sofinowski E. 2006 Evolution of the Earth Observing System (EOS) Data and Information System (EOSDIS)//2006 IEEE International Symposium on Geoscience and Remote Sensing. Denver: IEEE: 309-312 [DOI: 10.1109/IGARSS.2006.84http://dx.doi.org/10.1109/IGARSS.2006.84]

Fan Q and Liu S L. 2011. Study on azimuth non-uniform sampling of spaceborne DPC MAB SAR. Journal of Test and Measurement Technology, 25(5): 406-413

范强, 刘树立. 2011. 星载分离相位中心方位多波束SAR方位向非均匀采样. 测试技术学报, 25(5): 406-413 [DOI: 10.3969/j.issn.1671-7449.2011.05.007http://dx.doi.org/10.3969/j.issn.1671-7449.2011.05.007]

Gu X F, Chen X F, Cheng T H, Li Z Q, Yu T, Xie D H and Xu H. 2011. In-flight polarization calibration methods of directional polarizedremote sensing camera DPC. Acta Physica Sinica, 60(7): 070702

顾行发, 陈兴峰, 程天海, 李正强, 余涛, 谢东海, 许华. 2011. 多角度偏振遥感相机DPC在轨偏振定标. 物理学报, 60(7): 070702 [DOI: 10.7498/aps.60.070702http://dx.doi.org/10.7498/aps.60.070702]

Han B, Zhang Y J, Hu D H and Huang J L. 2011. Research on mending of space-borne sliding spotlight SAR imaging model error. Journal of Electronics and Information Technology, 33(7): 1694-1699

韩冰, 张永军, 胡东辉, 黄佳丽. 2011. 星载滑动聚束SAR成像模型误差校正方法研究. 电子与信息学报, 33(7): 1694-1699 [DOI: 10.3724/SP.J.1146.2010.01259http://dx.doi.org/10.3724/SP.J.1146.2010.01259]

He F, Dong Z, Zhang Y S, Jin G H and Yu A X. 2020. Processing of spaceborne squinted sliding spotlight and HRWS TOPS mode data using 2-D baseband azimuth scaling. IEEE Transactions on Geoscience and Remote Sensing, 58(2): 938-955 [DOI: 10.1109/TGRS.2019.2941983http://dx.doi.org/10.1109/TGRS.2019.2941983]

Kobler B, Berbert J, Caulk P and Hariharan P C. 1995. Architecture and design of storage and data management for the NASA Earth observing system Data and Information System (EOSDIS)//Proceedings of IEEE 14th Symposium on Mass Storage Systems. Monterey: IEEE: 65-76 [DOI: 10.1109/MASS.1995.528217http://dx.doi.org/10.1109/MASS.1995.528217]

Li Y T, Chen Y G, Deng Y K and Yin C B. 2012. Data reconstruction method for azimuth multi-channel SAR. Journal of Electronics and Information Technology, 34(3): 628-632

李云涛, 陈永光, 邓云凯, 尹灿斌. 2012. 方位多通道合成孔径雷达数据重建方法. 电子与信息学报, 34(3): 628-632 [DOI: 10.3724/SP.J.1146.2011.00702http://dx.doi.org/10.3724/SP.J.1146.2011.00702]

Liu Y N, Sun D X, Cao K Q, Liu S F, Chai M Y, Liang J and Yuan J. 2020. Evaluation of GF-5 AHSI on-orbit instrument radiometric performance. Journal of Remote Sensing, 24(4): 352-359

刘银年, 孙德新, 曹开钦, 刘书锋, 柴孟阳, 梁建, 原娟. 2020. 高分五号可见短波红外高光谱相机在轨辐射性能评估. 遥感学报, 24(4): 352-359 [DOI：10.11834/jrs.20209258http://dx.doi.org/10.11834/jrs.20209258]

Loghin V. 2017. Copernicus program of the European Union for environmental monitoring and civil security. Annals “Valahia” University of Targoviste-Agriculture, 11(1): 53-55 [DOI: https://doi.org/10.1515/agr-2017-0010https://doi.org/10.1515/agr-2017-0010]

Redd N T. 2018. Earth observing system: monitoring the planet's climate[EB/OL]. Science and Astronomy, https://www.space.com/39566-earth-observing-system.htmlhttps://www.space.com/39566-earth-observing-system.html [2020-08-25]

Tang Y, Wang Y F and Zhang B C. 2007. A study of sliding spotlight SAR imaging mode. Journal of Electronics and Information Technology, 29(1): 26-29

唐禹, 王岩飞, 张冰尘. 2007. 滑动聚束SAR成像模式研究. 电子与信息学报, 29(1): 26-29

Thépaut J N, Dee D, Engelen R and Pinty B. 2018. The copernicus programme and its climate change service//IGARSS 2018-2018 IEEE International Geoscience and Remote Sensing Symposium. Valencia: IEEE: 1591-1593 [DOI: 10.1109/IGARSS.2018.8518067http://dx.doi.org/10.1109/IGARSS.2018.8518067]

Zhang M M, Meng B H, Qian H H, Han L, Chen H J, Wang Y and Hong J. 2017. Research on correction method of stray light in directional polarization camera. Acta Optica Sinica, 37(11): 1112003

张苗苗, 孟炳寰, 钱鸿鹄, 韩琳, 陈怀军, 王羿, 洪津. 2017. 多角度偏振成像仪杂散光校正方法研究. 光学学报, 37(11): 1112003 [DOI: 10.3788/aos201737.1112003http://dx.doi.org/10.3788/aos201737.1112003]

Zhou Y M and Liu T. 2021. The latest development in the field of foreign space earth observation. Satellite Application, (2): 47-53

周一鸣, 刘韬. 2021. 国外空间对地观测领域最新发展. 卫星应用, (2): 47-53 [DOI: 10.3969/j.issn.1674-9030.2021.02.011]

Alert me when the article has been cited

提交

GF quantitative remote sensing production system： Core design

Three development stages of mobile geographic information system technology

21st century daily seamless data cube reconstruction and seasonal to annual land cover and land use dynamics mapping-iMap （China） 1.0

Research framework of geographical conditions and big data

Turning remote sensing to cloud services:Technical research and experiment