收稿：2020-08-31，
纸质出版：2021-07-07
移动端阅览
条件随机场模型由于其较强的上下文信息建模能力，被广泛应用于建筑物提取任务中。然而，面对高分辨率遥感影像丰富的地物信息，基于条件随机场的提取方法存在建筑物边界模糊的问题。本文提出了一种全局局部细节感知条件随机场框架，该框架提出全局局部一体化D-LinkNet，在有效利用多尺度建筑物信息的同时保留局部结构信息，解决了传统条件随机场一元势能丢失边界信息的问题。同时，该框架融合分割先验以缓解建筑物类内光谱差异较大的影响，利用更大尺度的上下文信息来精确提取建筑物，并引入局部类别标记代价从而保持细节信息以获取清晰的建筑物边界。实验结果表明，该框架在WHU卫星和航空数据集上的精度评价指标均优于其他对比方法，其IoU分别达89.82%和91.72%，对于复杂场景下的建筑物信息能够获得较好的提取效果。
Obtaining building distribution maps accurately and quickly has very important research value. For instance
it can help governments and non-governmental organizations plan urban infrastructure construction and assist disaster relief work. The Conditional Random Field (CRF) is widely used in building extraction tasks because of its flexible context information modeling and detail extraction capabilities. However
problems
such as blurry building boundaries
still exist when the CRF is used to extract buildings from high spatial resolution remote sensing images.
This study proposed a Global-Local-Aware conditional random fields framework for building extraction. Global-Local-Aware D-LinkNet (GLD-LinkNet) is proposed to solve the boundary blur problem of unary potential in this framework. GLD-LinkNet makes up for the loss of local structural information by D-LinkNet while using multi-scale building information effectively. In addition
the segmentation priors are fused
and the label cost is introduced. The pairwise potential reflects the linear combination of the spatial relationship of neighboring pixels and the cost of the local class label. It can maintain the detailed information inside the buildings effectively. In addition
to solve the problem of spectral similarity between buildings and noise
a larger range of context information is used to fuse segmentation prior to the extraction of buildings. The framework can eliminate the influence of image noise and spectral diversity within class. Moreover
it can also keep the detailed information of ground objects and determine the clear boundaries of buildings.
Experiments were carried out on the aerial building dataset and the satellite building dataset of the WHU building datasets. Experimental results demonstrate that the proposed dilated and segmented conditional random field framework is superior to the state-of-the art methods in terms of accuracy and IOU. The model proposed in this study
which performs well for building extraction of complex scenes
can maintain the detailed information of buildings effectively. In addition
it removes the small building blocks mistakenly extracted by D-LinkNet
and the problem of blurred building boundaries has also been improved effectively.
The use of the global and local integrated D-LinkNet to model the unary potential of CRF can realize the effective combination of building features of different scales. It makes the structure of the obtained buildings more complete. Furthermore
by adding a segmentation prior to the construction of pairwise potential
a building classification map with a clean background can be obtained. The introduction of the local class label cost term also meets the high requirements of the building extraction task for the extraction of building detail information and can capture detailed information that is difficult to identify on the network. The proposed model was tested on aerial and satellite datasets
and the IoU indicators on the two datasets reach 91.72% and 89.82%
respectively. These values imply that the framework can adapt to aerial and satellite datasets.
In the future
we will further study the application of large-scale high-resolution remote sensing images in building extraction and try to combine multi-source geographic information data to extract more complete building information.
AkÇay H G and Aksoy S . 2008 . Automatic detection of geospatial objects using multiple hierarchical segmentations . IEEE Transactions on Geoscience and Remote Sensing , 46 ( 7 ): 2097 - 2111 [ DOI: 10.1109/TGRS.2008.916644 http://dx.doi.org/10.1109/TGRS.2008.916644 ]
Boykov Y , Veksler O and Zabih R . 2001 . Fast approximate energy minimization via graph cuts . IEEE Transactions on Pattern Analysis and Machine Intelligence , 23 ( 11 ): 1222 - 1239 [ DOI: 10.1109/34.969114 http://dx.doi.org/10.1109/34.969114 ]
Boykov Y Y and Jolly M P . 2001 . Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images // Proceedings Eighth IEEE International Conference on Computer Vision . ICCV 2001. Vancouver: IEEE: 105 - 112 [ DOI: 10.1109/ICCV.2001.937505 http://dx.doi.org/10.1109/ICCV.2001.937505 ]
Chaurasia A and Culurciello E . 2017 . Linknet: exploiting encoder representations for efficient semantic segmentation // 2017 IEEE Visual Communications and Image Processing (VCIP) . St. Petersburg : IEEE : 1 - 4 [ DOI: 10.1109/VCIP.2017.8305148 http://dx.doi.org/10.1109/VCIP.2017.8305148 ]
Chen K Q , Fu K , Gao X , Yan M L , Sun X and Zhang H . 2017 . Building extraction from remote sensing images with deep learning in a supervised manner // 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS) . Fort Worth : IEEE : 1672 - 1675 [ DOI: 10.1109/IGARSS.2017.8127295 http://dx.doi.org/10.1109/IGARSS.2017.8127295 ]
Chen K Q , Gao X , Yan M L , Zhang Y and Sun X . 2020 . Building extraction in pixel level from aerial imagery with a deep encoder-decoder network . Journal of Remote Sensing , 24 ( 9 ): 1134 - 1142
陈凯强 , 高鑫 , 闫梦龙 , 张跃 , 孙显 . 2020 . 基于编解码网络的航空影像像素级建筑物提取 . 遥感学报 , 24 ( 9 ): 1134- 1142 [ DOI: 10.11834/jrs.20209056 http://dx.doi.org/10.11834/jrs.20209056 ]
Deng J , Dong W , Socher R , Li L J , Li K and Li F F . 2009 . ImageNet: a large-scale hierarchical image database // 2009 IEEE Conference on Computer Vision and Pattern Recognition . Miami : IEEE : 248 - 255 [ DOI: 10.1109/CVPR.2009.5206848 http://dx.doi.org/10.1109/CVPR.2009.5206848 ]
Gao X J , Wang M W , Yang Y W and Li G Q . 2018 . Building extraction from RGB VHR images using shifted shadow algorithm . IEEE Access , 6 : 22034 - 22045 [ DOI: 10.1109/ACCESS.2018.2819705 http://dx.doi.org/10.1109/ACCESS.2018.2819705 ]
He K M , Zhang X Y , Ren S Q and Sun J . 2016 . Deep residual learning for image recognition // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Las Vegas : IEEE : 770 - 778 [ DOI: 10.1109/CVPR.2016.90 http://dx.doi.org/10.1109/CVPR.2016.90 ]
Huang J F , Zhang X C , Xin Q C , Sun Y and Zhang P C . 2019 . Automatic building extraction from high-resolution aerial images and LiDAR data using gated residual refinement network . ISPRS Journal of Photogrammetry and Remote Sensing , 151 : 91 - 105 [ DOI: 10.1016/j.isprsjprs.2019.02.019 http://dx.doi.org/10.1016/j.isprsjprs.2019.02.019 ]
Huertas A and Nevatia R . 1988 . Detecting buildings in aerial images . Computer Vision, Graphics, and Image Processing , 41 ( 2 ): 131 - 152 [ DOI: 10.1016/0734-189X(88)90016-3 http://dx.doi.org/10.1016/0734-189X(88)90016-3 ]
Ji S P , Wei S Q and Lu M . 2019b . A scale robust convolutional neural network for automatic building extraction from aerial and satellite imagery . International Journal of Remote Sensing , 40 ( 9 ): 3308 - 3322 [ DOI: 10.1080/01431161.2018.1528024 http://dx.doi.org/10.1080/01431161.2018.1528024 ]
Jun W , Qiming Q , Xin Y , Jianhua W , Xuebin Q and Xiucheng Y . 2016 . A Survey of Building Extraction Methods from Optical High Resolution Remote Sensing Imagery , Remote Sensing Technology and Application , 31 : 653 - 62
Kang W C , Xiang Y M , Wang F and You H J . 2019 . EU-Net: an efficient fully convolutional network for building extraction from optical remote sensing images . Remote Sensing , 11 ( 23 ): 2813 [ DOI: 10.3390/rs11232813 http://dx.doi.org/10.3390/rs11232813 ]
Karantzalos K and Paragios N . 2009 . Recognition-driven two-dimensional competing priors toward automatic and accurate building detection . IEEE Transactions on Geoscience and Remote Sensing , 47 ( 1 ): 133 - 144 [ DOI: 10.1109/TGRS.2008.2002027 http://dx.doi.org/10.1109/TGRS.2008.2002027 ]
Krähenbühl P and Koltun V . 2012 . Efficient inference in fully connected CRFs with Gaussian edge potentials // Advances in Neural Information Processing Systems . London : MIT : 109 - 117
Krizhevsky A , Sutskever I and Hinton G E . 2012 . ImageNet classification with deep convolutional neural networks // Proceedings of the 25th International Conference on Neural Information Processing Systems . Lake Tahoe : Curran Associates Inc. : 1097 - 1105
Kumar S and Hebert M H . 2003 . Discriminative random fields: A discriminative framework for contextual interaction in classification // Proceedings Ninth IEEE International Conference on Computer Vision . Nice : IEEE : 1150 - 1157 [ DOI: 10.1109/ICCV.2003.1238478 http://dx.doi.org/10.1109/ICCV.2003.1238478 ]
Lafferty J D , McCallum A and Pereira F C N . 2001 . Conditional random fields: probabilistic models for segmenting and labeling sequence data // Proceedings of the Eighteenth International Conference on Machine Learning . San Francisco : Morgan Kaufmann Publishers : 282 - 289 [ DOI: 10.5555/645530.655813 http://dx.doi.org/10.5555/645530.655813 ]
Lai X D , Yang J R , Li Y X and Wang M W . 2019 . A building extraction approach based on the fusion of LiDAR point cloud and elevation map texture features . Remote Sensing , 11 ( 14 ): 1636 [ DOI: 10.3390/rs11141636 http://dx.doi.org/10.3390/rs11141636 ]
Li E , Femiani J , Xu S B , Zhang X P and Wonka P . 2015 . Robust rooftop extraction from visible band images using higher order CRF . IEEE Transactions on Geoscience and Remote Sensing , 53 ( 8 ): 4483 - 4495 [ DOI: 10.1109/TGRS.2015.2400462 http://dx.doi.org/10.1109/TGRS.2015.2400462 ]
Li Q Y , Shi Y L , Huang X and Zhu X X . 2020 . Building footprint generation by integrating convolution neural network with Feature Pairwise Conditional Random Field (FPCRF) . IEEE Transactions on Geoscience and Remote Sensing , 58 ( 11 ): 7502 - 7519 [ DOI: 10.1109/TGRS.2020.2973720 http://dx.doi.org/10.1109/TGRS.2020.2973720 ]
Liu F Y , Lin G H and Shen C H . 2015 . CRF learning with CNN features for image segmentation . Pattern Recognition , 48 ( 10 ): 2983 - 2992 [ DOI: 10.1016/j.patcog.2015.04.019 http://dx.doi.org/10.1016/j.patcog.2015.04.019 ]
Liu H , Luo J C , Huang B , Hu X D , Sun Y W , Yang Y P , Xu N and Zhou N . 2019a . DE-Net: deep encoding network for building extraction from high-resolution remote sensing imagery . Remote Sensing , 11 ( 20 ): 2380 [ DOI: 10.3390/rs11202380 http://dx.doi.org/10.3390/rs11202380 ]
Liu P H , Liu X P , Liu M X , Shi Q , Yang J X , Xu X C and Zhang Y Y . 2019b . Building footprint extraction from high-resolution images via spatial residual inception convolutional neural network . Remote Sensing , 11 ( 7 ): 830 [ DOI: 10.3390/rs11070830 http://dx.doi.org/10.3390/rs11070830 ]
Mayer H . 1999 . Automatic object extraction from aerial imagery—a survey focusing on buildings . Computer Vision and Image Understanding , 74 ( 2 ): 138 - 149 [10.1006/cviu.1999. 0750 ]
Qiao C , Luo J C , Wu Q Y , Shen Z F and Wang H . 2008 . Object-Oriented method based urban building extraction from high resolution remote sensing image . Geography and Geo-Information Science , 24 ( 5 ): 36 - 39
乔程 , 骆剑承 , 吴泉源 , 沈占锋 , 王宏 . 2008 . 面向对象的高分辨率影像城市建筑物提取 . 地理与地理信息科学 , 24 ( 5 ): 36 - 39
Shrestha S and Vanneschi L . 2018 . Improved fully convolutional network with conditional random fields for building extraction . Remote Sensing , 10 ( 7 ): 1135 [ DOI: 10.3390/rs10071135 http://dx.doi.org/10.3390/rs10071135 ]
Sun J X , Li W H , Zhang Y and Gong W G . 2019 . Building segmentation of remote sensing images using deep neural networks and domain transform CRF // Proceedings Volume 11155 , Image and Signal Processing for Remote Sensing XXV. Strasbourg : SPIE : 111550 N [ DOI: 10.1117/12.2532662 http://dx.doi.org/10.1117/12.2532662 ]
Szummer M , Kohli P and Hoiem D . 2008 . Learning CRFs using graph cuts // European Conference on Computer Vision . Marseille : ECCV
Teichmann M T T and Cipolla R . 2018 . Convolutional CRFs for semantic segmentation [EB/OL]. https://arxiv.org/abs/1805.04777 https://arxiv.org/abs/1805.04777
Wang Y J , Jiang T P , Yu M , Tao S B , Sun J and Liu S . 2020 . Semantic-based building extraction from LiDAR point clouds using contexts and optimization in complex environment . Sensors , 20 ( 12 ): 3386 [ DOI: 10.3390/s20123386 http://dx.doi.org/10.3390/s20123386 ]
Wegne J D , Soergel U and Rosenhahn B . 2011 . Segment-based building detection with conditional random fields // 2011 Joint Urban Remote Sensing Event . Munich : IEEE : 205 - 208 [ DOI: 10.1109/JURSE.2011.5764756 http://dx.doi.org/10.1109/JURSE.2011.5764756 ]
Wu K S , Otoo E and Suzuki K . 2009 . Optimizing two-pass connected-component labeling algorithms . Pattern Analysis and Applications , 12 ( 2 ): 117 - 135 [ DOI: 10.1007/s10044-008-0109-y http://dx.doi.org/10.1007/s10044-008-0109-y ]
Xu L L , Clausi D A , Li F and Wong A . 2017 . Weakly supervised classification of remotely sensed imagery using label constraint and edge penalty . IEEE Transactions on Geoscience and Remote Sensing , 55 ( 3 ): 1424 - 1436 [ DOI: 10.1109/TGRS.2016.2623942 http://dx.doi.org/10.1109/TGRS.2016.2623942 ]
Xu Y Y , Wu L , Xie Z and Chen Z L . 2018 . Building extraction in very high resolution remote sensing imagery using deep learning and guided filters . Remote Sensing , 10 ( 1 ): 144 [ DOI: 10.3390/rs10010144 http://dx.doi.org/10.3390/rs10010144 ]
Zhao J , Zhong Y F and Zhang L P . 2015 . Detail-preserving smoothing classifier based on conditional random fields for high spatial resolution remote sensing imagery . IEEE Transactions on Geoscience and Remote Sensing , 53 ( 5 ): 2440 - 2452 [ DOI: 10.1109/TGRS.2014.2360100 http://dx.doi.org/10.1109/TGRS.2014.2360100 ]
Zhong P and Wang R S . 2010 . Learning conditional random fields for classification of hyperspectral images . IEEE transactions on image processing , 19 ( 7 ): 1890 - 1907 [ DOI: 10.1109/TIP.2010.2045034 http://dx.doi.org/10.1109/TIP.2010.2045034 ]
Zhou L C , Zhang C and Wu M . 2018 . D-LinkNet: linknet with pretrained encoder and dilated convolution for high resolution satellite imagery road extraction // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) . Salt Lake City : IEEE : 192 - 1924 [ DOI: 10.1109/CVPRW.2018.00034 http://dx.doi.org/10.1109/CVPRW.2018.00034 ]
Zhu Q , Wang J , Liu X Y , Zhou Z W , Ma Z W and Gao X J . 2020 . Accurate extraction of buildings from remote sensing images based on improved Markov random field. Computer and Modernization, (7): 104 - 110
朱恰 , 王建 , 刘星雨 , 周再文 , 马紫雯 , 高贤君 . 2020 . 基于改进MRF的遥感影像建筑物精提取. 计算机与现代化, (7): 104 - 110 [DOI: 10.3969/j.issn.1006-2475.2020.07.020]
Zhu Q Q . 2018 . Topic Model for High Resolution Remote Sensing Imagery Semantic Scene Understanding . Wuhan : Wuhan University : 147
朱祺琪 . 2018 . 面向高分辨率遥感影像场景语义理解的概率主题模型研究 . 武汉 : 武汉大学 : 147
相关作者
相关机构
京公网安备11010802024621
