浏览全部资源
扫码关注微信
Published： 07 February 2023 ，
扫 描 看 全 文
吕少云，李佳田，阿晓荟，杨超，杨汝春，尚晓梅.2023.Res_ASPP_UNet++：结合分离卷积与空洞金字塔的遥感影像建筑物提取网络.遥感学报，27（2）： 502-519
Lyu S Y， Li J T， A X H，Yang C， Yang R C and Shang X M. 2023. Res_ASPP_UNet++： Building an extraction network from remote sensing imagery combining depthwise separable convolution with atrous spatial pyramid pooling. National Remote Sensing Bulletin， 27（2）：502-519
针对连续池化操作丢失低层语义信息而导致建筑物提取精度低的问题，尝试以UNet++网络为基础，通过将编码器的标准卷积及最大池化替换成深度可分离卷积，以及在编码器末端利用不同采样率的空洞卷积构建多尺度空洞空间金字塔池化结构ASPP（Atrous Spatial Pyramid Pooling）来提升网络性能，并将改进后的建筑物提取网络称为残差空洞空间金字塔网络（Res_ASPP_UNet++）。为验证Res_ASPP_UNet++网络结构的有效性和适用性，以经过数据增强预处理的WHU和Massachusetts数据集作为数据源，对Res_ASPP_UNet++网络与目前常用的语义分割网络进行了试验和精度评估，并将Res_ASPP_UNet++网络与文献中的研究成果进行了对比。结果表明Res_ASPP_UNet++在模型参数量与精度两个方面均表现出优势，能够在大幅压缩模型参数量的前提下，显著提升建筑物提取精度，提取建筑物的边界更加平滑和精确，对不同尺度的建筑物表现出较强的泛化能力。
Building extraction from remote sensing imagery is an important research direction for the interpretation of remote sensing imagery. UNet++ is constructed from U-Net by connecting the decoders
resulting in densely connected skip connections
enabling dense feature propagation along skip connections and thus more flexible feature fusion at the decoder nodes. However
the traditional standard convolution in the encoder fails to fully capture the multiscale features of remote sensing imagery because of the single path for semantic feature extraction
which affects the segmentation performance of the network to some extent. To address this problem
we propose a building extraction method to improve the accuracy of building extraction from remote sensing imagery.
On the basis of UNet++
whose backbone is a deep residual network
we propose a building extraction network by replacing the standard convolution and max pooling in the encoder with depthwise separable convolution and applying an atrous spatial pyramid pooling structure (ASPP) to the end of the encoder. The network is referred to as the residual atrous spatial pyramid network (Res_ASPP_UNet++). On the basis of using dense and short connections to reduce the semantic gap between the encoder and the decoder
the Res_ASPP_UNet++ architecture uses multiscale ASPP made of several atrous convolutions with different sampling rates to sample the image in parallel to enrich semantic information by expanding the field of view and applies image-level features to encode the global context
avoiding segmentation errors caused by local features and improving the target segmentation performance.
Experiments are conducted to validate the effectiveness of the proposed methodology. We compare the frequently used semantic segmentation networks with the Res_ASPP_UNet++ architecture using the WHU and Massachusetts datasets as data sources and take intersection over union (IoU)
accuracy
precision and F1-score as evaluation indexes to evaluate the accuracy of building extraction. The experimental results are as follows: (1) The integrity of buildings extracted by Res_ASPP_UNet++ network is better than other segmentation networks on the whole
the boundary of the segmentation result is smoother and more accurate
and the result has less segmentation noise; (2) Compared with the compared semantic segmentation networks
the number of parameters of Res_ASPP_UNET ++ network is greatly compressed after improvement
the segmentation accuracy of the method is better on the whole
and it has great improvement to the UNET++ network through quantitative analysis; (3) Res_ASPP_UNET++ network has higher IoU index values than the existing building extraction algorithms listed in this paper; (4) Comparied with the proposed method without ASPP module
the value of each evaluation indexes of Res_ASPP_UNet++ with ASPP module is higher and the whole accuracy of the network with depthwise separable convolution is slightly higher than that of the network with max pooling; (5) Res_ASPP_UNet++ also extract the buildings on the Massachusetts dataset effectively and is better than other segmentation networks on the whole.
The following conclusions can be drawn from the experimental results. Multiscale ASPP and depthwise separable convolution can improve the ability to express the detailed features of the model and thus effectively improve the building extraction performance from remote sensing imagery on the basis of greatly compressing the model parameters. In addition
the Res_ASPP_UNet++ architecture is robust to buildings of different scales and types and has strong generalization ability on datasets of different resolutions and sources. In future work
we will improve the segmentation for areas with a complex distribution of buildings and overcome the missing segmentation of small buildings.
遥感影像建筑物提取UNet++深度可分离卷积深度残差结构空洞空间金字塔池化
remote sensing imagerybuilding extractionUNet++depthwise separable convolutiondeep residual structureatrous spatial pyramid pooling
Badrinarayanan V, Kendall A and Cipolla R. 2017. SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(12): 2481-2495 [DOI: 10.1109/TPAMI.2016.2644615http://dx.doi.org/10.1109/TPAMI.2016.2644615]
Berman M, Triki A R and Blaschko M B. 2018. The lovász-softmax loss: a tractable surrogate for the optimization of the intersection-over-union measure in neural networks//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 4413-4421 [DOI: 10.1109/CVPR.2018.00464http://dx.doi.org/10.1109/CVPR.2018.00464]
Chen K Q, Gao X, Yan M L, Zhang Y and Sun X. 2020. Building extraction in pixel level from aerial imagery with a deep encoder-decoder network. Journal of Remote Sensing, 24(9): 1134-1142
陈凯强, 高鑫, 闫梦龙, 张跃, 孙显. 2020. 基于编解码网络的航空影像像素级建筑物提取. 遥感学报, 24(9): 1134-1142 [DOI: 10.11834/jrs.20209056http://dx.doi.org/10.11834/jrs.20209056]
Chen L C, Papandreou G, Schroff F and Adam H. 2017. Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv: 1706.05587
Chen L C, Zhu Y K, Papandreou G, Schroff F and Adam H. 2018. Encoder-decoder with atrous separable convolution for semantic image segmentation//Proceedings of the 15th European Conference on Computer Vision. Munich: Springer: 801-818 [DOI: 10.1007/978-3-030-01234-2_49http://dx.doi.org/10.1007/978-3-030-01234-2_49]
Chollet F. 2017. Xception: deep learning with depthwise separable convolutions//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA: IEEE: 1800-1807 [DOI: 10.1109/CVPR.2017.195http://dx.doi.org/10.1109/CVPR.2017.195]
He K M, Zhang X Y, Ren S Q and Sun J. 2016. Deep residual learning for image recognition//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA: IEEE: 770-778 [DOI: 10.1109/CVPR.2016.90http://dx.doi.org/10.1109/CVPR.2016.90]
Howard A G, Zhu M L, Chen B, Kalenichenko D, Wang W J, Weyand T, Andreetto M and Adam H. 2017. MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv: 1704.04861
Huang X and Zhang L P. 2011. A multidirectional and multiscale morphological index for automatic building extraction from multispectral GeoEye-1 imagery. Photogrammetric Engineering and Remote Sensing, 77(7): 721-732 [DOI: 10.14358/PERS.77.7.721http://dx.doi.org/10.14358/PERS.77.7.721]
Ji S P, Tian S Q and Zhang C. 2020. Urban land cover classification and change detection using fully atrous convolutional neural network. Geomatics and Information Science of Wuhan University, 45(2): 233-241
季顺平, 田思琦, 张驰. 2020. 利用全空洞卷积神经元网络进行城市土地覆盖分类与变化检测. 武汉大学学报(信息科学版), 45(2): 233-241 [DOI: 10.13203/j.whugis20180481http://dx.doi.org/10.13203/j.whugis20180481]
Kang W C, Xiang Y M, Wang F and You H J. 2019. EU-Net: an efficient fully convolutional network for building extraction from optical remote sensing images. Remote Sensing, 11(23): 2813 [DOI: 10.3390/rs11232813http://dx.doi.org/10.3390/rs11232813]
Krizhevsky A, Sutskever I and Hinton G E. 2012. ImageNet classification with deep convolutional neural networks//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, Nevada: Curran Associates Inc.: 1097-1105
Lin G S, Milan A, Shen C H and Reid I. 2017. RefineNet: multi-path refinement networks for high-resolution semantic segmentation//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA: IEEE: 5168-5177 [DOI: 10.1109/CVPR.2017.549http://dx.doi.org/10.1109/CVPR.2017.549]
Lin J B, Jing W P, Song H B and Chen G S. 2019. ESFNet: efficient network for building extraction from high-resolution aerial images. IEEE Access, 7: 54285-54294 [DOT: 10.1109/ACCESS.2019.2912822http://dx.doi.org/10.1109/ACCESS.2019.2912822]
Mnih V. 2013. Machine Learning for Aerial Image Labeling. Toronto: University of Toronto
Pesaresi M, Gerhardinger A and Kayitakire F. 2008. A robust built-up area presence index by anisotropic rotation-invariant textural measure. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 1(3): 180-192 [DOI: 10.1109/JSTARS.2008.2002869http://dx.doi.org/10.1109/JSTARS.2008.2002869]
Ronneberger O, Fischer P and Brox T. 2015. U-Net: convolutional networks for biomedical image segmentation//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich, Germany: Springer: 234-241 [DOI: 10.1007/978-3-319-24574-4_28http://dx.doi.org/10.1007/978-3-319-24574-4_28]
Saito S and Aoki Y. 2015. Building and road detection from large aerial imagery//Proceedings of SPIE 9405, Image Processing: Machine Vision Applications VIII. San Francisco: SPIE [DOI: 10.1117/12.2083273http://dx.doi.org/10.1117/12.2083273]
Shelhamer E, Long J and Darrell T. 2017. Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4): 640-651 [DOI: 10.1109/TPAMI.2016.2572683http://dx.doi.org/10.1109/TPAMI.2016.2572683]
Sherrah J. 2016. Fully convolutional networks for dense semantic labelling of high-resolution aerial imagery. arXiv preprint arXiv: 1606.02585
Simonyan K and Zisserman A. 2015. Very deep convolutional networks for large-scale image recognition//Proceedings of 3rd International Conference on Learning Representations. San Diego: ICLR: 463-476
Szegedy C, Liu W, Jia Y Q, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V and Rabinovich A. 2015. Going deeper with convolutions//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA: IEEE: 1-9 [DOT: 10.1109/cvpr.2015.7298594]
Tan Q L. 2010. Urban building extraction from VHR multi-spectral images using object-based classification. Acta Geodaetica et Cartographica Sinica, 39(6): 618-623
谭衢霖. 2010. 高分辨率多光谱影像城区建筑物提取研究. 测绘学报, 39(6): 618-623
Tao W B, Liu J and Tian J W. 2003. A new approach to extract rectangle building automatically from aerial images. Chinese Journal of Computers, 26(7): 866-873
陶文兵, 柳健, 田金文. 2003. 一种新型的航空图像城区建筑物自动提取方法. 计算机学报, 26(7): 866-873 [DOI: 10.3321/j.issn:0254-4164.2003.07.014http://dx.doi.org/10.3321/j.issn:0254-4164.2003.07.014]
Wang J, Qin Q M, Ye X, Wang J H, Qin X B and Yang X C. 2016. A survey of building extraction methods from optical high resolution remote sensing imagery. Remote Sensing Technology and Application, 31(4): 653-662
王俊, 秦其明, 叶昕, 王建华, 秦雪彬, 杨绣丞. 2016. 高分辨率光学遥感图像建筑物提取研究进展. 遥感技术与应用, 31(4): 653-662 [DOI: 10.11873/j.issn.1004-0323.2016.4.0653http://dx.doi.org/10.11873/j.issn.1004-0323.2016.4.0653]
Wang W, Yang X C, Qin X B, Ye X and Qin Q M. 2015. An efficient approach for automatic rectangular building extraction from very high resolution optical satellite imagery. IEEE Geoscience and Remote Sensing Letters, 12(3): 487-491 [DOI: 10.1109/LGRS.2014.2347332http://dx.doi.org/10.1109/LGRS.2014.2347332]
Wu G M, Chen Q, Shibasaki R, Guo Z L, Shao X W and Xu Y W. 2018. High precision building detection from aerial imagery using a U-net like convolutional architecture. Acta Geodaetica et Cartographica Sinica, 47(6): 864-872
伍广明, 陈奇, Shibasaki R, 郭直灵, 邵肖伟, 许永伟. 2018. 基于U型卷积神经网络的航空影像建筑物检测. 测绘学报, 47(6): 864-872 [DOI: 10.11947/j.AGCS.2018.20170651http://dx.doi.org/10.11947/j.AGCS.2018.20170651]
Wu G M, Shao X W, Guo Z L, Chen Q, Yuan W, Shi X D, Xu Y W and Shibasaki R. 2018. Automatic building segmentation of aerial imagery using multi-constraint fully convolutional networks. Remote Sensing, 10(3): 407 [DOI: 10.3390/rs10030407http://dx.doi.org/10.3390/rs10030407]
Wu W, Luo J C, Shen Z F and Zhu Z W. 2012. Building extraction from high resolution remote sensing imagery based on spatial-spectral method. Geomatics and Information Science of Wuhan University, 37(7): 800-805
吴炜, 骆剑承, 沈占锋, 朱志文. 2012. 光谱和形状特征相结合的高分辨率遥感图像的建筑物提取方法. 武汉大学学报(信息科学版), 37(7): 800-805 [DOI: 10.13203/j.whugis2012.07.022http://dx.doi.org/10.13203/j.whugis2012.07.022]
You Y F, Wang S Y, Wang B, Ma Y X, Shen M, Liu W H and Xiao L. 2019. Study on hierarchical building extraction from high resolution remote sensing imagery. Journal of Remote Sensing, 23(1): 125-136
游永发, 王思远, 王斌, 马元旭, 申明, 刘卫华, 肖琳. 2019. 高分辨率遥感影像建筑物分级提取. 遥感学报, 23(1): 125-136 [DOI: 10.11834/jrs.20197500http://dx.doi.org/10.11834/jrs.20197500]
Zhang C S, Ge Y W and Jiang X. 2020. High-resolution remote sensing image building extraction based on sparsely constrained SegNet. Journal of Xi’an University of Science and Technology, 40(3): 441-448
张春森, 葛英伟, 蒋萧. 2020. 基于稀疏约束SegNet的高分辨率遥感影像建筑物提取. 西安科技大学学报, 40(3): 441-448 [DOT: 10.13800/j.cnki.xakjdxxb.2020.0309http://dx.doi.org/10.13800/j.cnki.xakjdxxb.2020.0309]
Zhang Z X and Wang Y H. 2019. JointNet: a common neural network for road and building extraction. Remote Sensing, 11(6): 696 [DOI: 10.3390/rs11060696http://dx.doi.org/10.3390/rs11060696]
Zhao W Z, Guo Z, Yue J, Zhang X Y and Luo L Q. 2015. On combining multiscale deep learning features for the classification of hyperspectral remote sensing imagery. International Journal of Remote Sensing, 36(13): 3368-3379 [DOI: 10.1080/2150704X.2015.1062157http://dx.doi.org/10.1080/2150704X.2015.1062157]
Zhou D J, Wang G Z, He G J, Long T F, Yin R Y, Zhang Z M, Chen S B and Luo B. 2020. Robust building extraction for high spatial resolution remote sensing images with self-attention network. Sensors, 20(24): 7241 [DOI: 10.3390/s20247241http://dx.doi.org/10.3390/s20247241]
Zhou Z W, Siddiquee M, Tajbakhsh N and Liang J M. 2018. UNet++: a nested U-Net architecture for medical image segmentation//Proceedings of the 8th International Workshop on Deep Learning in Medical Image Analysis. Granada: Springer: 3-11 [DOI: 10.1007/978-3-030-00889-5_1http://dx.doi.org/10.1007/978-3-030-00889-5_1]
相关作者
相关机构
京公网安备11010802024621