Integrating ensemble prediction constraints and error prediction entropy maximization for MLS point cloud classification method

Lei Xiangda; Guan Haiyan; Dong Zhen

doi:10.11834/jrs.20233174

Views : 0 下载量: 161 CSCD: 0 更多指标

R-PDF
PDF
Export
Share
Collection
Album

Integrating ensemble prediction constraints and error prediction entropy maximization for MLS point cloud classification method
Pages: 1-12(2023)
Published Online： 24 October 2023 ，
DOI： 10.11834/jrs.20233174

扫描看全文

雷相达，管海燕，董震.XXXX.整合集成预测约束与错误预测熵最大化的MLS点云分类方法.遥感学报，XX（XX）： 1-12

Lei Xiangda，Guan Haiyan，Dong Zhen. XXXX. Integrating ensemble prediction constraints and error prediction entropy maximization for MLS point cloud classification method. National Remote Sensing Bulletin， XX（XX）：1-12
雷相达，管海燕，董震.XXXX.整合集成预测约束与错误预测熵最大化的MLS点云分类方法.遥感学报，XX（XX）： 1-12 DOI： 10.11834/jrs.20233174.

Lei Xiangda，Guan Haiyan，Dong Zhen. XXXX. Integrating ensemble prediction constraints and error prediction entropy maximization for MLS point cloud classification method. National Remote Sensing Bulletin， XX（XX）：1-12 DOI： 10.11834/jrs.20233174.

摘要

目前，许多深度学习点云分类方法通过增加点云特征聚合模块，增强点云特征的表达能力。但该类方法往往会带来训练参数增加以及模型过拟合的问题。针对该问题，本文提出了一个整合集成预测约束与错误预测熵最大化的深度学习方法用于移动激光扫描（Mobile Laser Scanning， MLS）点云分类。方法通过集成预测约束分支以及错误预测熵最大化分支可以在不增加训练参数的情况下，增强基线网络的点云特征表达，提高模型泛化能力。其中集成预测约束分支首先通过记录点云在训练过程中的预测值，生成集成预测值，然后采用一致性约束增强模型的点云特征表达。错误预测熵最大化方法鼓励模型对错误预测点进行熵值最大化，增加该点的不确定性，提高模型的泛化能力。所提方法在多个公开MLS点云数据集上进行验证，结果表明所提方法可以在不增加训练参数的情况下，提高基线方法的分类性能。与对比方法相比，所提方法在Toronto3D、WHU-MLS、Paris数据集上获得了最优的平均交并比（83.68%、44.19%、65.85%），表明了方法的有效性。

Abstract

Objective Mobile Laser Scanning (MLS) systems are widely used in various fields

owing to their ability for rapidly acquiring high-precision and high-density 3D point cloud data

particularly in the acquisition of urban spatial information. Since urban MLS point clouds exhibit complex scenes

large data volumes and uneven spatial distribution

accurate classification of large-scale urban point clouds presents significant challenges. Currently

many deep learning point cloud classification methods enhance feature representation of point cloud by adding the feature aggregation module. Nonetheless

this approach frequently results in increased training parameters and model over-fitting.Methods We propose an MLS point cloud classification method integrating resemble prediction constraints and error prediction entropy maximization. The proposed method is able to enhance the point cloud feature representation of the baseline network and improve the generalization ability of the model without increasing the training parameters. Our method consists of three main components: a basic supervision branch

an ensemble prediction constraint branch

and an error prediction entropy maximization branch. Specifically

we first employ RandLA-Net as the backbone network to obtain point cloud classification features. Then a basic supervised branch calculates the weighted cross-entropy loss based on the true labels

predicted probability distributions and category weights to provide the basic fully supervised signal for model training. For the ensemble prediction constraint branch

we first generate the ensemble predictions by recording the predicted values during the point cloud training process. Since the input to RandLA-Net is a random sub-point cloud

the ensemble predictions can be integrated not only for predictions at different stages

but also at different relative positions. Thus

the ensemble prediction is more robust to the current prediction

and then we apply the consistency constraint to minimize the difference between the two predictions to improve the point cloud feature representation. Finally

we design the error prediction entropy maximization branch to maximize the entropy of error prediction point sets

increasing their confusion to reduce the model over-fitting.Results and conclusion The public MLS point cloud dataset Toronto3D is chosen as the primary dataset to validate the performance of the proposed method. The qualitative result (Fig. 5) and quantitative result (Table 1) on the Toronto3D dataset show that the proposed method can correctly classify most points. To verify the validity of the method

we compare the accuracy of the proposed method with other popular methods. The comparison results (Table 1 and 2) show that the proposed method obtains the best OA and mIoU

in which the OA and mIoU can achieve 97.71% and 83.68%

respectively. To verify the effectiveness for each branch of the proposed method

a series of ablation experiments were carried out. The results (Table 3 and Fig. 7) show that each branch can effectively improve the model classification performance. The complexity analysis (Table 4) indicate that the proposed method can improve the accuracy of the baseline method without increasing the model parameters. Finally

the experimental results (Table 5) on other public MLS datasets (WHU-MLS and Paris datasets) show that the proposed method can obtain competitive results on multiple datasets.

关键词

MLS点云分类深度学习集成预测约束错误预测熵最大化

Keywords

MLS point cloud classificationdeep learningensemble prediction constraintserror prediction entropy maximization

references

Campbell M, Egerstedt M, How J P, and Murray R M. 2010. Autonomous driving in urban environments: approaches, lessons and challenges. Philosophical transactions. Physical sciences and engineering, 368(1928): 4649-4672. [DOI: 10.1098/rsta.2010.0110http://dx.doi.org/10.1098/rsta.2010.0110]

China Academy of Information and Communications Technology. Digital Twin Cities White Paper, 2020 (中国信息通信研究院. 数字孪生城市白皮书，2020).

Deschaud J, Duque D, Richa J P, Velasco-Forero S, Marcotegui B and Goulette F. 2021. Paris-CARLA-3D: A Real and Synthetic Outdoor Point Cloud Dataset for Challenging Tasks in 3D Mapping. Remote Sensing, 13(22), 4713. [DOI: 10.3390/rs13224713http://dx.doi.org/10.3390/rs13224713]

Du J, Cai G, Wang Z, Huang S, Su J, Marcato Junior J, Smit J and Li J. 2021. ResDLPS-Net: Joint residual-dense optimization for large-scale point cloud semantic segmentation. ISPRS Journal of Photogrammetry and Remote Sensing. 182: 37-51. [DOI: 10.1016/j.isprsjprs.2021.09.024http://dx.doi.org/10.1016/j.isprsjprs.2021.09.024]

Fan S, Dong Q, Zhu F, Lv Y, Ye P and Wang F. 2021. SCF-Net: Learning Spatial Contextual Features for Large-Scale Point Cloud Segmentation//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Kuala Lumpur: IEEE: 14499-14508. [DOI: 10.1109/ CVPR46437.2021.01427http://dx.doi.org/10.1109/CVPR46437.2021.01427]

Gong J Y, Lou Y J, Liu F Q, Zhang Z W, Chen H M, Zhang Z Z, Tan X, Xie Y and Ma L Z. 2023. Scene point cloud understanding and reconstruction technologies in 3D space. Journal of Image and Graphics，28(06): 1741-1766.

龚靖渝，楼雨京，柳奉奇，张志伟，陈豪明，张志忠，谭鑫，谢源，马利庄. 2023. 三维场景点云理解与重建技术. 中国图象图形学报，28(06): 1741-1766 [DOI: 10.11834/jig.230004http://dx.doi.org/10.11834/jig.230004］

Guo Y, Wang H, Hu Q, Liu H, Liu L and Bennamoun M. 2020. Deep Learning for 3D Point Clouds: A Survey. IEEE transactions on pattern analysis and machine intelligence. 43: 4338-4364. [DOI: 10.1109/TPAMI.2020. 3005434http://dx.doi.org/10.1109/TPAMI.2020.3005434]

Hackel T, Wegner J D and Schindler K. 2016. Fast semantic segmentation of 3D point clouds with strongly varying density. ISPRS Annals of Photogrammetry, Remote Sensing & Spatial Information Sciences. 3, 177-184. [DOI: 10.5194/isprsannals-III-3-177-2016http://dx.doi.org/10.5194/isprsannals-III-3-177-2016]

Han X, Dong Z and Yang B. 2021. A point-based deep learning network for semantic segmentation of MLS point clouds. ISPRS Journal of Photogrammetry and Remote Sensing. 175: 199-214. [DOI: 10.1016/j.isprsjprs.2021.03.001http://dx.doi.org/10.1016/j.isprsjprs.2021.03.001]

Hu X, and Yuan Y. 2016. Deep-learning-based classification for DTM extraction from ALS point cloud. Remote sensing. 8(9):730. [DOI: 10.3390/rs8090730http://dx.doi.org/10.3390/rs8090730]

Hu Q, Yang B, Xie L, Rosa S, Guo Y, Wang Z, Trigoni N and Markham A. 2022. Learning Semantic Segmentation of Large-Scale Point Clouds with Random Sampling. IEEE Transactions On Pattern Analysis and Machine Intelligence, 44(11):8338-8354. [DOI: 10.1109/ TPAMI.2021.3083288http://dx.doi.org/10.1109/TPAMI.2021.3083288]

Jing Z W, Guan H Y, Zang Y F, Ni H, Li D L and Yu Y T. 2021. Survey of Point Cloud Semantic Segmentation Based on Deep Learning. Journal of Frontiers of Computer Science and Technology, 15(1): 1-26.

景庄伟,管海燕,臧玉府,倪欢，李迪龙，于永涛. 基于深度学习的点云语义分割研究综述[J].计算机科学与探索, 2021, 15(01): 1-26. [DOI: 10.3778/j.issn.1673-9418.2006025http://dx.doi.org/10.3778/j.issn.1673-9418.2006025]

Larrazabal A, Martinez1 C, Dolz J and Ferrante N. 2021. Maximum entropy on erroneous predictions (MEEP): Improving model calibration for medical image segmentation. In arXiv preprint arXiv: 2112.12218. [DOI: 10.48550/arXiv.2112.12218http://dx.doi.org/10.48550/arXiv.2112.12218]

Lei X D, Guan H Y, Ma L F, Yu Y T, Dong Z, Gao K, Delavar M R and Li J. 2022. WSPointNet: A multi-branch weakly supervised learning network for semantic segmentation of large-scale mobile laser scanning point clouds, International journal of applied earth observation and geoinformation, 115: 103129. [DOI: 10.1016/j.jag.2022.103129http://dx.doi.org/10.1016/j.jag.2022.103129]

Liu K, Gao Z, Lin F and Chen B M. 2023. FG-Net: A Fast and Accurate Framework for Large-Scale LiDAR Point Cloud Understanding. IEEE transactions on cybernetics, 53(1):553-564. [DOI: 10.1109/TCYB.2022.3159815http://dx.doi.org/10.1109/TCYB.2022.3159815]

Liu R F, Ma X J, Lu X S, Wang M Y and Wang P. 2022. Automatic extraction of urban road boundaries using diverse LBP features. National Remote Sensing Bulletin, 26(3): 541-554.

刘如飞，马新江，卢秀山，王旻烨，王鹏. 利用多元LBP特征自动提取城市道路边界.遥感学报，2022, 26 (3): 541-554 [DOI: 10.11834/jrs.20209228http://dx.doi.org/10.11834/jrs.20209228]

Le X H, Wang C, Hao L M and Li J J. 2022. Application of Vehicle-mounted LiDAR Mobile Measurement System in Urban Parts Collection.Geomatics & Spatial Information Technology, 45(02):90-92+96.

靳兴浩,王超,郝利民,李军杰. 车载LiDAR移动测量系统在城市部件采集的应用[J].测绘与空间地理信息,2022,45(02):90-92+96.

Luo H, Khoshelham K, Fang L and Chen C. 2020. Unsupervised scene adaptation for semantic segmentation of urban mobile laser scanning point clouds. ISPRS Journal of Photogrammetry and Remote Sensing. 169: 253-267. [10.1016/j.isprsjprs.2020.10.002]

Qi C R, Su H, Mo K and Guibas L J. 2017a. Pointnet: Deep learning on point sets for 3d classification and segmentation//Proceedings of 2017 IEEE/CVF Conference on Computer Vision and Pattern Recognition, honolulu: IEEE: 652-660. [DOI: 10.1109/CVPR.2017.16http://dx.doi.org/10.1109/CVPR.2017.16]

Qi C R, Li Y, Su H and Guibas L J. 2017b. PointNet++: deep hierarchical feature learning on point sets in a metric space// Proceedings of 2017 Conference on Neural Information Processing Systems, Long Beach: 30:5099-5108.

Qin N, Hu X, Wang P and Li Y. 2020. Semantic Labeling of ALS Point Cloud via Learning Voxel and Pixel Representations. IEEE Geoscience and Remote Sensing Letters, 17(5): 859-863. [DOI: 10.1109/LGRS.2019.2931119http://dx.doi.org/10.1109/LGRS.2019.2931119]

Rafael M, Simon K and Geoffrey H. 2021. When Does Label Smoothing Help?. In arXiv preprint arXiv:1906.02629. [DOI: 10.48550/arXiv.1906.02629http://dx.doi.org/10.48550/arXiv.1906.02629]

Rim B, Lee A and Hong M. 2021. Semantic Segmentation of Large-Scale Outdoor Point Clouds by Encoder-Decoder Shared MLPs with Multiple Losses. Remote Sensing. 13: 3121. [DOI: 10.3390/rs13163121http://dx.doi.org/10.3390/rs13163121]

Shuai H, Xu X and Liu Q. 2021. Backward Attentive Fusing Network With Local Aggregation Classifier for 3D Point Cloud Semantic Segmentation. IEEE Transactions On Image Processing. 30: 4973-4984. [DOI: 10.1109/TIP.2021.3073660http://dx.doi.org/10.1109/TIP.2021.3073660]

Tan B, Zhong R F and Li Q. 2012. Objects classification with vehicle-borne laser scanning data. Journal of Remote Sensing, 16(1):50-66.

谭贲,钟若飞,李芹.车载激光扫描数据的地物分类方法[J].遥感学报,2012,16(01):50-66 [DOI: 10.11834/jrs.20120408http://dx.doi.org/10.11834/jrs.20120408]

Tan W, Qin N, Ma L, Li Y, Du J, Cai G, Yang K and Li J. 2020. Toronto-3D: A large-scale mobile lidar dataset for semantic segmentation of urban roadways//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 797-806. [DOI: 10.1109/CVPRW50498.2020.00109http://dx.doi.org/10.1109/CVPRW50498.2020.00109]

Tan X, Xu J, Ye Z, Song H, Qu Y, Xie Y and Ma L. 2021. Omni-Supervised Point Cloud Segmentation via Gradual Receptive Field Component Reasoning//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Kuala Lumpur: IEEE: 11673-11682. [DOI: 10.1109/CVPR46437.2021.01150http://dx.doi.org/10.1109/CVPR46437.2021.01150]

Thomas H, Qi C R, Deschaud J E, Marcotegui B and Guibas L J. 2019. KPConv: Flexible and deformable convolution for point clouds//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision, Seoul: IEEE: 6411-6420. [DOI: 10.1109/ICCV.2019.00651http://dx.doi.org/10.1109/ICCV.2019.00651]

Wang Y, Sun Y, Liu Z, Sarma S E, Bronstein M M and Solomon J M. 2019. Dynamic graph CNN for learning on point clouds. ACM Transactions on Graphics. 38(5):146. [DOI: 10.1145/3326362http://dx.doi.org/10.1145/3326362]

Weinmann M, Jutzi B, Hinz S and Mallet C. 2015. Semantic point cloud interpretation based on optimal neighborhoods, relevant features and efficient classifiers. ISPRS Journal of Photogrammetry and Remote Sensing, 105:286-304. [DOI: 10.1016/j.isprsjprs.2015.01.016http://dx.doi.org/10.1016/j.isprsjprs.2015.01.016]

Xu S, Vosselman G, and Oude Elberink S. 2015. Detection and classification of changes in buildings from airborne laser scanning data. Remote sensing, 7(12): 17051-76. [DOI: 10.3390/rs71215867http://dx.doi.org/10.3390/rs71215867]

Yan K, Hu Q, Wang H, Huang X, Li L and Ji S. 2022. Continuous Mapping Convolution for Large-Scale Point Clouds Semantic Segmentation. IEEE Geoscience and Remote Sensing Letters, 19:1-5. [DOI: 10.1109/LGRS.2021.3107006http://dx.doi.org/10.1109/LGRS.2021.3107006]

Yang B S, Han X and Dong Z. 2021. Point cloud benchmark dataset WHU-TLS and WHU-MLS for deep learning. National Remote Sensing Bulletin，25(1) : 231-240.

杨必胜,韩旭,董震.点云深度学习基准数据集. 遥感学报, 2021, 25(01), 231-240. ［DOI：10.11834/jrs.20210542http://dx.doi.org/10.11834/jrs.20210542］

Yang B S, Liang F X and Huang R G. 2017. Progress, Progress, Challenges and Perspectives of 3D LiDAR Point Cloud Processing. Acta Geodaetica et Cartographica Sinica, 46(10):1509-1516.

杨必胜,梁福逊,黄荣刚.三维激光扫描点云数据处理研究进展、挑战与趋势[J].测绘学报,2017,46(10):1509-1516. [DOI: 10.11947/j.AGCS.2017.20170351http://dx.doi.org/10.11947/j.AGCS.2017.20170351]

Zhang W and Xiao C. 2019. PCAN: 3D attention map learning using contextual information for point cloud based retrieval//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach: IEEE: 12428-12437. [DOI: 10.1109/CVPR.2019.01272http://dx.doi.org/10.1109/CVPR.2019.01272]

Alert me when the article has been cited

提交

SAR ship detection through generative knowledge transfer

MtSCCD： Land-use scene classification and change-detection dataset for deep learning

Optical-signal token guided change detection network for heterogeneous remote sensing image

Joint classification of hyperspectral and LiDAR data based on inter-modality match learning

Hyperspectral unmixing network considering spectral information and superpixel segmentation