A Multi-scale Crowd Counting Algorithm with Removing Background Interference

doi:10.19678/j.issn.1000-3428.0061423

Abstract

Abstract: Crowd counting technology is aimed at estimating the number of people in crowd pictures or videos.The technology can effectively be applied to prevent stampede accidents and is widely used in security and early warning, urban planning, and management of large gatherings.However, due to crowd scale variation, background interference, uneven crowd distribution, occlusion, and perspective effect, it is still a very challenging task to count a single image.Aiming at the problem of multi-scale changes and background interference in crowd counting, a multi-scale crowd counting algorithm with removing background interference is proposed.Based on the VGG16 network structure, the feature pyramid is introduced to form the multi-scale feature fusion backbone network to solve the problem of the multi-scale changes.The Double-Head-CC structure is designed to perform foreground-background segmentation and density map prediction on the fused feature map to suppress the background interference.Based on the local correlation of the density map and multi-task learning, the multiple loss functions, and multi-task joint loss function are defined to optimize the network.The network model is trained and evaluated on the ShanghaiTech, UCF-QNRF, and JHU-CROWD++ datasets.Experimental results show that the algorithm can predict the population density distribution and number of the crowd well, with high accuracy, strong robustness, and good generalization performance.

Key words: crowd counting, deep learning, feature pyramid, loss function, density map

摘要： 人群计数技术以估计人群图片或视频中的人数为目标，可以有效预防人群踩踏事故的发生，广泛应用于安防预警、城市规划及大型集会管理等领域。然而，由于人群尺度变化、背景干扰、人群分布不均、遮挡和透视效应等因素的影响，单幅图片的人群计数仍是一项非常具有挑战性的任务。针对人群计数中多尺度变化和背景干扰问题，提出一种抗背景干扰的多尺度人群计数算法。以VGG16网络结构为基础，引入特征金字塔构建多尺度特征融合骨干网络解决人群多尺度变化问题，设计Double-Head-CC结构对融合后的特征图进行前景背景分割和密度图预测以抑制背景干扰。基于密度图的局部相关性和多任务学习，定义多重损失函数和多任务联合损失函数进行网络优化。在ShanghaiTech、UCF-QNRF和JHU-CROWD++数据集上进行训练和评测，实验结果表明，该算法能够很好地预测人群密度分布和人群数量，具有较高的准确性，且鲁棒性强、泛化性能良好。

关键词: 人群计数, 深度学习, 特征金字塔, 损失函数, 密度图

CLC Number:

TP391

GUO Aixin, XIA Yinfeng, WANG Dawei, LU Bin. A Multi-scale Crowd Counting Algorithm with Removing Background Interference[J]. Computer Engineering, 2022, 48(5): 251-257.

郭爱心, 夏殷锋, 王大为, 芦宾. 一种抗背景干扰的多尺度人群计数算法[J]. 计算机工程, 2022, 48(5): 251-257.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0061423

http://www.ecice06.com/EN/Y2022/V48/I5/251

Figures/Tables 11

References

[1] 王陆洋.基于卷积神经网络的图像人群计数研究[D].合肥:中国科学技术大学, 2020. WANG L Y.Image crowd counting based on convolutional neural network[D].Hefei:University of Science and Technology of China, 2020.(in Chinese)
[2] ZHANG Y Y, ZHOU D S, CHEN S Q, et al.Single-image crowd counting via multi-column convolutional neural network[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2016:589-597.
[3] ZHANG A R, SHEN J Y, XIAO Z H, et al.Relational attention network for crowd counting[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2019:6787-6796.
[4] CHENG Z Q, LI J X, DAI Q, et al.Improving the learning of multi-column convolutional neural network for crowd counting[C]//Proceedings of the 27th ACM International Conference on Multimedia.New York, USA:ACM Press, 2019:1897-1906.
[5] GUO D, LI K, ZHA Z J, et al.DADNet:dilated-attention-deformable ConvNet for crowd counting[C]//Proceedings of the 27th ACM International Conference on Multimedia.New York, USA:ACM Press, 2019:1823-1832.
[6] LI Y H, ZHANG X F, CHEN D M.CSRNet:dilated convolutional neural networks for understanding the highly congested scenes[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2018:1091-1100.
[7] 马皓, 殷保群, 彭思凡.基于特征金字塔网络的人群计数算法[J].计算机工程, 2019, 45(7):203-207. MA H, YIN B Q, PENG S F.Crowd counting algorithm based on feature pyramid network[J].Computer Engineering, 2019, 45(7):203-207.(in Chinese)
[8] LIN T Y, DOLLAR P, GIRSHICK R, et al.Feature pyramid networks for object detection[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2017:936-944.
[9] CAO X K, WANG Z P, ZHAO Y Y, et al.Scale aggregation network for accurate and efficient crowd counting[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2018:734-750.
[10] LIU N, LONG Y C, ZOU C Q, et al.ADCrowdNet:an attention-injective deformable convolutional network for crowd understanding[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2019:3220-3229.
[11] SHI Z L, METTES P, SNOEK C.Counting with focus for free[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2019:4199-4208.
[12] TIAN Y K, LEI Y M, ZHANG J P, et al.PaDNet:pan-density crowd counting[J].IEEE Transactions on Image Processing, 2020, 29(5):2714-2727.
[13] SAM D B, SAJJAN N N, MAURYA H, et al.Almost unsupervised learning for dense crowd counting[C]//Proceedings of AAAI Conference on Artificial Intelligence.[S.1.]:AAAI Press, 2019:8868-8875.
[14] BAI S, HE Z Q, QIAO Y, et al.Adaptive dilated network with self-correction supervision for counting[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2020:4593-4602.
[15] IDREES H, TAYYAB M, ATHREY K, et al.Composition loss for counting, density map estimation and localization in dense crowds[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2018:532-546.
[16] SINDAGI V, YASARL R, PATEL V M.JHU-CROWD++:large-scale crowd counting dataset and a benchmark method[EB/OL].[2021-03-20].https://arxiv.org/abs/2004.03597.
[17] SIMONYAN K, ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[EB/OL].[2021-03-20].http://arxiv.org/abs/1409.1556.pdf.
[18] BOCHKOVSKIY A, WANG C Y, LIAO H Y M.YOLOv4:optimal speed and accuracy of object detection[EB/OL].[2021-03-20].https://arxiv.org/pdf/2004.10934.pdf.
[19] WU Y, CHEN Y P, YUAN L, et al.Rethinking classification and localization for object detection[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2020:10186-10195.
[20] DAI F, LIU H, MA Y K, et al.Dense scale network for crowd counting[EB/OL].[2021-03-20].https://arxiv.org/pdf/1906.09707.pdf.
[21] JIANG X L, XIAO Z H, ZHANG B C, et al.Crowd counting and density estimation by trellis encoder-decoder networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2019:6126-6135.
[22] LIU W Z, SALZMANN M, FUA P.Context-aware crowd counting[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2019:5094-5103.
[23] XIONG H P, LU H, LIU C X, et al.From open set to closed set:counting objects by spatial divide-and-conquer[C]//Proceedings of IEEE/CVF International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2019:8361-8370.
[24] WANG Q, GAO J Y, LIN W, et al.Learning from synthetic data for crowd counting in the wild[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2019:8190-8199.
[25] LIU L B, QIU Z L, LI G B, et al.Crowd counting with deep structured scale integration network[C]//Proceedings of IEEE/CVF International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2019:1774-1783.
[26] SINDAGI V A, PATEL V M.Multi-level bottom-top and top-bottom feature fusion for crowd counting[C]//Proceedings of IEEE/CVF International Conference on Computer Vision.Washington D.C., USA:IEEE Press, 2019:1002-1012.

Please choose a citation manager

Content to export