Multiscale Fusion Crowd Counting Algorithm Based on Attention Mechanism

doi:10.19678/j.issn.1000-3428.0066951

Abstract

Abstract:

A multiscale fusion crowd counting algorithm based on attention mechanism is proposed to addresses the issues of large head scale changes and high background noise in crowd counting images, fully aggregating multiscale information to effectively distinguish background noise. Atrous spatial pyramid pooling based on residual connection method is constructed to capture multiscale head target features while incorporating spatial details from shallow feature maps through residual structures and multiple dilated convolutions with different expansion rates, thereby improving the quality of feature maps. A cross-layer multiscale feature fusion module is built to integrate edge details and contextual semantic information of different sizes of shallow and deep branches. In addition, a feature fusion module based on multi-branch is designed to integrate multiscale information of different receptive field sizes, thereby alleviating the problem of large-scale head scale changes. A channel and spatial attention mechanism module is further constructed based on the matrix similarity operation to extract pixel level feature weights, enhance the network's discriminative ability for background and head targets, and adaptively correct position information. The experimental results show that compared to the optimal values of the 11 comparison algorithms, the proposed algorithm reduces the Mean Absolute Error(MAE)and Root Mean Square Error(RMSE)indicators by 1.4% and 4.2% on the SHA dataset, and reduced by 4.9% and 1.8% on the UCF_CC_50 dataset, the proposed algorithm can accurately predict the distribution status, estimate the number of people, and generate high-quality population density maps.

Key words: crowd counting, multiscale fusion, attention mechanism, Convolutional Neural Network(CNN), density map

摘要：

针对人群计数图像人头尺度变化大、背景噪声高等问题，提出一种基于注意力机制的多尺度融合人群计数算法，以充分聚合多尺度信息，并有效区分背景噪声。构建基于残差连接的空洞空间金字塔池化，通过残差结构以及多个不同扩张率的空洞卷积在捕获多尺度头部目标特征的同时融入浅层特征图的空间细节信息，提高特征图质量；构建跨层多尺度特征融合模块，融合浅层和深层分支不同大小的边缘细节信息和上下文语义信息，并设计基于多分支的特征融合模块，融合不同感受野大小的多尺度信息以缓解大规模人头尺度变化的问题；构建基于矩阵相似运算的通道和空间注意力机制模块提取像素级特征权重，加强网络对于背景和人头目标的判别能力，自适应矫正位置信息。实验结果表明，相比11种对比算法的最优值，所提算法在SHA数据集上的平均绝对误差和均方根误差指标降低1.4%、4.2%，在UCF_CC_50数据集上降低4.9%、1.8%，能够精确地预测人群分布状态和估计人群数量，生成高质量的人群密度图。

关键词: 人群计数, 多尺度融合, 注意力机制, 卷积神经网络, 密度图

Xinlin XIE, Dongxu YIN, Taoyuan ZHANG, Gang XIE. Multiscale Fusion Crowd Counting Algorithm Based on Attention Mechanism[J]. Computer Engineering, 2024, 50(3): 290-297.

谢新林, 尹东旭, 张涛源, 谢刚. 基于注意力机制的多尺度融合人群计数算法[J]. 计算机工程, 2024, 50(3): 290-297.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0066951

https://www.ecice06.com/EN/Y2024/V50/I3/290

Figures/Tables 9

Fig.1 Structure of atrous spatial pyramid pooling based on residual connection

Fig.2 Feature fusion module structure based on multi-branch

Fig.3 Attention mechanism module structure based on channel and spatia

Fig.4 Overall structure of multi-scale fusion network based on attention mechanism

Fig.5 Density map visualization results among different comparison algorithms

References 25

1	LIN Z, LARRY S D. Shape-based human detection and segmentation via hierarchical part-template matching. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(4): 604- 618. doi: 10.1109/TPAMI.2009.204
2	QIN X H, WANG X F, ZHOU X, et al. Counting people in various crowed density scenes using support vector regression. Journal of Image and Graphics, 2013, 18(4): 392- 398.
3	SINDAGI V A, PATEL V M. Multi-level bottom-top and top-bottom feature fusion for crowd counting[C]//Proceedings of International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2019: 1002-1012.
4	LIU L B, QIU Z L, LI G B, et al. Crowd counting with deep structured scale integration network[C]//Proceedings of International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2019: 1774-1783.
5	黄奕秋, 胡晓, 杨佳信, 等. 基于背景抑制与上下文感知的人群计数网络. 计算机工程, 2022, 48(9): 314- 320. URL
	HUANG Y Q, HU X, YANG J X, et al. Crowd counting network based on background suppression and context awareness. Computer Engineering, 2022, 48(9): 314- 320. URL
6	BABU S D, SURYA S, VENKATESH B R. Switching convolutional neural network for crowd counting[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 5744-5752.
7	ZHANG Y Y, ZHOU D S, CHEN S Q, et al. Single-image crowd counting via multi-column convolutional neural network[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2016: 589-597.
8	余鹰, 李剑飞, 钱进, 等. 基于多尺度特征融合的抗背景干扰人群计数网络. 模式识别与人工智能, 2022, 35(10): 915- 927. URL
	YU Y, LI J F, QIAN J, et al. Anti-background interference crowd counting network based on multi-scale feature fusion. Pattern Recognition and Aritifical Intelligence, 2022, 35(10): 915- 927. URL
9	万洪林, 王晓敏, 彭振伟, 等. 基于新型多尺度注意力机制的密集人群计数算法. 电子与信息学报, 2022, 44(3): 1129- 1136. URL
	WAN H L, WANG X M, PENG Z W, et al. Dense crowd counting algorithm based on novel multi-scale attention mechanism. Journal of Electronics & Information Technology, 2022, 44(3): 1129- 1136. URL
10	SINDAGI V A, PATEL V M. Generating high-quality crowd density maps using contextual pyramid cnns[C]//Proceedings of International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2017: 1861-1870.
11	严芳芳, 吴秦. 多通道融合分组卷积神经网络的人群计数算法. 小型微型计算机系统, 2020, 41(10): 2200- 2205. doi: 10.3969/j.issn.1000-1220.2020.10.029
	YAN F F, WU Q. Crowd counting algorithm based on multi-channel fusion group convolutional neural network. Journal of Chinese Computer Systems, 2020, 41(10): 2200- 2205. doi: 10.3969/j.issn.1000-1220.2020.10.029
12	LI Y H, ZHANG F, CHEN D. CSRNet: dilated convolutional neural networks for understanding the highly congested scenes[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 1091-1100.
13	CAO X K, WANG Z P, ZHAO Y Y, et al. Scale aggregation network for accurate and efficient crowd counting[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 734-750.
14	ZHANG A R, SHEN J Y, XIAO Z H, et al. Relational attention network for crowd counting[C]//Proceedings of International Conference on Computer Vision. Washington D. C., USA: IEEE Press, 2019: 6788-6797.
15	吴奇元, 王晓东, 章联军, 等. 融合注意力机制与上下文密度图的人群计数网络. 计算机工程, 2022, 48(5): 235- 241. URL
	WU Q Y, WANG X D, ZHANG L J, et al. Crowd counting network with attention mechanism and contextual density map. Computer Engineering, 2022, 48(5): 235- 241. URL
16	GAO J Y, WANG Q, YUAN Y. SCAR: spatial-/channel-wise attention regression networks for crowd counting. Neurocomputing, 2019, 363, 1- 8. doi: 10.1016/j.neucom.2019.08.018
17	GAO G S, LIU Q J, WANG Y H. Counting dense objects in remote sensing images[C]//Proceedings of International Conference on Acoustics, Speech and Signal Processing. Washington D. C., USA: IEEE Press, 2020: 4137-4141.
18	郭爱心, 夏殷锋, 王大为, 等. 一种抗背景干扰的多尺度人群计数算法. 计算机工程, 2022, 48(5): 251- 257. URL
	GUO A X, XIA Y F, WANG D W, et al. A multi-scale crowd counting algorithm with removing background interference. Computer Engineering, 2022, 48(5): 251- 257. URL
19	THANASUTIVES P, FUKUI K, NUMAO M, et al. Encoder-decoder based convolutional neural networks with multi-scale-aware modules for crowd counting[C]//Proceedings of the 25th International Conference on Pattern Recognition. Washington D. C., USA: IEEE Press, 2021: 2382-2389.
20	张宇倩, 李国辉, 雷军, 等. FF-CAM: 基于通道注意机制前后端融合的人群计数. 计算机学报, 2021, 44(2): 304- 317. URL
	ZHANG Y Q, LI G H, LEI J, et al. FF-CAM: crowd counting based on front-end and back-end fusion of channel attention mechanism. Chinese Journal of Computers, 2021, 44(2): 304- 317. URL
21	WANG F S, SANG J, WU Z Y, et al. Hybrid attention network based on progressive embedding scale-context for crowd counting. Information Sciences, 2022, 591, 306- 318. doi: 10.1016/j.ins.2022.01.046
22	IDREES H, SALEEMI I, SEIBERT C, et al. Multi-source multi-scale counting in extremely dense crowd images[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2013: 2547-2554.
23	JIANG X L, XIAO Z H, ZHANG B C, et al. Crowd counting and density estimation by trellis encoder-decoder networks[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 6133-6142.
24	LIU N, LONG Y C, ZOU C Q, et al. ADCrowdNet: an attention-injective deformable convolutional network for crowd understanding[C]//Proceedings of Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2019: 3225-3234.
25	QIAN W, TOBY P B. Crowd counting via segmentation guided attention networks and curriculum loss. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(9): 15233- 15243. doi: 10.1109/TITS.2021.3138896

[1]	LI Junjun, DONG Jiangang, LI Kun. Research on Kubernetes-based Cluster Energy-Saving Strategy [J]. Computer Engineering, 2024, 50(9): 82-91.
[2]	LIN Chang, GUO Wei, REN Zhecong, JIN Haibo. Unification Algorithm for Object Tracking and Segmentation Based on Transformer [J]. Computer Engineering, 2024, 50(9): 130-141.
[3]	LI Zelin, LÜ Zhaofeng, CHEN Fuqiang, LI Ke. Entity Alignment Model Based on Multi-Hop Information Fusion [J]. Computer Engineering, 2024, 50(9): 142-152.
[4]	WANG Ruying, MA Jiajun, DONG Jianqiang, LIU Wanlong, ZHANG Haitao, YIN Kai, ZHAO Bochao. Industrial Load Forecasting Method Based on MTS-BiGRU-DMHSA [J]. Computer Engineering, 2024, 50(9): 169-178.
[5]	ZHANG Lu, TIAN Chunwei, SONG Huansheng, LIU Shigang. Multi-Level Dual-Tree Complex Wavelet Network for Low-Dose CT Image Denoising [J]. Computer Engineering, 2024, 50(9): 266-275.
[6]	ZHU Kai, LI Li, ZHANG Tong, JIANG Sheng, BIE Yiming. Multi-Stage Motion Blur Image Restoration Network Based on Transformer [J]. Computer Engineering, 2024, 50(9): 276-285.
[7]	ZHANG Tianpeng, HAN Jing, LÜ Xueqiang. Super-Resolution-Aided Small-Target Detection Based on Multi-Task Learning [J]. Computer Engineering, 2024, 50(9): 304-312.
[8]	GUO Min, ZHANG Xihan, LI Yang. Integrated Attentional Teacher Mutual Consistency Semi-Supervised Medical Image Segmentation [J]. Computer Engineering, 2024, 50(9): 313-323.
[9]	GAO Yubao, WEN Zhicheng. Dual Decoder Image Denoising Method Based on Attention Mechanism [J]. Computer Engineering, 2024, 50(9): 324-332.
[10]	ZENG Yuqi, LIU Bo, ZHONG Baichang, ZHONG Jin. Student Classroom Behavior Detection Algorithm Based on Improved YOLOv8 in Smart Education [J]. Computer Engineering, 2024, 50(9): 344-355.
[11]	WANG Zhihao, QIAN Yuntao. Super-Resolution Reconstruction of Spatiotemporal Fusion for Dual-Stream Remote Sensing Images Based on Swin Transformer [J]. Computer Engineering, 2024, 50(9): 33-45.
[12]	Suzhe WANG, Xueying ZHANG, Xiaoyu CHEN, Fenglian LI, Zeling WU. EEG Enhancement Algorithm Based on Combination of Effective Attention and GAN [J]. Computer Engineering, 2024, 50(8): 336-344.
[13]	Yu WANG, Qi QI, Chun WANG, Cai XU. High-Precision Fault Diagnosis Method for Energy Storage Inverter Signals [J]. Computer Engineering, 2024, 50(8): 389-396.
[14]	Rixin RAO, Yiwen WANG, Lizhi ZENG, Xintian TONG, Haitao ZHAO. Lightweight Network Model for Waste Cable Detection [J]. Computer Engineering, 2024, 50(8): 22-30.
[15]	Huayu LI, Zhikang ZHANG, Yang YAN, Yang YUE. Enhanced Domain Multi-modal Entity Recognition Based on Knowledge Graph [J]. Computer Engineering, 2024, 50(8): 31-39.

Please choose a citation manager

Content to export