基于LPDMR-NET的鸟鸣声识别

doi:10.19678/j.issn.1000-3428.0067959

摘要/Abstract

摘要：

为了高效且快速地识别自然环境中的鸟鸣声, 提出一种基于轻量级逐点深度的多感受野注意力残差网络(LPDMR-NET)模型。首先, 通过Mel滤波器生成Mel频谱图。接着, 采用basicblock和downblock连接生成两层残差网络DBNet, 堆叠DBNet作为鸟鸣声识别的主干网络, 以提高训练速度。然后, 利用逐点深度卷积网络(PDNet)提取频谱图特征信息, 替代主干网络下采样模块, 将两个残差模块的basicblock中的3×3卷积替换为分离分支块(DBB), 引入不同的感受野, 在复杂多分支结构下显著提高网络的识别性能。最后, 在两个残差模块间嵌入轻量级高效置换注意力(SA)模块用于传递两层残差模块间的有效信息, 增强频谱图波纹特征, 进一步提高网络识别性能。在自建的30类鸟鸣声数据集Birdselfdata上的实验结果表明, 该模型的识别准确率为96.82%、F1值为96.73%, 在识别效率和准确性方面超越了对比模型。

关键词: 卷积神经网络, 鸟鸣声分类, 深度学习, Mel频谱图, 残差网络, 深度可分离卷积

Abstract:

A Lightweight Point-by-point Depth-based Multisensory wild attention Residual NETwork (LPDMR-NET) model is proposed to efficiently and quickly recognize birdsong in natural environments. First, Mel spectrograms are generated using Mel filters. Second, a two-layer residual network, DBNet, is generated using basic-block and down-block connections, and the stacked DBNet is used as the backbone network for birdsong recognition to improve training speed. Subsequently, a Point-by-point Deep convolutional Network (PDNet) is utilized to extract the spectrogram feature information, replace the downsampling module of the backbone network, replace the 3×3 convolution in the basic block of the two residual modules with the Detached Branching Block (DBB), and introduce different sensory fields. These changes significantly improve the recognition performance of the network under the complex multi-branch structure. Finally, a lightweight efficient Substitution Attention(SA) module is embedded between the two residual modules to transfer effective information between the two layers of the residual modules. This addition enhances the spectrogram ripple features and further improves the recognition performance of the network. Experimental results on a self-constructed 30-class bird song dataset, Birdselfdata, show that the model has a recognition accuracy of 96.82% and an F1 value of 96.73%. Thus, the proposed model outperforms the comparison model in terms of recognition efficiency and accuracy.

Key words: Convolutional Neural Network(CNN), birdsong classification, deep learning, Mel spectrogram, Residual Network(ResNet), Depth Separable Convolution(DSC)

王娅茹, 唐璐, 陈爱斌, 彭伟雄, 沈平. 基于LPDMR-NET的鸟鸣声识别[J]. 计算机工程, 2024, 50(10): 174-184.

WANG Yaru, TANG Lu, CHEN Aibin, PENG Weixiong, SHEN Ping. Birdsong Recognition Based on LPDMR-NET[J]. Computer Engineering, 2024, 50(10): 174-184.

https://www.ecice06.com/CN/Y2024/V50/I10/174

图/表 13

图1 LPDMR-NET模型结构

Fig.1 Structure of LPDMR-NET model

图2 Birdselfdata数据集部分类别数量分布

Fig.2 Distribution of the number of partial categories on the Birdselfdata dataset

图3 ResNet18与LPDMR-NET识别性能对比

Fig.3 Comparison of recognition performance between ResNet18 and LPDMR-NET

图4 ResNet18和LPDMR-NET的最佳混淆矩阵

Fig.4 Optimal confusion matrix of ResNet18 and LPDMR-NET

参考文献 30

1	孙戈, 曾立雄, 钱法文, 等. 鸟类监测技术现状与发展趋势. 地理信息世界, 2022, 29 (6): 26- 29. URL
	SUN G, ZENG L X, QIAN F W, et al. Current status and development trend of bird diversity monitoring technology. Geomatics World, 2022, 29 (6): 26- 29. URL
2	GIL-TENA A, SAURA S, BROTONS L. Effects of forest composition and structure on bird species richness in a Mediterranean context: implications for forest ecosystem management. Forest Ecology and Management, 2007, 242 (2/3): 470- 476. doi: 10.1016/j.foreco.2007.01.080
3	SEKERCIOǦLU C H, DAILY G C, EHRLICH P R. Ecosystem consequences of bird declines. Biological Sciences, 2004, 101 (52): 18042- 18047. doi: 10.1073/pnas.0408049101
4	乔玉, 钱昆, 赵子平. 基于机器听觉的鸟声识别的中文研究综述. 复旦学报(自然科学版), 2020, 59 (3): 375- 380. URL
	QIAO Y, QIAN K, ZHAO Z P. A review of Chinese studies on bird sound recognition based on machine hearing. Journal of Fudan (Natural Science Edition), 2020, 59 (3): 375- 380. URL
5	MPORAS I, GANCHEV T, KOCSIS O, et al. Automated acoustic classification of bird species from real-field recordings[C]//Proceedings of the 24th International Conference on Tools with Artificial Intelligence. Washington D. C., USA: IEEE Press, 2012: 778-781.
6	QIAO Y, QIAN K, ZHAO Z P. Learning higher representations from bioacoustics: a sequence-to-sequence deep learning approach for bird sound classification. Berlin, Germany: Springer International Publishing, 2020.
7	屈丹, 杨绪魁, 闫红刚, 等. 低资源少样本连续语音识别最新进展. 郑州大学学报(工学版), 2023, 44 (4): 1- 9. URL
	QU D, YANG X K, YAN H G, et al. Recent advances in continuous speech recognition with low resources and few samples. Journal of Zhengzhou University (Engineering Edition), 2023, 44 (4): 1- 9. URL
8	CHANG P C, CHEN Y S, LEE C H. MS-SincResNet: joint learning of 1D and 2D kernels using multi-scale SincNet and ResNet for music genre classification[C]//Proceedings of the 2021 International Conference on Multimedia Retrieval. New York, USA: ACM Press, 2021: 29-36.
9	PENG N, CHEN A B, ZHOU G X, et al. Environment sound classification based on visual multi-feature fusion and GRU-AWS. IEEE Access, 2020, 8, 191100- 191114. doi: 10.1109/ACCESS.2020.3032226
10	陈莎莎, 李应. 结合时-频纹理特征的随机森林分类器应用于鸟声识别. 计算机应用与软件, 2014, 31 (1): 154-157, 161. doi: 10.3969/j.issn.1000-386x.2014.01.040
	CHEN S S, LI Y. Applying random forest classifier combined with time-frequency texture features to bird sounds recognition. Computer Applications and Software, 2014, 31 (1): 154-157, 161. doi: 10.3969/j.issn.1000-386x.2014.01.040
11	杜孟洋, 王红斌, 普祥和. 融入词性自注意力机制的方面级情感分类方法. 吉林大学学报(理学版), 2023, 61 (6): 1375- 1386. URL
	DU M Y, WANG H B, PU X H. Aspect-level sentiment classification method incorporating part-of-speech self-attention mechanism. Journal of Jilin University (Science Edition), 2023, 61 (6): 1375- 1386. URL
12	李怀城, 杨道武, 温治芳, 等. 基于Inception-CSA深度学习模型的鸟鸣分类. 华中农业大学学报, 2023, 42 (3): 97- 104. URL
	LI H C, YANG D W, WEN Z F, et al. Birdsong classification based on Inception-CSA deep learning model. Journal of Huazhong Agricultural University, 2023, 42 (3): 97- 104. URL
13	KOGAN J A, MARGOLIASH D. Automated recognition of bird song elements from continuous recordings using dynamic time warping and hidden Markov models: a comparative study. The Journal of the Acoustical Society of America, 1998, 103 (4): 2185- 2196. doi: 10.1121/1.421364
14	SOMERVUO P, HARMA A, FAGERLUND S. Parametric representations of bird sounds for automatic species recognition. IEEE Transactions on Audio, Speech, and Language Processing, 2006, 14 (6): 2252- 2263. doi: 10.1109/tasl.2006.872624
15	LEE C H, HSU S B, SHIH J L, et al. Continuous birdsong recognition using Gaussian mixture modeling of image shape features. IEEE Transactions on Multimedia, 2013, 15 (2): 454- 464. URL
16	ADAVANNE S, DROSSOS K, ÇAKIR E, et al. Stacked convolutional and recurrent neural networks for bird audio detection[C]//Proceedings of the 25th European Signal Processing Conference (EUSIPCO). Washington D. C., USA: IEEE Press, 2017: 1729-1733.
17	KONG Q Q, XU Y, PLUMBLEY M D. Joint detection and classification convolutional neural network on weakly labelled bird audio detection[C]//Proceedings of the 25th European Signal Processing Conference (EUSIPCO). Washington D. C., USA: IEEE Press, 2017: 1749-1753.
18	XU W T, ZHANG X, YAO L N, et al. A multi-view CNN-based acoustic classification system for automatic animal species identification. Ad Hoc Networks, 2020, 102, 102115. URL
19	NANNI L, MAGUOLO G, PACI M. Data augmentation approaches for improving animal audio classification. Ecological Informatics, 2020, 57, 101084. URL
20	GUPTA G, KSHIRSAGAR M, ZHONG M, et al. Comparing recurrent convolutional neural networks for large scale bird species classification. Scientific Reports, 2021, 11, 17085. URL
21	PERMANA S D H, SAPUTRA G, ARIFITAMA B, et al. Classification of bird sounds as an early warning method of forest fires using Convolutional Neural Network (CNN) algorithm. Journal of King Saud University(Computer and Information Sciences), 2022, 34 (7): 4345- 4357. URL
22	SANCHEZ F J B, HOSSAIN M R, ENGLISH N B, et al. Bioacoustic classification of avian calls from raw sound waveforms with an open-source deep learning architecture. Scientific Reports, 2021, 11, 15733. URL
23	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Washington D. C., USA: IEEE Press, 2016: 770-778.
24	DING X H, ZHANG X Y, HAN J G, et al. Diverse branch block: building a convolution as an inception-like unit[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D. C., USA: IEEE Press, 2021: 10881-10890.
25	ZHANG Q L, YANG Y B. SA-Net: shuffle attention for deep convolutional neural networks[C]//Proceedings of the 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Washington D. C., USA: IEEE Press, 2021: 2235-2239.
26	CHOLLET F. Xception: deep learning with depthwise separable convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Washington D. C., USA: IEEE Press, 2017: 1800-1807.
27	HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2018: 7132-7141.
28	WANG Q L, WU B G, ZHU P F, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D. C., USA: IEEE Press, 2020: 11531-11539.
29	ZHANG X, CHEN A B, ZHOU G X, et al. Spectrogram-frame linear network and continuous frame sequence for bird sound classification. Ecological Informatics, 2019, 54, 101009. URL
30	LIU Z H, CHEN W J, CHEN A B, et al. Birdsong classification based on multi feature channel fusion. Multimedia Tools and Applications, 2022, 81 (11): 15469- 15490. URL

[1]	李俊俊, 董建刚, 李坤. 基于Kubernetes的集群节能策略研究[J]. 计算机工程, 2024, 50(9): 82-91.
[2]	张鲁, 田春伟, 宋焕生, 刘侍刚. 用于低剂量CT图像去噪的多级双树复小波网络[J]. 计算机工程, 2024, 50(9): 266-275.
[3]	朱凯, 李理, 张彤, 江晟, 别一鸣. 基于Transformer的多阶段运动模糊图像修复网络[J]. 计算机工程, 2024, 50(9): 276-285.
[4]	魏嵬, 丁香香, 郭梦星, 杨钊, 刘辉. 文本相似度计算方法综述[J]. 计算机工程, 2024, 50(9): 18-32.
[5]	王志浩, 钱沄涛. 基于Swin Transformer的双流遥感图像时空融合超分辨率重建[J]. 计算机工程, 2024, 50(9): 33-45.
[6]	张天鹏, 韩晶, 吕学强. 基于多任务学习的超分辨率辅助小目标检测[J]. 计算机工程, 2024, 50(9): 304-312.
[7]	高煜宝, 文志诚. 基于注意力机制的双路解码器图像去噪方法[J]. 计算机工程, 2024, 50(9): 324-332.
[8]	高伟, 李帅龙, 茆琳, 王磊, 李颖颖, 韩林. 一种基于TVM的算子生成加速策略[J]. 计算机工程, 2024, 50(8): 353-362.
[9]	王宇, 祁琦, 王纯, 许才. 储能变流器信号高精度故障诊断方法[J]. 计算机工程, 2024, 50(8): 389-396.
[10]	张华青, 夏张涛, 陆晓庆, 童基均. 基于字形特征的血管外科命名实体识别[J]. 计算机工程, 2024, 50(8): 13-21.
[11]	王蕾, 党时鹏, 潘丰. 基于卷积神经网络的隐匿性旁路预测模型[J]. 计算机工程, 2024, 50(8): 40-49.
[12]	张亚洲, 和玉, 戎璐, 王祥凯. 基于上下文知识增强型Transformer网络的抑郁检测[J]. 计算机工程, 2024, 50(8): 75-85.
[13]	肖慈, 徐杨, 张永丹, 冯明文, 黄易仟. 结合注意力和低光增强的夜间语义分割[J]. 计算机工程, 2024, 50(7): 271-281.
[14]	张诗婧, 莫绪涛, 赵行, 董杨林. 基于球面折反射成像和YOLOv7的内螺纹缺陷检测[J]. 计算机工程, 2024, 50(7): 282-292.
[15]	耿丽丽, 牛保宁. 基于通道相似度熵的卷积神经网络裁剪[J]. 计算机工程, 2024, 50(7): 133-143.

选择文件类型/文献管理软件名称

选择包含的内容