Audio Visual Fusion Speech Recognition Model Based on Articulatory Feature

doi:10.3969/j.issn.1000-3428.2011.22.089

Computer Engineering ›› 2011, Vol. 37 ›› Issue (22): 268-269. doi: 10.3969/j.issn.1000-3428.2011.22.089

• Networks and Communications • Previous Articles Next Articles

Audio Visual Fusion Speech Recognition Model Based on Articulatory Feature

WU Peng ¹, JIANG Dong-mei ^1, WANG Feng-na ^2, Hichem SAHLI ², Werner VERHELST ²

(1. Shaanxi Provincial Key Laboratory on Speech, Image and Information Processing, Northwestern Polytechnical University, Xi’an 710072, China; 2. Department of ETRO, Vrije Universiteit Brussel, Brussels 1050, Belgium)

Received:2011-05-11 Online:2011-11-18 Published:2011-11-20

基于发音特征的音视频融合语音识别模型

吴鹏¹，蒋冬梅 ¹，王风娜 ²，Hichem SAHLI ²，Werner VERHELST ²

(1. 西北工业大学陕西省语音与图像信息处理重点实验室，西安 710072；2. 布鲁塞尔自由大学电子与信息工程系，比利时布鲁塞尔 1050)

作者简介:吴鹏(1984－)，男，硕士研究生，主研方向：语音识别，可视语音合成；蒋冬梅，教授；王风娜，博士；Hichem SAHLI、Werner VERHELST，教授
基金资助:
国家自然科学基金资助项目(60703104)；陕西省自然科学基金资助项目(SJ08F28)；西北工业大学基础研究基金资助项目(JC200943)

Abstract

Abstract: A multi-stream Dynamic Bayesian Network(DBN) model(AF_AV_DBN) based on Articulatory Feature(AF) is proposed for audio visual speech recognition. Conditional probability distribution of each node and the degree of asynchrony between the AFs are defined, and speech recognition experiments are carried out on an audio visual connected digit database. Compared with the audio-only AF_A_DBN model, the state synchronous DBN model and the state asynchronous DBN model, the designed AF_AV_DBN model gets the highest recognition rate under various signal to noise ratios, and is more robust to background noise.

Key words: Dynamic Bayesian Network(DBN), articulatory feature, audio visual fusion, speech recognition, asynchronous

摘要： 构建一种基于发音特征的音视频双流动态贝叶斯网络(DBN)语音识别模型(AF_AV_DBN)，定义节点的条件概率关系，使发音特征状态的变化可以异步。在音视频语音数据库上的语音识别实验表明，通过调整发音特征之间的异步约束，AF_AV_DBN模型能得到比基于状态的同步和异步DBN模型以及音频单流模型更高的识别率，对噪声也具有较好的鲁棒性。

关键词: 动态贝叶斯网络, 发音特征, 音视频融合, 语音识别, 异步

CLC Number:

TP391.42

TUN Feng, JIANG Dong-Mei, WANG Feng-Na, Hichem SAHLI, Werner VERHELST. Audio Visual Fusion Speech Recognition Model Based on Articulatory Feature[J]. Computer Engineering, 2011, 37(22): 268-269.

吴鹏, 蒋冬梅, 王风娜, Hichem SAHLI, Werner VERHELST. 基于发音特征的音视频融合语音识别模型[J]. 计算机工程, 2011, 37(22): 268-269.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.3969/j.issn.1000-3428.2011.22.089

http://www.ecice06.com/EN/Y2011/V37/I22/268

[1]	LI Yiting, QU Dan, YANG Xukui, ZHANG Hao, SHEN Xiaolong. Efficient Conformer Model Based on Factorized Gated Attention Unit [J]. Computer Engineering, 2023, 49(5): 73-80.
[2]	SHI Yuhu, ZHANG Qigui. Method for Fast Video Object Detection Based on Local Attention [J]. Computer Engineering, 2022, 48(5): 314-320.
[3]	BAI Caitong, CUI Xiaolong, LI Ai. Robust Speech Recognition Technology Based on Federal Learning with Local Distillation [J]. Computer Engineering, 2022, 48(10): 103-109.
[4]	PENG Long, CHEN Junshi, AN Hong. Migration and Optimization of AMBER Software Based on Sunway TaihuLight [J]. Computer Engineering, 2020, 46(12): 12-20.
[5]	WANG Junchao,HUANG Hao,XU Haihua,HU Ying. Low-resource Uyghur Speech Recognition Based on Transfer Learning [J]. Computer Engineering, 2018, 44(10): 281-285,291.
[6]	HU Wenjun,FU Meijun,PAN Wenlin. Primi Speech Recognition Based on Kaldi [J]. Computer Engineering, 2018, 44(1): 199-205.
[7]	ZHANG Le,ZHANG Xueying,SUN Ying,ZHANG Wei. Feature Extraction of Emotional Speech Based on Ensemble Empirical Mode Decomposition [J]. Computer Engineering, 2017, 43(8): 306-309,315.
[8]	XIANG Bingwei,JING Xinxing,YANG Haiyan. Vehicular Speech Recognition Based on Noise Classification and Compensation [J]. Computer Engineering, 2017, 43(3): 220-224.
[9]	SHANG Xiongwei,ZHANG Zhixiang,QIU Shuting. A Design Method for General Intelligent Spoken Tutoring System in Limited Domain [J]. Computer Engineering, 2016, 42(6): 299-304.
[10]	ZHAO Caiguang,ZHANG Shuqun,LEI Zhaoyi. Speech Recognition of Gaussian-Bernoulli Restricted Boltzmann Machine Based on Improved Contrastive Divergence [J]. Computer Engineering, 2015, 41(5): 213-218.
[11]	WANG Yong,ZHANG Yanyan. Stability Analysis of Network Control System with Time Delay and Packet Dropout [J]. Computer Engineering, 2015, 41(10): 111-116.
[12]	XIAN Xiaodong,LV Jianzhong,FAN Yuxing. Initial Estimation of CHMM Acoustic Model Based on Density and Distance Parameter [J]. Computer Engineering, 2015, 41(10): 318-321.
[13]	ZHANG Zhen, ZHAO Qing-wei, YAN Yong-hong. Unsupervised Speech Pattern Extraction Based on Speech Recognition and Feature [J]. Computer Engineering, 2014, 40(5): 262-265.
[14]	ZHENG Qian,YAN Min,ZHAO Jian-zhong,LI You,ZHANG Feng. Design of Elastic Buffer at Physical Layer Based on PCIE2． 0 [J]. Computer Engineering, 2014, 40(10): 71-75.
[15]	SONG Yu,JIANG Lie-hui,DONG Wei-yu,CHEN Hui-chao. Research on a Stand-alone I / O Virtualization Method [J]. Computer Engineering, 2014, 40(10): 81-85.

Please choose a citation manager

Content to export

Audio Visual Fusion Speech Recognition Model Based on Articulatory Feature

基于发音特征的音视频融合语音识别模型

PDF

Knowledge

Cited

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

Comments

模态框（Modal）标题

Please choose a citation manager

Content to export

Audio Visual Fusion Speech Recognition Model Based on Articulatory Feature

基于发音特征的音视频融合语音识别模型

PDF

Knowledge

Cited

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

Comments