基于BiLSTM-CRF的商情实体识别模型

doi:10.19678/j.issn.1000-3428.0052810

计算机工程 ›› 2019, Vol. 45 ›› Issue (5): 308-314. doi: 10.19678/j.issn.1000-3428.0052810

基于BiLSTM-CRF的商情实体识别模型

张应成¹,杨洋²,蒋瑞^3,4,全兵⁵,张利君³,任晓雷⁶

1.四川大学计算机学院,成都 610065; 2.四川省计算机研究院,成都 610041; 3.成都瑞贝英特信息技术有限公司,成都 610041; 4.四川智仟科技有限公司,成都 610041; 5.中移(苏州)软件技术有限公司,江苏苏州 215000; 6.四川黑马数码科技有限公司,四川泸州 646000

收稿日期:2018-10-08 出版日期:2019-05-15 发布日期:2019-05-15
作者简介:张应成(1994—),男,硕士研究生,主研方向为自然语言处理、人工智能;杨洋、蒋瑞、全兵、张利君,工程师、硕士;任晓雷,工程师。
基金资助:
四川省科技计划项目(18PTDJ0085,2019YFH0075,2018GZDZX0030);泸州市科技计划项目(2017CDLZ-G25)。

Commercial intelligence entity recognition model based on BiLSTM-CRF

ZHANG Yingcheng¹,YANG Yang²,JIANG Rui^3,4,QUAN Bing⁵,ZHANG Lijun³,REN Xiaolei⁶

1.College of Computer Science,Sichuan University,Chengdu 610065,China; 2.Sichuan Institute of Computer Sciences,Chengdu 610041,China; 3.Chengdu Ruibeiyingte Information Technology Co.,Ltd.,Chengdu 610041,China; 4.Sichuan Zhiqian Science and Technology Co.,Ltd.,Chengdu 610041,China; 5.China Mobile(Suzhou) Software Technolgy Co.,Ltd.,Suzhou,Jiangsu 215000,China; 6.Sichuan Heima Digital Technology Co.,Ltd.,Luzhou,Sichuan 646000,China

Received:2018-10-08 Online:2019-05-15 Published:2019-05-15

摘要/Abstract

摘要：

结合语言模型条件随机场(CRF)和双向长短时记忆(BiLSTM)网络,构建一种BiLSTM-CRF模型,以提取商情文本序列中的招标人、招标代理以及招标编号3类实体信息。将规范化后的招标文本序列按字进行向量化,利用BiLSTM神经网络获取序列化文本的前向、后向文本特征,并通过CRF提取出双向本文特征中相应的实体。实验结果表明,与传统机器学习算法CRF相比,该模型3类实体的精确率、召回率和F1值平均提升15.21%、12.06%和13.70%。

关键词: 条件随机场, 双向长短时记忆网络, 语言模型, 命名实体识别, 深度学习

Abstract:

A BiLSTM-CRF model is constructed by combining the Conditional Random Field(CRF) model of Bidirectional Long Short-Term Memory(BiLSTM) network to extract three kinds of entity information,tenderer,bidding agent and bidding number,in a commercial text sequence.The normalized bidding text sequence is vectorized by word.The forward and backward text features of the serialized text are obtained by BiLSTM neural network,and the corresponding entities in the two-way text features are extracted by CRF.Experimental results show that compared with the traditional machine learning algorithm CRF,the precision,recall rate and F1 value of the three types of entities in the proposed model are improved by 15.21%,12.06% and 13.70% in average,respectively.

Key words: Conditional Random Field(CRF), Bidirectional Long Short-Term Memory(BiLSTM) network, language model, Named Entity Recognition(NER), deep learning

中图分类号:

TP391

张应成,杨洋,蒋瑞,全兵,张利君,任晓雷. 基于BiLSTM-CRF的商情实体识别模型[J]. 计算机工程, 2019, 45(5): 308-314.

ZHANG Yingcheng,YANG Yang,JIANG Rui,QUAN Bing,ZHANG Lijun,REN Xiaolei. Commercial intelligence entity recognition model based on BiLSTM-CRF[J]. Computer Engineering, 2019, 45(5): 308-314.

http://www.ecice06.com/CN/Y2019/V45/I5/308

参考文献

［1］ZHANG Lei,ZHANG Yi.Big data analysis by infinite deep neural networks［J］.Journal of Computer Research and Development,2016,53(1):68-79.
［2］BOSCO A,LAGAN D,MUSMANNO R,et al.Modeling and solving the mixed capacitated general routing problem［J］.Optimization Letters,2013,7(7):1451-1469.
［3］MIKOLOV T,SUTSKEVER I,CHEN K,et al.Distri-buted representations of words and phrases and their compositionality［J］.Advances in Neural Information Processing Systems,2013,26:3111-3119.
［4］CHIU J P C,NICHOLS E.Namedentity recognition with bidirectional LSTM-CNNs［EB/OL］.［2018-09-30］.https://arxiv.org/pdf/1511.08308.pdf.
［5］ZHANG Suxiang,WANG Xiaojie.Automatic recognition of Chinese organization name based on conditional random fields［C］//Proceedings of International Conference on Natural Language Processing and Knowledge Engineering.Washington D.C.,USA:IEEE Press,2007:229-233.
［6］BORTHWICK A E.A maximum entropy approach to named entity recognition［D］.New York,USA:New York University,1999.
［7］BIKEL D M,MILLER S,SCHWARTZ R,et al.Nymble:a high-performance learning name-finder［C］//Proceedings of the 15th Conference on Applied Natural Language Processing.Washington D.C.,USA:IEEE Press,1997:194-201.
［8］ASAHARA M,MATSUMOTO Y.Japanese named entity extraction with redundant morphological analysis［C］//Proceedings of NAACL’03.Stroudsburg,USA:Association for Computational Linguistics,2003:8-15.
［9］MCCALLUM A,LI Wei.Early results for named entity recognition with conditional random fields,feature induction and Web-enhanced lexicons［C］//Proceedings of CONLL’03.Stroudsburg,USA:Association for Computational Linguistics,2003:188-191.
［10］CHO K,VAN MERRIENBOER B,GULCEHRE C,et al.Learning phrase representations using RNN encoder-decoder for statistical machine translation［EB/OL］.［2018-09-30］.http://anthology.aclweb.org/D/D14/D14-1179.pdf.
［11］SANTOS C N D,GATTIT M.Deep convolutional neural networks for sentiment analysis of short texts［EB/OL］.［2018-09-30］.http://www.aclweb.org/anthology/C14-1008.
［12］LI Jiwei,GALLEY M,BROCKETT C,et al.A diversity-promoting objective function for neural conversation models［EB/OL］.［2018-09-25］.https://arxiv.org/pdf/1510.03055.pdf.
［13］KARJALA T W,HIMMELBLAU D M,MIIKKULAINEN R.Data rectification using recurrent (Elman) neural networks［C］//Proceedings of International Joint Confe-rence on Neural Networks.Washington D.C.,USA:IEEE Press,1992:901-906.
［14］GRAVES A.Long short-term memory［M］//GRAVES A.Supervised sequence labelling with recurrent neural networks.Berlin,Germany:Springer,2012:1735-1780.
［15］ZHOU Guobing,WU Jianxin,ZHANG Chenlin,et al.Minimal gated unit for recurrent neural networks［J］.International Journal of Automation and Computing,2016,13(3):226-234.
［16］MIKOLOV T,CHEN Kai,CORRADO G,et al.Efficient estimation of word representations in vector space［EB/OL］.［2018-09-10］.http://export.arxiv.org/pdf/1301.3781.
［17］BENGIO Y,SCHWENK H,SENCAL J S,et al.Neural probabilistic language models［J］.Journal of Machine Learning Research,2001,3(6):1137-1155.
［18］MIKOLOV T,KARAFIT M,BURGET L,et al.Recurrent neural network based language model［EB/OL］.［2018-09-25］.http://www.fit.vutbr.cz/research/groups/speech/publi/2010/mikolov_interspeech2010_IS100722.pdf.
［19］MIKOLOV T,ZWEIG G.Context dependent recurrent neural network language model［C］//Proceedings of 2012 IEEE Spoken Language Technology Workshop.Washington D.C.,USA:IEEE Press,2012:234-239.
［20］MIKOLOV T,DEORAS A,POVEY D,et al.Strategies for training large scale neural network language models［C］//Proceedings of 2011 IEEE Workshop on Automatic Speech Recognition and Understanding.Washington D.C.,USA:IEEE Press,2011:196-201.
［21］GRAVES A,SCHMIDHUBER J.Framewise phoneme classification with bidirectional LSTM and other neural network architectures［J］.Neural Networks,2005,18(5):602-610.
［22］HOCHREITER S,SCHMIDHUBER J.Long short-term memory［J］.Neural Computation,1997,9(8):1735-1780.
［23］GRAVES A,JAITLY N,MOHAMED A R .Hybrid speech recognition with deep bidirectional LSTM［C］//Proceedings of 2013 IEEE Workshop on Automatic Speech Recognition and Understanding.Washington D.C.,USA:IEEE Press,2013:273-278.
［24］BAHDANAU D,CHO K,BENGIO Y.Neural machine translation by jointly learning to align and translate［EB/OL］.［2018-09-15］.https://arxiv.org/pdf/1409.0473.pdf.
［25］RATNAPARKHI A.A maximum entropy model for part-of-speech tagging［C］//Proceedings of Conference on Empirical Methods in Natural Language Processing.Washington D.C.,USA:IEEE Press,1996:133-142.
［26］MCCALLUM A,FREITAG D,PEREIRA F C N.Maximum entropy Markov models for information extraction and segmentation［C］//Proceedings of the 17th International Conference on Machine Learning.Washington D.C.,USA:IEEE Press,2000:591-598.
［27］LAFFERTY J D,MCCALLUM A,PEREIRA F C N.Conditional random fields:probabilistic models for segmenting and labeling sequence data［C］//Proceedings of the 18th International Conference on Machine Learning.［S.l.］:Morgan Kaufmann Publishers Inc.,2001:282-289.

[1]	江雨燕, 陶承凤, 李平. 数据增强和自适应自步学习的深度子空间聚类算法[J]. 计算机工程, 2023, 49(8): 96-103, 110.
[2]	李泽水, 冀俊忠, 杨翠翠. 基于边权重信息深度网络嵌入的PPIN功能模块检测[J]. 计算机工程, 2023, 49(8): 69-76.
[3]	杨长沛, 廖列法. 基于门控空洞卷积特征融合的中文命名实体识别[J]. 计算机工程, 2023, 49(8): 85-95.
[4]	王可铮, 徐玉芬, 周尚波. 结合对比感知损失和融合注意力的图像去雾模型[J]. 计算机工程, 2023, 49(8): 207-214.
[5]	刘俊豪, 王美林, 谢兴, 宋烨兴, 许莉花. 基于改进YOLOv5的皮革瑕疵检测算法[J]. 计算机工程, 2023, 49(8): 240-249.
[6]	张家熔, 苑津莎, 许珈宁, 罗志宏. 基于多元信息嵌入与协同神经网络的力学实体识别算法[J]. 计算机工程, 2023, 49(7): 125-134.
[7]	闫兴亚, 匡娅茜, 白光睿, 李月. 基于深度学习的学生课堂行为识别方法[J]. 计算机工程, 2023, 49(7): 251-258.
[8]	李军侠, 王星驰, 殷梓, 石德硕. 边缘深度挖掘的弱监督显著性目标检测[J]. 计算机工程, 2023, 49(7): 169-178.
[9]	吴珊, 周凤. 基于改进SSD算法的小目标检测[J]. 计算机工程, 2023, 49(7): 179-188.
[10]	席建锐, 唐红梅, 梁春阳, 刘鑫. 基于改进隐函数的点云物体重建[J]. 计算机工程, 2023, 49(7): 214-222.
[11]	齐咏生, 杜晓旭, 朱俊峰, 高胜利, 刘利强. 基于增强型轻量深度网络的牧区牲畜高效检测[J]. 计算机工程, 2023, 49(7): 278-287.
[12]	谌雨章, 黄逸姿, 张钧涵. 基于多速率空洞卷积的多尺度水下小目标检测[J]. 计算机工程, 2023, 49(6): 257-264.
[13]	陈明, 刘蓉, 张晔. 基于多重注意力机制的中文医疗实体识别[J]. 计算机工程, 2023, 49(6): 314-320.
[14]	张博旭, 蒲智, 程曦. 基于提示学习的维吾尔语文本分类研究[J]. 计算机工程, 2023, 49(6): 292-299,313.
[15]	于海洋, 景鹏, 张文涛, 谢赛飞, 滑志华, 宋草原. 基于残差与注意力机制的道路裂缝检测U-Net改进模型[J]. 计算机工程, 2023, 49(6): 265-273.

选择文件类型/文献管理软件名称

选择包含的内容

基于BiLSTM-CRF的商情实体识别模型

Commercial intelligence entity recognition model based on BiLSTM-CRF

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

基于BiLSTM-CRF的商情实体识别模型

Commercial intelligence entity recognition model based on BiLSTM-CRF

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价