基于字形特征的血管外科命名实体识别

doi:10.19678/j.issn.1000-3428.0068508

摘要/Abstract

摘要：

电子病历(EMR)作为医疗信息化建设的核心, 蕴含着众多有价值的医疗实体, 对电子病历进行命名实体识别有助于推进医学研究。为解决血管外科电子病历研究数据匮乏、实体复杂识别困难等问题, 基于某三甲医院血管外科的真实临床数据, 构建一个小规模的专科数据集作为实验数据集, 并提出一种基于字形特征的命名实体识别模型。首先, 采用掩码校正的来自Transformer的双向编码器表示(MacBERT)生成动态字向量, 引入汉字四角码与汉字五笔两个维度的字形信息; 然后, 将文本表示传入双向门控循环单元(BiGRU)与门控空洞卷积神经网络(DGCNN)进行特征提取, 并对输出结果进行拼接; 最后, 通过多头自注意力机制捕捉序列内部元素间的关系, 利用条件随机场(CRF)进行标签解码。实验结果表明, 所提模型在自建血管外科数据集上的精确率、召回率、F1值分别为96.45%、97.77%、97.10%, 均优于对比模型, 具有更好的实体识别性能。

关键词: 电子病历, 血管外科, 命名实体识别, 特征融合, 深度学习

Abstract:

As core components of healthcare information systems, Electronic Medical Record (EMR) entails numerous important medical entities. Named Entity Recognition (NER) of EMRs can significantly advance medical research. To address the challenges of limited research data and complex entity recognition in vascular surgery EMRs, a small-scale specialized dataset is constructed using real clinical data obtained from the vascular surgery department of a tertiary hospital. A NER model based on glyph features is proposed to improve the recognition accuracy. First, dynamic character vectors are generated using the Masked Language Model (MLM) as correction Bidirectional Encoder Representations from Transformers (MacBERT) and incorporating glyph information via the Chinese four-corner code and Wubi input methods. The text representations are then fed into a Bi-directional Gated Recurrent Unit (BiGRU) and Gated Dilated Convolutional Neural Network(DGCNN) for feature extraction, and the outputs are subsequently concatenated. Finally, the model employs a multihead self-attention mechanism to capture the relationships between sequence elements and uses Conditional Random Field (CRF) for label decoding. Experimental results demonstrate that the proposed model achieves precision, recall, and F1 scores of 96.45%, 97.77%, and 97.10%, respectively, on the self-constructed vascular surgery dataset. These results indicate that the proposed model outperforms the comparison models and demonstrates superior entity recognition performance.

Key words: Electronic Medical Record(EMR), vascular surgery, Named Entity Recognition(NER), feature fusion, deep learning

张华青, 夏张涛, 陆晓庆, 童基均. 基于字形特征的血管外科命名实体识别[J]. 计算机工程, 2024, 50(8): 13-21.

Huaqing ZHANG, Zhangtao XIA, Xiaoqing LU, Jijun TONG. Named Entity Recognition of Vascular Surgery Based on Glyph Features[J]. Computer Engineering, 2024, 50(8): 13-21.

https://www.ecice06.com/CN/Y2024/V50/I8/13

图/表 12

图1 MBDAC-FF模型结构

Fig.1 Structure of MBDAC-FF model

图2 GRU网络结构

Fig.2 Structure of GRU network

图3 门控卷积架构

Fig.3 Architecture of gated convolution

图4 多头自注意力机制结构

Fig.4 Structure of multihead self-attention mechanism

图5 CRF标签约束

Fig.5 CRF label constraints

参考文献 25

1	史振宇, 符伟国. 中国血管外科的现状和展望. 上海医学, 2017, 40(3): 135- 138. URL
	SHI Z Y, FU W G. Present situation and prospect of vascular surgery in China. Shanghai Medical Journal, 2017, 40(3): 135- 138. URL
2	KOCAMAN V, TALBY D. Accurate clinical and biomedical named entity recognition at scale. Software Impacts, 2022, 13, 100373. doi: 10.1016/j.simpa.2022.100373
3	RABINER L, JUANG B. An introduction to hidden Markov models. IEEE ASSP Magazine, 1986, 3(1): 4- 16. doi: 10.1109/MASSP.1986.1165342
4	CORTES C, VAPNIK V. Support-vector networks. Machine Learning, 1995, 20(3): 273- 297. doi: 10.1023/A:1022627411411
5	JAYNES E T. Information theory and statistical mechanics. Physical Review, 1957, 106(4): 620- 630. doi: 10.1103/PhysRev.106.620
6	LAFFERTY J D, MCCALLUM A, PEREIRA F C N. Conditional random fields: probabilistic models for segmenting and labeling sequence data[C]//Proceedings of the 18th International Conference on Machine Learning. New York, USA: ACM Press, 2001: 282-289.
7	LECUN Y, BOSER B, DENKER J S, et al. Backpropagation applied to handwritten zip code recognition. Neural Computation, 1989, 1(4): 541- 551. doi: 10.1162/neco.1989.1.4.541
8	ELMAN J. Finding structure in time. Cognitive Science, 1990, 14(2): 179- 211. doi: 10.1207/s15516709cog1402_1
9	HOCHREITER S, SCHMIDHUBER J. Long short-term memory. Neural Computation, 1997, 9(8): 1735- 1780. doi: 10.1162/neco.1997.9.8.1735
10	CHO K, VAN MERRIENBOER B, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Stroudsburg, USA: Association for Computational Linguistics, 2014: 1724-1734.
11	LIN Y F, TSAI T H, CHOU W C, et al. A maximum entropy approach to biomedical named entity recognition[C]//Proceedings of the 4th International Conference on Data Mining in Bioinformatics. New York, USA: ACM Press, 2004: 56-61.
12	YI E, LEE G G, SONG Y, et al. SVM-based biological named entity recognition using minimum edit-distance feature boosted by virtual examples[M]. Berlin, Germany: Springer, 2005.
13	ZHAO Z H, YANG Z H, LUO L, et al. ML-CNN: a novel deep learning based disease named entity recognition architecture[C]//Proceedings of the IEEE International Conference on Bioinformatics and Biomedicine (BIBM). Washington D. C., USA: IEEE Press, 2016: 794.
14	PHAN R, LUU T M, DAVEY R, et al. Biomedical named entity recognition based on hybrid multistage CNN-RNN learner[C]//Proceedings of the International Conference on Machine Learning and Data Engineering (iCMLDE). Washington D. C., USA: IEEE Press, 2018: 128-135.
15	WU F Z, LIU J X, WU C H, et al. Neural Chinese named entity recognition via CNN-LSTM-CRF and joint training with word segmentation[C]//Proceedings of the World Wide Web Conference. New York, USA: ACM Press, 2019: 3342-3348.
16	CHANG Y, KONG L, JIA K J, et al. Chinese named entity recognition method based on BERT[C]//Proceedings of the IEEE International Conference on Data Science and Computer Application (ICDSCA). Washington D. C., USA: IEEE Press, 2021: 294-299.
17	CAI X C, SUN E H, LEI J L. Research on application of named entity recognition of electronic medical records based on BERT-IDCNN-CRF model[C]//Proceedings of the 6th International Conference on Graphics and Signal Processing. New York, USA: ACM Press, 2022: 80-85.
18	YIN X W, ZHENG S, WANG Q M. Fine-grained Chinese named entity recognition based on RoBERTa-WWM-BiLSTM-CRF model[C]//Proceedings of the 6th International Conference on Image, Vision and Computing (ICIVC). Washington D. C., USA: IEEE Press, 2021: 408-413.
19	ZHANG L, LAI P C, YE F Y, et al. Chinese medical named entity recognition using external knowledge[C]// Proceedings of the 19th Pacific Rim International Conference on Artificial Intelligence. Berlin, Germany: Springer, 2022: 359-371.
20	KONG J, ZHANG L X, JIANG M, et al. Incorporating multi-level CNN and attention mechanism for Chinese clinical named entity recognition. Journal of Biomedical Informatics, 2021, 116, 103737. doi: 10.1016/j.jbi.2021.103737
21	CUI Y M, CHE W X, LIU T, et al. Revisiting pre-trained models for Chinese natural language processing[C]//Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2020. Stroudsburg, USA: Association for Computational Linguistics, 2020: 657-668.
22	STRUBELL E, VERGA P, BELANGER D, et al. Fast and accurate entity recognition with iterated dilated convolutions[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: Association for Computational Linguistics, 2017: 2670-2680.
23	梁文桐, 朱艳辉, 詹飞, 等. 基于深度学习多模型融合的医疗命名实体识别. 计算机应用与软件, 2022, 39(10): 162-168, 229. URL
	LIANG W T, ZHU Y H, ZHAN F, et al. Medical named entity recognition based on deep learning multi-model fusion. Computer Applications and Software, 2022, 39(10): 162-168, 229. URL
24	吕江海, 杜军平, 周南, 等. 基于膨胀卷积迭代与注意力机制的实体名识别方法. 计算机工程, 2021, 47(1): 58-65, 71. URL
	LÜ J H, DU J P, ZHOU N, et al. Entity Name recognition method based on dilated convolutional iterative and attention mechanism. Computer Engineering, 2021, 47(1): 58-65, 71. URL
25	王笑月, 李茹, 段菲. 一种基于门控空洞卷积的高效中文命名实体识别方法. 中文信息学报, 2021, 35(1): 72- 80. URL
	WANG X Y, LI R, DUAN F. An efficient Chinese named entity recognition method based on gated-dilated convolution. Journal of Chinese Information Processing, 2021, 35(1): 72- 80. URL

[1]	魏嵬, 丁香香, 郭梦星, 杨钊, 刘辉. 文本相似度计算方法综述[J]. 计算机工程, 2024, 50(9): 18-32.
[2]	党小超, 刘涧, 董晓辉, 祝忠彦, 李芬芳. 面向不平衡数据的机械设备故障命名实体识别[J]. 计算机工程, 2024, 50(9): 104-112.
[3]	李俊仪, 李向阳, 龙朝勋, 李海燕, 李红松, 余鹏飞. 基于多级区域选择与跨层特征融合的野生菌分类[J]. 计算机工程, 2024, 50(9): 179-188.
[4]	朱凯, 李理, 张彤, 江晟, 别一鸣. 基于Transformer的多阶段运动模糊图像修复网络[J]. 计算机工程, 2024, 50(9): 276-285.
[5]	张天鹏, 韩晶, 吕学强. 基于多任务学习的超分辨率辅助小目标检测[J]. 计算机工程, 2024, 50(9): 304-312.
[6]	高煜宝, 文志诚. 基于注意力机制的双路解码器图像去噪方法[J]. 计算机工程, 2024, 50(9): 324-332.
[7]	李华昱, 张智康, 闫阳, 岳阳. 基于知识图谱增强的领域多模态实体识别[J]. 计算机工程, 2024, 50(8): 31-39.
[8]	刘锁兰, 王炎, 王洪元, 朱生升. 基于多流语义图卷积网络的人体行为识别[J]. 计算机工程, 2024, 50(8): 64-74.
[9]	张亚洲, 和玉, 戎璐, 王祥凯. 基于上下文知识增强型Transformer网络的抑郁检测[J]. 计算机工程, 2024, 50(8): 75-85.
[10]	赵婉秋, 张俊虎, 李海涛. 用于建筑物分割的平行结构特征融合网络[J]. 计算机工程, 2024, 50(8): 239-248.
[11]	赵宏, 王枭. 基于Swin-Transformer的黑色素瘤图像病灶分割研究[J]. 计算机工程, 2024, 50(8): 249-258.
[12]	王富平, 刘鸿玮, 张锲石, 段冠庄. 基于深度特征抑制的遮挡人脸识别网络[J]. 计算机工程, 2024, 50(8): 259-269.
[13]	闵莉, 董冰洁, 安冬. 基于多注意力机制与跨特征融合的语义分割算法[J]. 计算机工程, 2024, 50(8): 282-289.
[14]	高伟, 李帅龙, 茆琳, 王磊, 李颖颖, 韩林. 一种基于TVM的算子生成加速策略[J]. 计算机工程, 2024, 50(8): 353-362.
[15]	陈宇航, 杨勇, 先木斯亚·买买提明, 帕力旦·吐尔逊, 樊小超, 任鸽, 刁宇峰. 基于主题感知和语义增强的作文自动评分方法[J]. 计算机工程, 2024, 50(8): 363-371.

选择文件类型/文献管理软件名称

选择包含的内容