教育领域下多维度特征命名实体识别方法

doi:10.19678/j.issn.1000-3428.0068669

摘要/Abstract

摘要：

信息技术的发展与进步促使“互联网+教育”成为目前教育领域的研究热点, 教育教学的各个环节都在向智能化的方向发展。中学数学的命名实体识别(NER)任务的研究, 可为后续构建中学数学学科知识图谱及自动问答等任务奠定基础, 进而满足中学生个性化知识获取的需求, 助力新型智能化教育体系的构建。目前中学数学知识语义复杂, 其NER和研究数据较少, 且在当前主流模型特征提取任务中容易忽略掉部分局部特征。为解决该领域的实体识别困难问题, 以自建的中学数学知识语料库为研究对象, 提出一种融合多头注意力的多维度特征NER方法。该方法首先采用BERT进行文本表征预训练得到词向量, 接着引入对抗训练对每个嵌入向量进行扰动, 将得到的对抗样本和嵌入向量传送到多维度特征提取层进行特征提取, 再将输出的特征进行拼接, 通过多头注意力机制进行动态融合, 最终经过条件随机场(CRF)修正后输出。实验结果表明, 该方法在自建Educ数据集上的识别准确率、召回率以及F1值分别达到96.68%、97.71%和97.19%, 证明了该方法在中学数学知识实体识别上的有效性。

关键词: 命名实体识别, 教育领域, 对抗训练, 多维度特征提取, 多头注意力机制

Abstract:

The development and progress of information technology have resulted in extensive investigations into ″Internet + Education″ in the field of education, and all aspects of education and teaching are being developed in the direction of intelligence. The study of Named Entity Recognition (NER) in secondary school mathematics can provide a foundation for the subsequent construction of secondary school mathematics knowledge mapping and automatic question-and-answer tasks to fulfill the demands of secondary school students for personalized knowledge acquisition and facilitate the construction of a new intelligent education system. Currently, owing to the semantic complexity of secondary school mathematics knowledge, its NER and research data are insufficient, and the current mainstream model for feature extraction disregards some local features. To solve the challenges of entity recognition in this field, a multidimensional feature NER method incorporating multihead attention is proposed using a self-constructed corpus of secondary school mathematics knowledge. First, the method adopts Bidirectional Encoder Representations from Transformers (BERT) for pre-training text representations to obtain word vectors. Subsequently, this method introduces adversarial training to perturb each embedding vector and then transmits the obtained adversarial samples and embedding vectors to the multidimensional feature extraction layer for feature extraction. Next, it splices the output features, dynamically fuses them via the multihead attention mechanism, and finally outputs them after correction by a Conditional Random Field (CRF). Experimental results show that the accuracy, recall, and F1 value of this method for recognizing the self-constructed Educ dataset are 96.68%, 97.71%, and 97.19%, respectively, thus demonstrating its effectiveness in recognizing mathematical knowledge entities in secondary schools.

Key words: Named Entity Recognition(NER), educational domain, adversarial training, multidimensional feature extraction, multi-head attention mechanism

任义, 苏博, 袁帅. 教育领域下多维度特征命名实体识别方法[J]. 计算机工程, 2024, 50(10): 110-118.

REN Yi, SU Bo, YUAN Shuai. Multidimensional Feature Named Entity Recognition Method in Education Domain[J]. Computer Engineering, 2024, 50(10): 110-118.

https://www.ecice06.com/CN/Y2024/V50/I10/110

图/表 17

图1 序列标注示例

Fig.1 Example of sequence labeling

图2 融合多头注意力的多维度特征命名实体识别模型

Fig.2 Multidimensional feature named entity recognition model incorporating multi-head attention

图3 BERT的网络结构

Fig.3 Network structure of BERT

图4 BERT的输入剖析

Fig.4 Input profiling of BERT

图5 多维度特征提取层结构

Fig.5 Multidimensional feature extraction layer structure

图6 膨胀卷积示意图

Fig.6 Schematic drawing of expansion convolution

图7 多头注意力机制网络结构

Fig.7 Structure of multi-head attention mechanism network

参考文献 23

1	涂冬波, 蔡艳, 丁树良. 认知诊断理论、方法与应用. 北京: 北京师范大学出版社, 2012.
	TU D B , CAI Y , DING S L . Cognitive diagnosis: theory, methods and applications. Beijing: Beijing Normal University Press, 2012.
2	LI J , SUN A X , HAN J L , et al. A survey on deep learning for named entity recognition. IEEE Transactions on Knowledge and Data Engineering, 2022, 34 (1): 50- 70. doi: 10.1109/TKDE.2020.2981314
3	BI M W , ZHANG Q C , ZUO M , et al. Bi-directional long short-term memory model with semantic positional attention for the question answering system. ACM Transactions on Asian and Low-Resource Language Information Processing, 2021, 20 (5): 1- 13.
4	赵京胜, 宋梦雪, 高祥, 等. 自然语言处理中的文本表示研究. 软件学报, 2022, 33 (1): 102- 128.
	ZHAO J S , SONG M X , GAO X , et al. Research on text representation in natural language processing. Journal of Software, 2022, 33 (1): 102- 128.
5	张毅, 王爽胜, 何彬, 等. 基于BERT的初等数学文本命名实体识别方法. 计算机应用, 2022, 42 (2): 433- 439.
	ZHANG Y , WANG S S , HE B , et al. Named entity recognition method of elementary mathematical text based on BERT. Journal of Computer Applications, 2022, 42 (2): 433- 439.
6	HOCHREITER S , SCHMIDHUBER J . Long short-term memory. Neural Computation, 1997, 9 (8): 1735- 1780. doi: 10.1162/neco.1997.9.8.1735
7	刘世泽, 秦艳君, 王晨星, 等. 基于多尺度特征提取的交通模式识别算法. 计算机应用, 2021, 41 (6): 1573- 1580.
	LIU S Z , QIN Y J , WANG C X , et al. Transportation mode recognition algorithm based on multi-scale feature extraction. Journal of Computer Applications, 2021, 41 (6): 1573- 1580.
8	DEVLIN J, CHANG M W, LEE K, et al. Bert: pre-training of deep bidirectional transformers for language understanding[EB/OL]. [2023-09-10]. https://arxiv.org/abs/1810.04805.
9	刘斐, 文中, 吴艺. 基于BERT-BILSTM-CRF模型的电力行业事故文本智能分析. 中国安全生产科学技术, 2023, 19 (1): 209- 215.
	LIU F , WEN Z , WU Y . Intelligent analysis on text of power industry accident based on BERT-BILSTM-CRF model. Journal of Safety Science and Technology, 2023, 19 (1): 209- 215.
10	肖勇, 郑楷洪, 王鑫, 等. 基于联合神经网络学习的中文电力计量命名实体识别. 浙江大学学报(理学版), 2021, 48 (3): 321- 330.
	XIAO Y , ZHENG K H , WANG X , et al. Chinese named entity recognition in electric power metering domain based on neural joint learning. Journal of Zhejiang University (Science Edition), 2021, 48 (3): 321- 330.
11	曾青霞, 熊旺平, 杜建强, 等. 结合自注意力的BiLSTM-CRF的电子病历命名实体识别. 计算机应用与软件, 2021, 38 (3): 159-162, 234.
	ZENG Q X , XIONG W P , DU J Q , et al. Electronic medical record named entity recognition combined with self-attention bilstm-crf. Computer Applications and Software, 2021, 38 (3): 159-162, 234.
12	廖涛, 勾艳杰, 张顺香. 融合注意力机制的BERT-BiLSTM-CRF中文命名实体识别. 阜阳师范大学学报(自然科学版), 2021, 38 (3): 86- 91.
	LIAO T , GOU Y J , ZHANG S X . BERT-BiLSTM-CRF Chinese named entity recognition combined with attention mechanism. Journal of Fuyang Normal University (Natural Science), 2021, 38 (3): 86- 91.
13	AN Y , XIA X Y , CHEN X L , et al. Chinese clinical named entity recognition via multi-head self-attention based BiLSTM-CRF. Artificial Intelligence in Medicine, 2022, 127, 102282. doi: 10.1016/j.artmed.2022.102282
14	ZHANG Q H , WU M , LÜ P T , et al. Research on named entity recognition of Chinese electronic medical records based on multi-head attention mechanism and character-word information fusion. Journal of Intelligent & Fuzzy Systems, 2022, 42 (4): 4105- 4116.
15	苏博. 教育领域中学数学NER数据集[EB/OL]. [2023-09-10]. https://blog.csdn.net/sooxiaober/article/details/134898790.
	SU B. NER dataset for middle school mathematics in the field of education [EB/OL]. [2023-12-10]. https://blog.csdn.net/sooxiaober/article/details/134898790. (in Chinese)
16	杨长沛, 廖列法. 基于门控空洞卷积特征融合的中文命名实体识别. 计算机工程, 2023, 49 (8): 85- 95.
	YANG C P , LIAO L F . Chinese named entity recognition based on dilated gated convolution feature fusion. Computer Engineering, 2023, 49 (8): 85- 95.
17	赵建强, 朱万彤, 陈诚. 基于多重卷积神经网络模型的命名实体识别. 计算机技术与发展, 2023, 33 (1): 187- 192.
	ZHAO J Q , ZHU W T , CHEN C . Named entity recognition based on duplex convolution neural network model. Computer Technology and Development, 2023, 33 (1): 187- 192.
18	YANG Z, DAI Z, YANG Y, et al. XLNet: generalized autoregressive pretraining for language understanding[C]//Proceedings of Neural Information Processing Systems. Cambridge, USA: MIT Press, 2019: 5753-5763.
19	LAN Z H, CHEN M D, GOODMAN S, et al. ALBERT: a lite BERT for self-supervised learning of language representations[C]//Proceedings of the 8th International Conference on Learning Representations. Washington D. C., USA: IEEE Press, 2020: 1-17.
20	LIU Y, OTT M, GOYAL N, et al. RoBERTa: a robustly optimized BERT pretraining approach[EB/OL]. [2023-09-10]. https://arxiv.org/abs/1907.11692.
21	方美丽, 郑莹莹, 陶坤旺, 等. 基于MacBERT和对抗训练的城市内涝信息识别方法. 集成技术, 2023, 12 (1): 56- 67.
	FANG M L , ZHENG Y Y , TAO K W , et al. Urban waterlogging information recognition method based on MacBERT and adversarial training. Journal of Integration Technology, 2023, 12 (1): 56- 67.
22	LEVOW G. The third international Chinese language processing bakeoff: word segmentation and named entity recognition[C]//Proceedings of the 5th SIGHAN Workshop on Chinese Language Processing. Sydney, Australia: Association for Computational Linguistics, 2006: 108-117.
23	GUI T, MA R T, ZHANG Q, et al. CNN-based Chinese NER with lexicon rethinking[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence. Washington D. C., USA: IEEE Press, 2019: 4982-4988.

[1]	党小超, 刘涧, 董晓辉, 祝忠彦, 李芬芳. 面向不平衡数据的机械设备故障命名实体识别[J]. 计算机工程, 2024, 50(9): 104-112.
[2]	王言国, 吕鹏远, 兰金江, 刘明哲, 秦冠军, 张硕桦, 周宇. 基于对抗训练与Transformer的风力发电机故障分类方法[J]. 计算机工程, 2024, 50(9): 377-384.
[3]	张华青, 夏张涛, 陆晓庆, 童基均. 基于字形特征的血管外科命名实体识别[J]. 计算机工程, 2024, 50(8): 13-21.
[4]	李华昱, 张智康, 闫阳, 岳阳. 基于知识图谱增强的领域多模态实体识别[J]. 计算机工程, 2024, 50(8): 31-39.
[5]	魏琢艺, 罗迈, 李文兵, 曾远松, 余伟江, 杨跃东. 基于多源域适应的单细胞智能分类[J]. 计算机工程, 2024, 50(6): 48-55.
[6]	周昭辰, 方清茂, 吴晓红, 胡平, 何小海. 基于MacBERT与对抗训练的机器阅读理解模型[J]. 计算机工程, 2024, 50(5): 41-50.
[7]	代巍, 王丰羽, 冀常鹏. 基于情感增强与双图卷积网络的方面级情感分析[J]. 计算机工程, 2024, 50(5): 120-127.
[8]	隗昊, 刁宏悦, 孔亮宸, 邓耀臣. 东北亚舆情文本细粒度命名实体识别方法研究[J]. 计算机工程, 2024, 50(5): 354-362.
[9]	隗昊, 刁宏悦, 孔亮宸, 邓耀臣. 东北亚舆情文本细粒度命名实体识别方法研究[J]. 计算机工程, 2024, 50(5): 354-362.
[10]	刘威, 马磊, 李凯, 李蓉. 基于多粒度字形增强的中文医学命名实体识别[J]. 计算机工程, 2024, 50(2): 337-344.
[11]	唐卓然, 柳毅. 基于词汇融合和依存关系的中文命名实体识别[J]. 计算机工程, 2024, 50(10): 145-153.
[12]	沈志东, 岳恒宪. 基于分布式扰动的文本对抗训练方法[J]. 计算机工程, 2023, 49(9): 16-22.
[13]	杨长沛, 廖列法. 基于门控空洞卷积特征融合的中文命名实体识别[J]. 计算机工程, 2023, 49(8): 85-95.
[14]	张家熔, 苑津莎, 许珈宁, 罗志宏. 基于多元信息嵌入与协同神经网络的力学实体识别算法[J]. 计算机工程, 2023, 49(7): 125-134.
[15]	陈明, 刘蓉, 张晔. 基于多重注意力机制的中文医疗实体识别[J]. 计算机工程, 2023, 49(6): 314-320.

选择文件类型/文献管理软件名称

选择包含的内容