融合数据增强与半监督学习的药物不良反应检测

doi:10.19678/j.issn.1000-3428.0062170

计算机工程 ›› 2022, Vol. 48 ›› Issue (6): 314-320. doi: 10.19678/j.issn.1000-3428.0062170

• 开发研究与工程应用 • 上一篇

融合数据增强与半监督学习的药物不良反应检测

佘朝阳^1,2, 严馨^1,2, 徐广义³, 陈玮^1,2, 邓忠莹¹

1. 昆明理工大学信息工程与自动化学院, 昆明 650504;
2. 昆明理工大学云南省人工智能重点实验室, 昆明 650504;
3. 云南南天电子信息产业股份有限公司, 昆明 650041

收稿日期:2021-07-22 修回日期:2021-09-07 发布日期:2021-09-10
作者简介:佘朝阳(1996—),女,硕士研究生,主研方向为自然语言处理;严馨,副教授、硕士;徐广义,高级工程师、硕士;陈玮、邓忠莹,讲师、硕士。
基金资助:
国家自然科学基金（61462055，61562049）。

Adverse Drug Reaction Detection Combined with Data Augmentation and Semi-supervised Learning

SHE Zhaoyang^1,2, YAN Xin^1,2, XU Guangyi³, CHEN Wei^1,2, DENG Zhongying¹

1. Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650504, China;
2. Yunnan Key Laboratory of Artificial Intelligence, Kunming University of Science and Technology, Kunming 650504, China;
3. Yunnan Nantian Electronic Information Industry Co., Ltd., Kunming 650041, China

Received:2021-07-22 Revised:2021-09-07 Published:2021-09-10

摘要/Abstract

摘要： 目前药物不良反应（ADR）研究使用的数据主要来源于英文语料，较少选用存在标注数据稀缺问题的中文医疗社交媒体数据集，导致对中文医疗社交媒体的研究有限。为解决标注数据稀缺的问题，提出一种新型的ADR检测方法。采用ERNIE预训练模型获取文本的词向量，利用BiLSTM模型和注意力机制学习文本的向量表示，并通过全连接层和softmax函数得到文本的分类标签。对未标注数据进行文本增强，使用分类模型获取低熵标签，此标签被作为原始未标注样本及其增强样本的伪标签。此外，将带有伪标签的数据与人工标注数据进行混合，在分类模型的编码层和分类层间加入Mixup层，并在文本向量空间中使用Mixup增强方法插值混合样本，从而扩增样本数量。通过将数据增强和半监督学习相结合，充分利用标注数据与未标注数据，实现ADR的检测。实验结果表明，该方法无需大量的标注数据，缓解了标注数据不足对检测结果的影响，有效提升了药物不良反应检测模型的性能。

关键词: 医疗社交媒体, 药物不良反应, 数据增强, 半监督学习, 预训练语言模型

Abstract: At present, the data used in the study of Adverse Drug Reaction (ADR) are mainly from English corpus, fewer Chinese medical social media data sets are selected because of label data scarcity, resulting in limited research on Chinese medical social media.To deal with the problem of lack of labeled data, this study proposes an ADR detection method that combines data augmentation and semi-supervised learning.The pre-training ERNIE model is used to obtain the word vectors.BiLSTM and the attention mechanism are used to learn the vector representation of the text.The classification layer consists of a fully connected layer and a softmax function to obtain the classification label.First, the unlabeled data are augmented several times.The low-entropy label, which is the weighted average of the predicted values of the original and augmented samples, is shared by these samples.The pseudo-label data are then mixed with the labeled data.Based on the classification model, a Mixup layer is added between the encoding and classification layers.In the text vector space, Mixup is used to interpolate the mixed samples, and the number of samples will be higher.By combining data augmentation and semi-supervised learning, labeled and unlabeled data are fully utilized to detect adverse drug reactions.Experimental results show that this method does not require a large amount of labeled data, alleviates the impact of insufficient labeled data, and effectively improves the performance.

Key words: medical social media, Adverse Drug Reaction(ADR), data augmentation, semi-supervised learning, pre-training language model

中图分类号:

TP391

佘朝阳, 严馨, 徐广义, 陈玮, 邓忠莹. 融合数据增强与半监督学习的药物不良反应检测[J]. 计算机工程, 2022, 48(6): 314-320.

SHE Zhaoyang, YAN Xin, XU Guangyi, CHEN Wei, DENG Zhongying. Adverse Drug Reaction Detection Combined with Data Augmentation and Semi-supervised Learning[J]. Computer Engineering, 2022, 48(6): 314-320.

http://www.ecice06.com/CN/Y2022/V48/I6/314

图/表 7

20220625181646

20220625181649

20220625181659

20220625181704

20220625181708

20220625181711

20220625181715

参考文献

[1] HARPAZ R, DUMOUCHEL W, SHAH N H, et al.Novel data-mining methodologies for adverse drug event discovery and analysis[J].Clinical Pharmacology and Therapeutics, 2012, 91(6):1010-1021.
[2] WANG W, HAERIAN K, SALMASIAN H, et al.A drug-adverse event extraction algorithm to support pharmacovigilance knowledge mining from PubMed citations[J].AMIA Annual Symposium Proceedings, 2011, 25(3):64-70.
[3] SOHN S, KOCHER J P A, CHUTE C G, et al.Drug side effect extraction from clinical narratives of psychiatry and psychology patients[J].Journal of the American Medical Informatics Association, 2011, 18(1):144-149.
[4] WARRER P, HANSEN E H, JUHL JENSEN L, et al.Using text-mining techniques in electronic patient records to identify ADRs from medicine use[J].British Journal of Clinical Pharmacology, 2012, 73(5):674-684.
[5] WU H, FANG H, STANHOPE S J.Exploiting online discussions to discover unrecognized drug side effects[J].Methods of Information in Medicine, 2013, 52(2):152-159.
[6] YATES A, GOHARIAN N.ADRTrace:detecting expected and unexpected adverse drug reactions from user reviews on social media sites[C]//Proceedings of the 35th European Conference on Advances in Information Retrieval.Berlin, Germany:Springer, 2013:816-819.
[7] SARKER A, GONZALEZ G.Portable automatic text classification for adverse drug reaction detection via multi-corpus training[J].Journal of Biomedical Informatics, 2015, 53(4):196-207.
[8] NIKFARJAM A, SARKER A, O'CONNOR K, et al.Pharmacovigilance from social media:mining adverse drug reaction mentions using sequence labeling with word embedding cluster features[J].Journal of the American Medical Informatics Association, 2015, 22(3):671-681.
[9] LEE K, QADIR A, HASAN S A, et al.Adverse drug event detection in tweets with semi-supervised convolutional neural networks[EB/OL].[2021-06-20].https://dl.acm.org/doi/10.1145/3038912.3052671.
[10] COCOS A, FIKS A G, MASINO A J.Deep learning for pharmacovigilance:recurrent neural network architectures for labeling adverse drug reactions in Twitter posts[J].Journal of the American Medical Informatics Association, 2017, 24(4):813-821.
[11] HUYNH T, HE Y, WILLIS A, et al.Adverse drug reaction classification with deep neural networks[C]//Proceedings of the 26th International Conference on Computational Linguistics:Technical Papers.Osaka, Japan:[s.n.], 2016:877-887.
[12] PANDEY C, IBRAHIM Z, WU H H, et al.Improving RNN with attention and embedding for adverse drug reactions[C]//Proceedings of 2017 International Conference on Digital Health.New York, USA:ACM Press, 2017:67-71.
[13] WEI J, ZOU K.EDA:easy data augmentation techniques for boosting performance on text classification tasks[C]//Proceedings of 2019 Conference on Empirical Methods in Natural Language.Stroudsburg, USA:Association for Computational Linguistics, 2019:6381-6387.
[14] EDUNOV S, OTT M, AULI M, et al.Understanding back-translation at scale[EB/OL].[2021-06-22].https://arxiv.org/abs/1808.09381.
[15] XIE Q Z, DAI Z H, HOVY E, et al.Unsupervised data augmentation for consistency training[EB/OL].[2021-06-22].https://arxiv.org/abs/1904.12848.
[16] GUO H Y, MAO Y Y, ZHANG R C.Augmenting data with mixup for sentence classification:an empirical study[EB/OL].[2021-06-22].https://arxiv.org/abs/1905.08941.
[17] BERTHELOT D, CARLINI N, GOODFELLOW I, et al.MixMatch:a holistic approach to semi-supervised learning[EB/OL].[2021-06-22].https://www.researchgate.net/publication/332932671_MixMatch_A_Holistic_Approach_to_Semi-Supervised_Learning.
[18] SOHN K, BERTHELOT D, LI C L, et al.FixMatch:simplifying semi-supervised learning with consistency and confidence[EB/OL].[2021-06-22].https://arxiv.org/abs/2001.07685.
[19] ZHANG H Y, CISSE M, DAUPHIN Y N, et al.Mixup:beyond empirical risk minimization[EB/OL].[2021-06-22].https://arxiv.org/abs/1710.09412.
[20] SUN Y, WANG S, LI Y, et al.ERNIE:enhanced representation through knowledge integration[C]//Proceedings of AAAI Conference on Artificial Intelligence.San Francisco, USA:AAAI Press, 2020:8968-8975.
[21] CHEN J A, YANG Z C, YANG D Y.MixText:linguistically-informed interpolation of hidden space for semi-supervised text classification[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.Stroudsburg, USA:Association for Computational Linguistics, 2020:2147-2157.
[22] LEE D H.Pseudo-label:the simple and efficient semi-supervised learning method for deep neural networks[EB/OL].[2021-06-22].https://www.researchgate.net/publication/280581078.
[23] LAINE S, AILA T M.Temporal ensembling for semi-supervised learning[J].IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, 2016, 12:143-152.
[24] TARVAINEN A, VALPOLA H.Mean teachers are better role models:weight-averaged consistency targets improve semi-supervised deep learning results[C]//Proceedings of International Conference on Learning Representations.Vancouver, Canada:[s.n.], 2017:156-168.

选择文件类型/文献管理软件名称

选择包含的内容

融合数据增强与半监督学习的药物不良反应检测

Adverse Drug Reaction Detection Combined with Data Augmentation and Semi-supervised Learning

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 7

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	江雨燕, 陶承凤, 李平. 数据增强和自适应自步学习的深度子空间聚类算法[J]. 计算机工程, 2023, 49(8): 96-103, 110.
[2]	陈仲磊, 伊鹏, 陈祥, 胡涛. 基于集成学习的系统调用实时异常检测框架[J]. 计算机工程, 2023, 49(6): 162-169,179.
[3]	王禹博, 陈利锋, 许卫霞. 结合多解码器与两阶段通道选择的异常检测方法[J]. 计算机工程, 2023, 49(3): 37-48.
[4]	席荣康, 蔡满春, 芦天亮. 基于数据增强与流数据处理的Tor流量分析模型[J]. 计算机工程, 2023, 49(3): 177-184.
[5]	陈柏霖, 王天极, 任丽娜, 黄瑞章. 融合ELECTRA和文本局部信息的中文语法错误检测方法[J]. 计算机工程, 2023, 49(3): 304-311.
[6]	禹克强, 黄芳, 吴琪, 欧阳洋. 基于双向语义的中文实体关系联合抽取方法[J]. 计算机工程, 2023, 49(1): 92-99,112.
[7]	黄君扬, 王振宇, 梁家卿, 肖仰华. 基于自裁剪异构图的NL2SQL模型[J]. 计算机工程, 2022, 48(9): 71-77,88.
[8]	孙伟, 常鹏帅, 戴亮, 张小瑞, 陈旋, 代广昭. 基于注意力引导数据增强的车型识别[J]. 计算机工程, 2022, 48(7): 300-306.
[9]	王士浩, 王中卿, 李寿山, 周国栋. 基于知识蒸馏与模型集成的事件论元抽取方法[J]. 计算机工程, 2022, 48(7): 97-103.
[10]	李军怀, 陈苗苗, 王怀军, 崔颖安, 张爱华. 基于ALBERT-BGRU-CRF的中文命名实体识别方法[J]. 计算机工程, 2022, 48(6): 89-94,106.
[11]	曹瑞阳, 郭佑民, 牛满宇. 基于最大最小距离的多中心数据综合增强方法[J]. 计算机工程, 2022, 48(6): 174-181.
[12]	鲁统伟, 徐子昕, 闵锋. 基于生成对抗网络的知识蒸馏数据增强[J]. 计算机工程, 2022, 48(4): 70-80.
[13]	谭润, 叶武剑, 刘怡俊. 结合双语义数据增强与目标定位的细粒度图像分类[J]. 计算机工程, 2022, 48(2): 237-242,249.
[14]	乔栋, 陈章进, 邓良, 屠程力. 基于改进语音处理的卷积神经网络中文语音情感识别方法[J]. 计算机工程, 2022, 48(2): 281-290.
[15]	于尊瑞, 毛震东, 王泉, 张勇东. 基于预训练语言模型的关键词感知问题生成[J]. 计算机工程, 2022, 48(2): 125-131.

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

融合数据增强与半监督学习的药物不良反应检测

Adverse Drug Reaction Detection Combined with Data Augmentation and Semi-supervised Learning

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 7

参考文献

相关文章 15

编辑推荐

Metrics

本文评价