基于用户意图的微博文本生成技术研究

doi:10.19678/j.issn.1000-3428.0060079

摘要/Abstract

摘要： 微博是个人和组织用户分享或获取简短实时信息的重要社交平台，微博文本自动生成技术能帮助用户在微博平台上快速实现各种社交意图。为辅助用户发表博文并表达社交意图，提出一种基于用户意图的微博文本生成技术，以挖掘提取微博文本特征，并在给定微博主题的条件下生成与用户意图相一致的微博文本。采用预训练语言模型与微调相结合的方法，在预训练语言模型GPT2上实现联合主题和用户意图的文本控制生成，以及具备用户对话功能的文本预测生成。实验结果表明，该技术生成的文本具有较高的可读性且符合微博文本语言风格，结合主题和5类用户意图的生成样本人工评分达77分以上。

关键词: 微博文本, 自动生成, 用户意图, 主题, 预训练语言模型, 微调

Abstract: Weibo is a mainstream social platform for individuals and organizational users to share or obtain short real-time information.The technique of Weibo text generation can help users quickly realize various social intentions on Weibo.In order to assist users in publishing blog posts and express social intentions, this paper proposes a Weibo text generation technology based on user intention, which mines and extracts Weibo text features, and generates Weibo texts that are consistent with the user intention under a given topic.Using the combination of pre-training language model and fine-tuning, the text control generation of joint topic and user intention and the text prediction generation with user dialogue function are realized on the pre-training language model, GPT2.The experimental results show that the proposed technology can generate texts with high readability and adherence to the language style of Weibo texts.The manual score of the generated samples combined with the theme and five types of user intention is more than 77 points.

Key words: Weibo text, automatic generation, user intention, topic, pre-training language model, fine-tuning

中图分类号:

TP18

高永兵, 黎预璇, 高军甜, 马占飞. 基于用户意图的微博文本生成技术研究[J]. 计算机工程, 2022, 48(1): 119-126.

GAO Yongbing, LI Yuxuan, GAO Juntian, MA Zhanfei. Research on Weibo Text Generation Technology Based on User Intention[J]. Computer Engineering, 2022, 48(1): 119-126.

http://www.ecice06.com/CN/Y2022/V48/I1/119

图/表 9

20220108122212

20220108122216

20220108122220

20220108122224

20220108122228

20220108122233

20220108122237

20220108122241

20220108122246

参考文献

[1] GOODFELLOW I J, JEAN P A, MEHDI M, et al.Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems.Montreal, Canada:NIPS, 2014:2672-2680.
[2] GUO J X, LU S D, HAN C, et al.Long text generation via adversarial training with leaked information[EB/OL].[2020-10-02].https://arxiv.org/pdf/1709.08624.pdf.
[3] ZHU Y, LU S, LEI Z, et al.Texygen:a benchmarking platform for text generation models[C]//Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval.New York, USA:ACM Press, 2018:1097-1100.
[4] YU L, ZHANG W, WANG J, et al.SeqGAN:sequence generative adversarial nets with policy gradient[EB/OL].[2020-10-02].https://arxiv.org/pdf/1609.05473.pdf.
[5] CHE T, LI Y, ZHANG R, et al.Maximum-likelihood augmented discrete generative adversarial networks[EB/OL].[2020-10-02].https://arxiv.org/pdf/1702.07983.pdf.
[6] SUTSKEVER H, VINYALS ORIOL, QUOC V L, et al.Sequence to sequence learning with neural networks[EB/OL].[2020-10-02].http://cs224d.stanford.edu/papers/seq2seq.pdf.
[7] LI J, SONG Y, ZHANG H, et al.Generating classical Chinese poems via conditional variational autoencoder and adversarial training[C]//Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing.Washington D.C., USA:IEEE Press, 2018:3890-3900.
[8] YAO L, PENG N, WEISCHEDEL R, et al.Plan-and-write:towards better automatic storytelling[J].Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33:7378-7385.
[9] ZENG W, ABUDUWEILI A, LI L, et al.Automatic generation of personalized comment based on user profile[C]//Proceedings of the 57th Conference of the Association for Computational Linguistics.Florence, Italy:Association for Computational Linguistics, 2019:229-235.
[10] DU Z Y.GPT2-Chinese:tools for training GPT2 model in Chinese language[EB/OL].[2020-10-02].https://github.com/Morizeyao/GPT2-Chinese.
[11] DEVLIN J, CHANG M W, LEE K, et al.BERT:pre-training of deep bidirectional transformers for language understanding[EB/OL].[2020-10-02].https://arxiv.org/pdf/1810.04805.pdf.
[12] PETERS M, NEUMANN M, IYYER M, et al.Deep contextualized word representations[C]//Proceedings of 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Florence, Italy:Association for Computational Linguistics, 2018:2227-2237.
[13] RADFORD A.Improving language understanding by generative pre-training[EB/OL].[2020-10-02].https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf.
[14] RADFORD A, JEFFREY W, REWON C, et al.Language models are unsupervised multitask learners[EB/OL].[2020-10-02].https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners. pdf.
[15] TOM B, BROWN B M, NICK R, et al.Language models are few-shot learners[EB/OL].[2020-10-02].https://papers.nips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf.
[16] YANG Z L, DAI Z H, YANG Y M, et al.XLNet:generalized autoregressive pretraining for language understanding[EB/OL].[2020-10-02].https://www.cs.cmu.edu/~jgc/publication/XLNET.pdf.
[17] 付博, 陈毅恒, 邵艳秋, 等.基于用户自然标注的微博文本的消费意图识别[J].中文信息学报, 2017, 31(4):208-215. FU B, CHEN Y H, SHAO Y Q, et al.Consumption intent recognition based on user natural annotation[J].Journal of Chinese Information Processing, 2017, 31(4):208-215.(in Chinese)
[18] 贾云龙, 韩东红, 林海原, 等.面向微博用户的消费意图识别算法[J].北京大学学报(自然科学版), 2020, 56(1):68-74. JIA Y L, HAN D H, LIN H Y, et al.Consumption intent recognition algorithms for Weibo users[J].Acta Scientiarum Naturalium Universitatis Pekinensis, 2020, 56(1):68-74.(in Chinese)
[19] 桂思思, 陆伟, 张晓娟.基于查询表达式特征的时态意图识别研究[J].数据分析与知识发现, 2019, 3(3):66-75. GUI S S, LU W, ZHANG X J.Temporal intent classification with query expression feature[J].Data Analysis and Knowledge Discovery, 2019, 3(3):66-75.(in Chinese)
[20] 廖胜兰, 吉建民, 俞畅, 等.基于BERT模型与知识蒸馏的意图分类方法[J].计算机工程, 2021, 47(5):73-79. LIAO S L, JI J M, YU C, et al.Intention classification method based on BERT model and knowledge distillation[J].Computer Engineering, 2021, 47(5):73-79.(in Chinese)
[21] CHEN Q, ZHUO Z, WANG W, et al.BERT for joint intent classification and slot filling[EB/OL].[2020-10-02].https://arxiv.org/pdf/1902.10909.pdf.
[22] QIN L, CHE W, LI Y, et al.A stack-propagation framework with token-level intent detection for spoken language understanding[EB/OL].[2020-10-02].https://aclanthology.org/D19-1214.pdf.
[23] 高永兵, 李越超.微博中的社交意图识别与分类技术研究[J].内蒙古科技大学学报, 2020, 39(2):85-89. GAO Y B, LI Y C.Social intent recognitionand classification in Weibo[J].Journal of Inner Mongolia University of Science and Technology, 2020, 39(2):85-89.(in Chinese)
[24] MAO H H, MAJUMDER B P, MCAULEY J, et al.Improving neural story generation by targeted common sense grounding[EB/OL].[2020-10-02].https://arxiv.org/pdf/1908.09451v1.pdf.
[25] KESKAR N, MCCANN B, VARSHNEY L R, et al.CTRL:a conditional transformer language model for controllable generation[EB/OL].[2020-10-02].https://einstein.ai/presentations/ctrl.pdf.
[26] VASWANI A, NOAM S, NIKI P, et al.Attention is all you need[EB/OL].[2020-10-02].https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf.
[27] ZACHARY M Z, LUKE M K, SEBASTIAN G, et al.Encoder-agnostic adaptation for conditional language generation[EB/OL].[2020-10-02].https://arxiv.org/pdf/1908.06938.pdf.
[28] DATHATHRI S, ANDREA M, JANICE L, et al.Plug and play language models:a simple approach to controlled text generation[EB/OL].[2020-10-02].https://arxiv.org/pdf/1912.02164.pdf.
[29] ANGELA F, MIKE L, YANN D.Hierarchical neural story generation[EB/OL].[2020-10-02].https://aclanthology.org/P18-1082.pdf.

选择文件类型/文献管理软件名称

选择包含的内容