基于异构图卷积网络的小样本短文本分类方法

doi:10.19678/j.issn.1000-3428.0059920

摘要/Abstract

摘要： 为了解决小样本短文本分类中语义稀疏与过拟合问题，提出了一种基于异构图卷积网络的小样本短文本分类模型。首先利用 BTM 主题模型在短文本数据集上提取主题信息，并构建一个集成实体和主题信息的短文本异构信息网络，用于解决短文本语义稀疏问题；其次，建立基于随机去邻法和双重注意力机制的异构图卷积网络，用于提取短文本异构信息网络中的语义信息，其中，随机去邻法用于数据增强，缓解过拟合问题，双重注意力机制可以学习不同相邻节点的重要性和不同节点类型对当前节点的重要性。在三个短文本数据集上的实验结果表明，与 LSTM、Text GCN、HGAT 等基准模型相比，所构建的模型在每个类别只有十个标记样本时仍能达到最优性能。

Abstract: In order to solve the problem of semantic sparseness and overfitting in few-shot short text classification, this paper proposed a few-shot short text classification method based on heterogeneous graph convolutional network. Firstly, use the BTM to extract topic information from the short text datasets, and construct a short-text heterogeneous information network that can integrate entity and topic information to solve the problem of short text semantic sparseness. Secondly, construct a novel Heterogeneous Graph Convolutional Networks based on dual-level attention mechanism and the method of randomly reducing neighbors to extract semantic information from short text heterogeneous information network. The method of randomly reducing neighbors is used for data enhancement to alleviate the problem of over-fitting. The dual-level attention mechanism can learn the importance of different neighboring nodes and the importance of different node types to the current node. The experimental results on three short text datasets show that compared with the benchmark model such as LSTM, Text GCN, HGAT and so on, the model still achieves state-of-the-art performance when there are only ten labeled samples per class.

袁自勇 , 高曙 , 曹娇 , 陈良臣. 基于异构图卷积网络的小样本短文本分类方法[J]. 计算机工程, doi: 10.19678/j.issn.1000-3428.0059920.

Yuan Ziyong , Gao Shu , Cao Jiao , Chen Liangchen. Few-shot Short Text Classification Method Based On Heterogeneous Graph Convolutional Network[J]. Computer Engineering, doi: 10.19678/j.issn.1000-3428.0059920.

参考文献

[1] ALSMADI I, GAN K H. Review of short-text classification[J]. International Journal of Web Information Systems, 2019, 15(2): 155-182.
[2] PANG Wei. Short text classification via term graph [EB/OL]. [2020-10-11]. https://arxiv.org/abs/2001.10338.
[3] JOULIN A, GRAVE E, BOJANOWSKI P, et al. Bag of tricks for efficient text classification[C]//Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics. Valencia, Spain: ACL Press, 2017: 427-431.
[4] SHIMURA K, LI J, FUKUMOTO F. HFT-CNN: Learning hierarchical category structure for multi-label short text categorization[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium: ACL Press, 2018: 811-816.
[5] SINHA K, DONG Y, CHEUNG J C K, et al. A hierarchical neural attention-based text classifier[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium: ACL Press, 2018: 817-823.
[6] KIM Y. Convolutional Neural Networks for Sentence Classification[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014. Doha, Qatar: ACL Press, 2014: 1746-1751.
[7] ZHANG Xiang, ZHAO Junbo, LECUN Y. Character-level convolutional networks for text classification[C]// Advances in neural information processing systems. Montreal, Quebec, Canada: MIT Press, 2015: 649-657.
[8] WANG Jin, WANG Zhongyuan, ZHANG Dawei, et al. Combining knowledge with deep convolutional neural networks for short text classification[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence. Melbourne, Australia: IJCAI Press, 2017: 2915-2921.
[9] HOWARD J, RUDER S. Universal language model fine-tuning for text classification[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Melbourne, Australia: ACL Press, 2018:328-339.
[10] GENG Ruiying, LI Binhua, LI Yongbin, et al. Few-shot text classification with induction network[J]. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Hong Kong, China: ACL Press, 2019:3904-3913.
[11] JASON W, KAI Z. EDA: Easy data augmentation techniques for boosting performance on text classification Tasks[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Hong Kong, China: ACL Press, 2019: 6381-6387.
[12] YAO Liang, MAO Chengsheng, LUO Yuan. Graph convolutional networks for text classification[C]// Proceedings of the AAAI Conference on Artificial Intelligence.Honolulu, Hawaii, USA: AAAI Press, 2019, 33: 7370-7377.
[13] HU Linmei, YANG Tianchi, SHI Chuan, et al. Heterogeneous graph attention networks for semi-supervised short text classification[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Hong Kong, China: ACL Press, 2019: 4821–4830.
[14] YAN Xiaohu, GUO Jiafeng, LAN Yanyan, et al. A biterm topic model for short texts[C]//Proceedings of the 22nd international conference on World Wide Web. Rio de Janeiro, Brazil: ACM Press, 2013: 1445-1456.
[15] FERRAGINA P, SCAIELLA U. Tagme: On-the-fly annotation of short text fragments (by wikipedia entities)[C]//Proceedings of the 19th ACM international conference on Information and knowledge management. Toronto, Ontario, Canada：ACM Press, 2010: 1625-1628.
[16] RONG Yu, HUANG Wenbing, XU Tingyang, et al. DropEdge:Towards deep graph convolutional networks on node classification[EB/OL]. [2020-10-11]. https://openreview.net/forum?id=Hkx1qkrKPr
[17] LIU Pengfei, QIU Xipeng, HUANG Xuanjing. Recurrent neural network for text classification with multi-task learning[C]//Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence. New York, USA: IJCAI/AAAI Press, 2016: 2873–2879.
[18] TANG Jian, QU Meng, and MEI Qiaozhu. Pte: Predictive text embedding through large-scale heterogeneous text networks[C]//Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Sydney, NSW, Australia: ACM Press, 2015: 1165-1174.
[19] WANG Guoyin, LI Chunyuan, WANG Wenlin, et al. Joint embedding of words and labels for text classification[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Melbourne, Australia: ACL Press, 2018: 2321-2331.
[20] PHAN Xuan Hieu, NGUYEN Le Minh, HORIGUCHI Susumu. Learning to classify short and sparse text & web with hidden topics from large-scale data collections[C]// Proceedings of the 17th international conference on World Wide Web. Beijing, China: ACM Press, 2008: 91-100.
[21] BO P, LEE L. Seeing stars: exploiting class relationships for sentiment categorization with respect to rating scales[C]//Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics. University of Michigan, USA: ACL Press, 2005: 115-124.