摘要： 针对现有中文短文本分类算法通常存在特征稀疏、用词不规范和数据海量等问题，提出一种基于Transformer的双向编码器表示（BERT）的中文短文本分类算法，使用BERT预训练语言模型对短文本进行句子层面的特征向量表示，并将获得的特征向量输入Softmax回归模型进行训练与分类。实验结果表明，随着搜狐新闻文本数据量的增加，该算法在测试集上的整体F1值最高达到93%，相比基于TextCNN模型的短文本分类算法提升6个百分点，说明其能有效表示句子层面的语义信息，具有更好的中文短文本分类效果。

关键词: 中文短文本分类, 基于Transformer的双向编码器表示, Softmax回归模型, TextCNN模型, word2vec模型

Abstract: The existing Chinese short text classification algorithms are faced with sparse features,informal words and massive data.To address the problems,this paper proposes a Chinese short text classification algorithm based on the Bidirectional Encoder Representation from Transformer(BERT) model.The algorithm uses BERT pre-training language model to perform eigenvector representation of short text on the sentence level,and then the obtained eigenvector is input into the Softmax regression model for training and classification.Experimental results show that with the growth of data from Sohu news,the overall F1 value of the proposed algorithm on the test dataset is up to 93%,which is 6 percentage points higher than that of the TextCNN-based short text classification algorithm.The result demonstrates that the proposed algorithm performs better in semantic information representation at the sentence level,and in the classification of Chinese short texts.

Key words: Chinese short text classification, Bidirectional Encoder Representation from Transformer(BERT), Softmax regression model, TextCNN model, word2vec model

中图分类号: 

• TP391