摘要： 未登录词识别是汉语自动分词的难题之一。中文新词多为时间敏感词或者领域相关词汇,占据未登录词很大比例。针对新词识别问 题,首先总结了已有新词识别特征,然后引入了一个语言学特征。实验结果表明该特征可以提高新词识别率。

关键词: 中文分词, 新词识别, 支持向量机, 语素生产率

Abstract: One of the problems of Chinese word segmentation is the identification of OOV (out of vocabulary) words. Chinese new words are often time-sensitive or domain specific, which take large percentage of OOV words. As to new words identification (NWI) problem, this paper summarizes the previous work on NWI, and then introduces a novel feature. Final result shows that the performance of NWl can be improved by the feature.

中图分类号: 

• TP391.4