摘要： 提出了一种基于PAT树型结构的高频字串提取的改进算法,并以此用来获得特定领域网页中的未登录词集合,利用基于语义距离的概念相似度计算公式来获得任一概念的相关概念,从而给出了用于特定领域搜索引擎的语义词典完整的自动构建方法。将生成的语义词典用于搜索引擎FlyingSender中。实验结果证明新的词典比原有的手工构建的词典分词效果要理想得多,而且提供了概念联想的功能。
关键词:
语义词典,
未登录词,
PAT树,
语义距离
Abstract: An improved algorithm based on PAT tree structure to extract high frequency strings appearing in the doc is presented. Through it the paper gains a set of words of specific domain not listed in the general dictionary, computes the similarity for each two concepts to obtain associative concepts based on semantic distance. Finally, it concludes a complete automatic method to construct a thesaurus of specific domain used by search engine. Experimental results prove new thesaurus with concept association functions makes better performance than the general one.
中图分类号:
张彦,邵志清. 具有概念联想功能的特定领域分词词典的自动构建[J]. , 2004, 30(20): 148-150.
ZHANG Yan,SHAO Zhiqing. Automatic Creation of Special Domain Word-segmentation Dictionary with Concept Association[J]. , 2004, 30(20): 148-150.