基于路径与词林编码的词语相似度计算方法

doi:10.19678/j.issn.1000-3428.0048357

摘要/Abstract

摘要： 现有词语相似度计算方法主要针对词语的路径结构进行计算,较少深入考虑词语的语义信息,导致计算结果不够准确。针对该问题,提出一种改进的词语语义相似度计算方法。将词语的词林编码与路径结构相结合,同时利用局部敏感哈希算法和海明距离计算词林编码之间的相似度。在MC和RG数据集上的实验结果表明,该方法可使皮尔逊相关系数分别达到0.897 4和0.866 8,较传统基于路径和深度的计算方法准确性更高。

关键词: 同义词, 路径结构, 编码, 词语相似度, 局部敏感哈希算法, 语义

Abstract: The existed similarity calculation methods of words are mainly focus on the path structure of words and consider less about the semantic information of words in detail,which lead to inaccurate calculation results.Aiming at this problem,an improved semantic word similarity calculation method is proposed.The CiLin coding and path structure are combined to calculate the similarity between CiLin coding,while using local sensitive Hash algorithm and Hamming distance.Experimental results show that the proposed method can make Pearson correlation coefficients achieve 0.897 4 and 0.866 8 on the MC data set and the RG data set respectively.It is more accurate than the traditional path-based and depth-based calculation methods.

Key words: synonym, path structure, coding, word similarity, local sensitive Hash algorithm, semantic

中图分类号:

TP391

王松松,高伟勋,徐逸凡. 基于路径与词林编码的词语相似度计算方法[J]. 计算机工程, 2018, 44(10): 160-167.

WANG Songsong,GAO Weixun,XU Yifan. Word Similarity Calculation Method Based on Path and CiLin Coding[J]. Computer Engineering, 2018, 44(10): 160-167.

参考文献

［1］王小林,陆骆勇,邰伟鹏.基于信息熵的新的词语相似度算法研究［J］.计算机技术与发展,2015,25(9):119-122.br/ ［2］吕亚伟,李芳,戴龙龙.基于LDA的中文词语相似度计算［J］.北京化工大学学报,2016,43(5):79-83.br/ ［3］NATALIA L,ALEKSEY A.Use of neighbor sentence co-occurrence to improve word semantic similarity detection［C］//Proceedings of International FRUCT Conference on Intelligence,Social Media and Web.Washington D.C.,USA:IEEE Press,2016:1-7.br/ ［4］FAN M,ZHANG Y,LI J.Word similarity computation based on HowNet［C］//Proceedings of the 12th International Conference on Fuzzy Systems and Knowledge Discovery.Washington D.C.,USA:IEEE Press,2015:1487-1492.br/ ［5］陈宏朝,李飞,朱新华,等.基于路径与深度的同义词词林词语相似度计算［J］.中文信息学报,2016,30(9):80-88.br/ ［6］JONES D J,MANSINGH G.Not just dissimilar,but opposite:an algorithm for measuring similarity and oppositeness between words［C］//Proceedings of International Conference on Data Science and Engineering.Washington D.C.,USA:IEEE Press,2016:1-6.br/ ［7］MILLER G A,FELLBAUM C.Semantic network of English［M］//LEVIN B.Lexical and Conceptual Semantics.Amsterdam,Holland:Elsevier Science Publishers,1991.br/ ［8］BAKER C.The Berkeley FrameNet project［C］//Proceedings of COLING-ACL’98.Washington D.C.,USA:IEEE Press,1998:86-90.br/ ［9］RICHARDSON S,DOLAN W B.MindNet:acquiring and structuring semantic information from text［C］//Proceedings of COLING-ACL’98.Washington D.C.,USA:IEEE Press,1998:1098-1102.br/ ［10］梅家驹.同义词词林［M］.上海:上海辞书出版社,1983.br/ ［11］张俐,李晶皎,胡明涵,等.中文WordNet的研究及实现［J］.东北大学学报(自然科学版),2003,24(4):327-329.br/ ［12］HUANG C R,HSIEH S K,HONG J F,et al.Chinese Wordnet:design,implementation,and application of an infrastructure for cross-lingual knowledge processing［J］.Journal of Chinese Information Processing,2010,24(2):14-23.br/ ［13］ADHIKARI A,SINGH S,DUTTA A,et al.A novel information theoretic approach for finding semantic similarity in WordNet［C］//Proceedings of TENCON’15.Washington D.C.,USA:IEEE Press,2015:1-6.br/ ［14］AOUICHA M B,TAIEB M A H.G2WS:gloss-based WordNet and Wiktionary semantic similarity measure［C］//Proceedings of the 12th International Conference of Computer Systems and Applications.Washington D.C.,USA:IEEE Press,2015:1-7.br/ ［15］ZHAO C,CAI A.The similarity calculation of concept names［C］//Proceedings of the 2nd International Conference on Cloud Computing and Internet of Things.Washington D.C.,USA:IEEE Press,2016:93-96.br/ ［16］田久乐,赵蔚.基于同义词词林的词语相似度计算方法［J］.吉林大学学报(信息科学版),2010,28(6):602-608.br/ ［17］ZHANG P,ZHANG Z,ZHANG W.An approach of semantic similarity by combining HowNet and CiLin［C］//Proceedings of IEEE International Conference on Green Computing and Communications.Washington D.C.,USA:IEEE Press,2013:1638-1643.br/ ［18］CHI T,WANG H,LIU L,et al.Text similarity calculation method based on ontology model［C］//Proceedings of 2014 International Conference on Cloud Computing and Internet of Things.Washington D.C.,USA:IEEE Press,2014:213-217.br/ ［19］XU L,SUN S,WANG Q.Text similarity algorithm based on semantic vector space model［C］//Proceedings of the 15th IEEE/ACIS International Conference on Computer and Information Science.Washington D.C.,USA:IEEE Press,2016:1-4.br/ ［20］HAR-PELED S,INDYK P,MOTWANI R.Approximate nearest neighbor:towards removing the curse of dimensionality［J］.Theory of Computing,2012,8(1):321-350.br/ ［21］WU Z,PALMER M.Verbs semantics and lexical selection［C］//Proceedings of the 32nd Annual Meeting on Association for Computational Linguistics.Stroudsburg,USA:Association for Computational Linguistics,1994:133-138.br/ ［22］HAO D,ZUO W,PENG T.An approach for calculating semantic similarity between words using WordNet［C］//Proceedings of the 2nd International Conference on Digital Manufacturing and Automation.Washington D.C.,USA:IEEE Press,2011:177-180.br/ ［23］LIU X,ZHOU Y,ZHENG R.Measuring semantic similarity in WordNet［C］//Proceedings of the 6th International Conference on Machine Learning and Cybernetics.Washington D.C.,USA:IEEE Press,2007:3431-3435.br/ ［24］张沪寅,刘道波,温春艳.基于《知网》的词语语义相似度改进算法研究［J］.计算机工程,2015,41(2):151-156.br/ ［25］MILLER G A,CHARLES W G.Contextual correlates of semantic similarity［J］.Language and Cognitive Processes,1991,6(1):1-28.br/ ［26］RUBENSTEIN H,GOODENOUGH J B.Contextual correlates of synonymy［J］.Communications of the ACM,1965,8(10):627-633.br/

[1]	段丹丹, 唐加山, 温勇, 袁克海. 基于BERT模型的中文短文本分类算法[J]. 计算机工程, 2021, 47(1): 79-86.
[2]	陈文杰. 一种融合主题特征的自适应知识表示方法[J]. 计算机工程, 2021, 47(1): 87-93,100.
[3]	蒋楚, 王永杰. 一种基于表达式树的gadget语义分析技术[J]. 计算机工程, 2021, 47(1): 109-116.
[4]	张国令, 王晓丹, 李睿, 来杰, 向前. 基于栈式降噪稀疏自编码器的极限学习机[J]. 计算机工程, 2020, 46(9): 61-67.
[5]	李俊, 吕学强. 融合BERT语义加权与网络图的关键词抽取方法[J]. 计算机工程, 2020, 46(9): 89-94.
[6]	芮雄丽, 曹雪虹. 基于叠加编码的协作传输与性能分析[J]. 计算机工程, 2020, 46(9): 172-177.
[7]	谭镭, 孙怀江. SKASNet:用于语义分割的轻量级卷积神经网络[J]. 计算机工程, 2020, 46(9): 261-267.
[8]	方瑞, 于俊洋, 董李锋. 基于特征矩阵构造与BP神经网络的垃圾文本过滤模型[J]. 计算机工程, 2020, 46(8): 271-276.
[9]	张显炀, 朱晓宇, 林浩申, 刘刚, 安喜彬. 基于高斯混合-变分自编码器的轨迹预测算法[J]. 计算机工程, 2020, 46(7): 50-57.
[10]	张志敏, 柴变芳, 李文斌. 基于稀疏自编码器的属性网络嵌入算法[J]. 计算机工程, 2020, 46(7): 98-103,109.
[11]	张腾飞, 周书仁, 彭建. 基于双孪生网络的自适应选择跟踪系统[J]. 计算机工程, 2020, 46(6): 103-107.
[12]	罗彬珅, 刘利民, 董健, 刘璟麒. 基于SAE-GA-SVM模型的雷达新型干扰识别[J]. 计算机工程, 2020, 46(6): 281-287.
[13]	陈恒, 韩雨婷, 李冠宇, 王京徽. 基于多级关系路径语义组合的关系推理算法[J]. 计算机工程, 2020, 46(5): 109-114.
[14]	王众, 韩益亮. 基于Niederreiter密码体制的抗量子签密方案[J]. 计算机工程, 2020, 46(5): 193-199.
[15]	马震环, 高洪举, 雷涛. 基于增强特征融合解码器的语义分割算法[J]. 计算机工程, 2020, 46(5): 254-258,266.