Word Similarity Calculation Method Based on Path and CiLin Coding

doi:10.19678/j.issn.1000-3428.0048357

Abstract

Abstract: The existed similarity calculation methods of words are mainly focus on the path structure of words and consider less about the semantic information of words in detail,which lead to inaccurate calculation results.Aiming at this problem,an improved semantic word similarity calculation method is proposed.The CiLin coding and path structure are combined to calculate the similarity between CiLin coding,while using local sensitive Hash algorithm and Hamming distance.Experimental results show that the proposed method can make Pearson correlation coefficients achieve 0.897 4 and 0.866 8 on the MC data set and the RG data set respectively.It is more accurate than the traditional path-based and depth-based calculation methods.

Key words: synonym, path structure, coding, word similarity, local sensitive Hash algorithm, semantic

摘要： 现有词语相似度计算方法主要针对词语的路径结构进行计算,较少深入考虑词语的语义信息,导致计算结果不够准确。针对该问题,提出一种改进的词语语义相似度计算方法。将词语的词林编码与路径结构相结合,同时利用局部敏感哈希算法和海明距离计算词林编码之间的相似度。在MC和RG数据集上的实验结果表明,该方法可使皮尔逊相关系数分别达到0.897 4和0.866 8,较传统基于路径和深度的计算方法准确性更高。

关键词: 同义词, 路径结构, 编码, 词语相似度, 局部敏感哈希算法, 语义

CLC Number:

TP391

WANG Songsong,GAO Weixun,XU Yifan. Word Similarity Calculation Method Based on Path and CiLin Coding[J]. Computer Engineering, 2018, 44(10): 160-167.

王松松,高伟勋,徐逸凡. 基于路径与词林编码的词语相似度计算方法[J]. 计算机工程, 2018, 44(10): 160-167.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0048357

http://www.ecice06.com/EN/Y2018/V44/I10/160

References

［1］王小林,陆骆勇,邰伟鹏.基于信息熵的新的词语相似度算法研究［J］.计算机技术与发展,2015,25(9):119-122.br/ ［2］吕亚伟,李芳,戴龙龙.基于LDA的中文词语相似度计算［J］.北京化工大学学报,2016,43(5):79-83.br/ ［3］NATALIA L,ALEKSEY A.Use of neighbor sentence co-occurrence to improve word semantic similarity detection［C］//Proceedings of International FRUCT Conference on Intelligence,Social Media and Web.Washington D.C.,USA:IEEE Press,2016:1-7.br/ ［4］FAN M,ZHANG Y,LI J.Word similarity computation based on HowNet［C］//Proceedings of the 12th International Conference on Fuzzy Systems and Knowledge Discovery.Washington D.C.,USA:IEEE Press,2015:1487-1492.br/ ［5］陈宏朝,李飞,朱新华,等.基于路径与深度的同义词词林词语相似度计算［J］.中文信息学报,2016,30(9):80-88.br/ ［6］JONES D J,MANSINGH G.Not just dissimilar,but opposite:an algorithm for measuring similarity and oppositeness between words［C］//Proceedings of International Conference on Data Science and Engineering.Washington D.C.,USA:IEEE Press,2016:1-6.br/ ［7］MILLER G A,FELLBAUM C.Semantic network of English［M］//LEVIN B.Lexical and Conceptual Semantics.Amsterdam,Holland:Elsevier Science Publishers,1991.br/ ［8］BAKER C.The Berkeley FrameNet project［C］//Proceedings of COLING-ACL’98.Washington D.C.,USA:IEEE Press,1998:86-90.br/ ［9］RICHARDSON S,DOLAN W B.MindNet:acquiring and structuring semantic information from text［C］//Proceedings of COLING-ACL’98.Washington D.C.,USA:IEEE Press,1998:1098-1102.br/ ［10］梅家驹.同义词词林［M］.上海:上海辞书出版社,1983.br/ ［11］张俐,李晶皎,胡明涵,等.中文WordNet的研究及实现［J］.东北大学学报(自然科学版),2003,24(4):327-329.br/ ［12］HUANG C R,HSIEH S K,HONG J F,et al.Chinese Wordnet:design,implementation,and application of an infrastructure for cross-lingual knowledge processing［J］.Journal of Chinese Information Processing,2010,24(2):14-23.br/ ［13］ADHIKARI A,SINGH S,DUTTA A,et al.A novel information theoretic approach for finding semantic similarity in WordNet［C］//Proceedings of TENCON’15.Washington D.C.,USA:IEEE Press,2015:1-6.br/ ［14］AOUICHA M B,TAIEB M A H.G2WS:gloss-based WordNet and Wiktionary semantic similarity measure［C］//Proceedings of the 12th International Conference of Computer Systems and Applications.Washington D.C.,USA:IEEE Press,2015:1-7.br/ ［15］ZHAO C,CAI A.The similarity calculation of concept names［C］//Proceedings of the 2nd International Conference on Cloud Computing and Internet of Things.Washington D.C.,USA:IEEE Press,2016:93-96.br/ ［16］田久乐,赵蔚.基于同义词词林的词语相似度计算方法［J］.吉林大学学报(信息科学版),2010,28(6):602-608.br/ ［17］ZHANG P,ZHANG Z,ZHANG W.An approach of semantic similarity by combining HowNet and CiLin［C］//Proceedings of IEEE International Conference on Green Computing and Communications.Washington D.C.,USA:IEEE Press,2013:1638-1643.br/ ［18］CHI T,WANG H,LIU L,et al.Text similarity calculation method based on ontology model［C］//Proceedings of 2014 International Conference on Cloud Computing and Internet of Things.Washington D.C.,USA:IEEE Press,2014:213-217.br/ ［19］XU L,SUN S,WANG Q.Text similarity algorithm based on semantic vector space model［C］//Proceedings of the 15th IEEE/ACIS International Conference on Computer and Information Science.Washington D.C.,USA:IEEE Press,2016:1-4.br/ ［20］HAR-PELED S,INDYK P,MOTWANI R.Approximate nearest neighbor:towards removing the curse of dimensionality［J］.Theory of Computing,2012,8(1):321-350.br/ ［21］WU Z,PALMER M.Verbs semantics and lexical selection［C］//Proceedings of the 32nd Annual Meeting on Association for Computational Linguistics.Stroudsburg,USA:Association for Computational Linguistics,1994:133-138.br/ ［22］HAO D,ZUO W,PENG T.An approach for calculating semantic similarity between words using WordNet［C］//Proceedings of the 2nd International Conference on Digital Manufacturing and Automation.Washington D.C.,USA:IEEE Press,2011:177-180.br/ ［23］LIU X,ZHOU Y,ZHENG R.Measuring semantic similarity in WordNet［C］//Proceedings of the 6th International Conference on Machine Learning and Cybernetics.Washington D.C.,USA:IEEE Press,2007:3431-3435.br/ ［24］张沪寅,刘道波,温春艳.基于《知网》的词语语义相似度改进算法研究［J］.计算机工程,2015,41(2):151-156.br/ ［25］MILLER G A,CHARLES W G.Contextual correlates of semantic similarity［J］.Language and Cognitive Processes,1991,6(1):1-28.br/ ［26］RUBENSTEIN H,GOODENOUGH J B.Contextual correlates of synonymy［J］.Communications of the ACM,1965,8(10):627-633.br/

[1]	Chunbo XU, Juan YAN, Huibin YANG, Bo WANG, Han WU. Visual SLAM Algorithm Based on Target Detection and Semantic Segmentation [J]. Computer Engineering, 2023, 49(8): 199-206, 214.
[2]	Xiaodan CUI, Dawei LIU, Yifan LIU, Zhibin ZHAO, Yougui REN, Yongming YAN. Research and Implementation of Key Frame Summarization Model for News Short Video [J]. Computer Engineering, 2023, 49(8): 182-189.
[3]	Yiyin GU, Hongkui WANG, Haibing YIN. Fast Dependent Quantization Algorithm Based on Context Adaptive Threshold Pruning [J]. Computer Engineering, 2023, 49(7): 143-149.
[4]	Jianrui XI, Hongmei TANG, Chunyang LIANG, Xin LIU. Point Cloud Object Reconstruction Based on Improved Implicit Function [J]. Computer Engineering, 2023, 49(7): 214-222.
[5]	Kuan WANG, Shibin XUAN, Xuedong HE, Ziwei LI, Jiaxiang LI. Human Pose Estimation Method Based on Cross Attention Transformer [J]. Computer Engineering, 2023, 49(7): 223-231.
[6]	FU Jiahao, YANG Jiayi, LI Aiguo. High-Utility Semantic Trajectory Pattern Mining for Security System [J]. Computer Engineering, 2023, 49(6): 62-70.
[7]	CHEN Ming, LIU Rong, ZHANG Ye. Chinese Medical Entity Recognition Based on Multiple Attention Mechanism [J]. Computer Engineering, 2023, 49(6): 314-320.
[8]	ZHAO Hong, CHEN Zhiwen, GUO Lan, AN Dong. Video Content Caption Generation Based on ViT and Semantic Guidance [J]. Computer Engineering, 2023, 49(5): 247-254.
[9]	YE Qi, ZHANG Yiqian, RUAN Tong, DU Wen. Quality Verification Method for Knowledge Graph Based on Semantic and Structural Trustworthiness [J]. Computer Engineering, 2023, 49(5): 48-55.
[10]	CHEN Wenxuan, ZENG Bi, GUO Zhixing. Fall Detection Method Integrating Multi-Feature and Semantic Graph Convolution Network [J]. Computer Engineering, 2023, 49(5): 277-285,294.
[11]	HENG Hongjun, MIAO Jing. Joint Extraction of Binary Tagging Entity Relation for Enhanced Semantic and Syntactic Information [J]. Computer Engineering, 2023, 49(4): 77-84.
[12]	PANG Tao, ZHANG Xuemin, YAO Yazhou, GAO Mingke. Optical Remote Sensing Image Building Change Detection Based on Feature Enhancement [J]. Computer Engineering, 2023, 49(4): 182-187.
[13]	BAI Junqing, HAN Boxun, ZHANG Fengxia. Deep Learning-Based UAV Image Semantic Segmentation Algorithm Research [J]. Computer Engineering, 2023, 49(4): 233-239.
[14]	MA Sugang, CHEN Qimei, HOU Zhiqiang, YANG Xiaobao, ZHANG Zixian. Semantic Segmentation Algorithm Based on Dense Connection and Feature Enhancement [J]. Computer Engineering, 2023, 49(3): 263-270.
[15]	SU Mingfang, HU Likun, HUANG Runhui. Semantic Segmentation Method for Outdoor Point Clouds Based on Contextual Attention [J]. Computer Engineering, 2023, 49(3): 248-256.

Please choose a citation manager

Content to export