基于自然语言处理的算法知识名称发现

doi:10.3969/j.issn.1000-3428.2014.12.023

计算机工程 ›› 2014, Vol. 40 ›› Issue (12): 126-131.doi: 10.3969/j.issn.1000-3428.2014.12.023

基于自然语言处理的算法知识名称发现

朱国进,郑宁

东华大学计算机科学与技术学院,上海 201620

收稿日期:2013-12-20 修回日期:2014-02-27 出版日期:2014-12-15 发布日期:2015-01-16
作者简介:朱国进(1958-),男,副教授,主研方向:知识发现,语义网络,自然语言处理;郑宁,硕士研究生。
基金项目:
国家自然科学基金资助项目(60973121)

Discovering Terms of Algorithmic Knowledge Based on Natural Language Processing

ZHU Guojin,ZHENG Ning

School of Computer Science and Technology,Donghua University,Shanghai 201620,China

Received:2013-12-20 Revised:2014-02-27 Online:2014-12-15 Published:2015-01-16

摘要/Abstract

摘要： 网络中的很多程序资源在知识概念上有内在的联系,却没有超链接将它们连接在一起。将网络程序资源中的算法知识名称获取出来,组织成一个算法知识专家库文件,用于识别程序设计资源所含的知识点,即可将程序设计资源按知识点相互联系。为了自动获取程序资源中的算法知识名称,提出一种基于自然语言处理的算法知识名称发现方法。通过发现含有算法知识名称语句的字符串模式,从程序资源中提取可能含算法知识名称的字符串,从中找出最有可能出现在算法知识名称中的分词,并根据这些分词获取算法知识名称。实验结果表明,与原有人工整理出的算法知识名称集合相比,该方法新增了11.2%的算法知识点和13.6%的算法知识名称。

关键词: 知识发现, 模式发现, 自然语言处理, 算法知识名称, 中文分词, 词性标注

Abstract: There are many programming resources on the Internet.Although these programming resources have internal relations,there are often no hyperlinks connecting them.Getting the terms of algorithmic knowledge,organizing the terms to an expert file,which is used for recognizing the knowledge in the programming resources,the programming resources can be connected by the knowledge.To get the terms of algorithmic knowledge,this paper proposes a method to discover terms of algorithmic knowledge based on natural language processing.This method consists of discovering the patterns of strings which contain terms of algorithmic knowledge,extracting from programming resources that probably contain terms of algorithmic knowledge according to the discovered patterns,finding the word segmentation most likely appearing in the terms of algorithmic knowledge,and fetching the terms of algorithmic knowledge according to the word segmentation.This method increases 11.2% algorithmic knowledge and 13.6% terms of algorithmic knowledge in comparison with the manual collection of terms of algorithmic knowledge which is obtained by previous work.

Key words: knowledge discovery, pattern discovery, natural language processing, terms of algorithmic knowledge, Chinese word segmentation, part-of-speech tagging

中图分类号:

TP391.1

朱国进,郑宁. 基于自然语言处理的算法知识名称发现[J]. 计算机工程, 2014, 40(12): 126-131.

ZHU Guojin,ZHENG Ning. Discovering Terms of Algorithmic Knowledge Based on Natural Language Processing[J]. Computer Engineering, 2014, 40(12): 126-131.

参考文献

[1] Zhu Guojin,Fu Lichao.Automatic Organization of Programming Resources on the Web[M]//Jin D,Lin S.Advances in Computer Science and Information Engineering.Berlin,Germany:Springer,2012.
[2] Zhu Guojin.Automatic Discovery of Solutions to Programming Problems on the Web[C]//Proceedings of IEEE International Conference on Information Science and Technology.Phuket,Thailand:[s.n.],2013.
[3] Han Jiawei,Cheng Hong,Xin Dong,et al.Frequent Pattern Mining:Current Status and Future Directions[J].Data Mining and Knowledge Discovery,2007,15(1):55-86.
[4] Yuan Junsong,Wu Ying,Yang Ming.Discovery of Collocation Patterns:From Visual Words to Visual Phrases[C]//Proceedings of the 25th IEEE Conference on Computer Vision and Pattern Recognition.[S.l.]:IEEE Press,2007:1-8.
[5] Jiao Hui,Liu Qian,Jia Huibo.Chinese Keyword Extraction Based on N-gram and Word Cooccurrence[C]//Pro-ceedings of International Conference on Computational Intelligence and Security Workshops.[S.l.]:IEEE Press,2007:152-155.
[6] Resnik P.Semantic Similarity in a Taxonomy:An Information-based Measure and Its Application to Problems of Ambiguity in Natural Language[J].Journal of Artificial Intelligence Research,1998,11:95-130.
[7] Collobert R,Weston J,Bottou L,et al.Natural Language Processing (Almost) from Scratch[J].Journal of Machine Learning Research,2011,12:2493-2537.
[8] Maccartney B.Stanford Parser[EB/OL].(2013-06-20).http://nlp.stanford.edu /software/lex-parser.shtml.
[9] Sproat R,Emerson T.The First International Chinese Word Segmentation Bakeoff[C]//Proceedings of the 2nd SIGHAN Workshop on Chinese Language Processing.Sapporo,Japan:[s.n.],2003:133-143.
[10] Li Juanzi,Fan Qi’na,Zhang Kuo.Keyword Extraction Based on TF/IDF for Chinese News Document[J].Wuhan University Journal of Natural Sciences:English Vision,2007,12(5):917-921.
[11] Rajaraman A,Ullman J D.Mining of Massive Datasets[M].Cambridge,UK:Cambridge University Press,2011.

[1]	李宇霞, 孙永奇, 闫茹, 朱卫国. 基于CNN图像识别与语义可靠性的路径搜索方法[J]. 计算机工程, 2021, 47(1): 255-263,274.
[2]	韩虎, 赵启涛, 孙天岳, 刘国利. 面向社交媒体评论的上下文语境讽刺检测模型[J]. 计算机工程, 2021, 47(1): 66-71.
[3]	丁辰晖, 夏鸿斌, 刘渊. 融合知识图谱与注意力机制的短文本分类模型[J]. 计算机工程, 2021, 47(1): 94-100.
[4]	李冠宇, 张鹏飞, 贾彩燕. 一种注意力增强的自然语言推理模型[J]. 计算机工程, 2020, 46(7): 91-97.
[5]	冯读娟, 杨璐, 严建峰. 基于双编码器结构的文本自动摘要研究[J]. 计算机工程, 2020, 46(6): 60-64.
[6]	段大高, 梁少虎, 赵振东, 韩忠明. 基于自注意力机制的中文标点符号预测模型[J]. 计算机工程, 2020, 46(5): 291-297.
[7]	唐素勤, 孙亚茹, 李志欣, 张灿龙. 基于强化学习的壮语词性标注[J]. 计算机工程, 2020, 46(4): 309-315.
[8]	彭祝亮, 刘博文, 范程岸, 王杰, 肖明, 廖泽恩. 基于BLSTM与方面注意力模块的情感分类方法[J]. 计算机工程, 2020, 46(3): 60-65,72.
[9]	张海军, 陈映辉. 基于类图像处理与向量化的大数据脚本攻击智能检测[J]. 计算机工程, 2020, 46(3): 129-137,143.
[10]	王芝辉, 王晓东. 基于神经网络的文本分类方法研究[J]. 计算机工程, 2020, 46(3): 11-17.
[11]	李自荐, 迟呈英, 战学刚. 基于Bi-LSTM与CRF的泰语句子切分模型[J]. 计算机工程, 2020, 46(10): 294-300.
[12]	李抒音, 刘洋. 权重模糊粗糙集的分类规则挖掘算法[J]. 计算机工程, 2019, 45(9): 211-215.
[13]	蔡林森,彭超,陈思远,郭兰英. 基于多样化特征卷积神经网络的情感分析[J]. 计算机工程, 2019, 45(4): 169-174,180.
[14]	陈斌,周勇,刘兵. 基于卷积双向长短期记忆网络的事件触发词抽取[J]. 计算机工程, 2019, 45(1): 153-158.
[15]	张祯,樊兴悦,郭禹田,吴国华. 基于Topic Signature的动态文摘更新方法[J]. 计算机工程, 2018, 44(6): 169-175.

基于自然语言处理的算法知识名称发现

Discovering Terms of Algorithmic Knowledge Based on Natural Language Processing

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

Metrics

本文评价

推荐阅读 10