Similarity Computation of Patent Phrases Based on Knowledge Injection Prompt Learning

doi:10.19678/j.issn.1000-3428.0067595

Abstract

Abstract:

A patent is a legal right conferred to inventors to protect their inventions for a limited time, and it plays a crucial role in present-day social activities. Existing research has not optimized the adaptation of patent similarity data, which has negatively affected matching patent phrase similarity. Previous research has shown that in low-resource scenarios, prompt learning uses text fragments (i.e., templates) as input, transforming the classification problem into a mask language modeling problem; here, a key step is to construct a projection between the label space and label word space. This study presents a knowledge-based prompt learning method and applies it to the similarity matching of patent phrases. To solve the problem of insufficient information related to patent phrases, this study uses similarity label information in patent phrases and knowledge to enhance the patent phrases and label information. This study first establishes the relationship between patent phrases and external knowledge using entity-linking technology. The study then designs a neighborhood information filtering mechanism based on the degree of entity influence to expand the problem of insufficient patent phrase information. Finally, based on the effects of different types of external knowledge on the similarity calculation of patent phrases, the study generates a variety of enhanced prompt text applied to patent phrases. Experimental results show that the Pearson Correlation Coefficient (PCC) and Spearman Rank Correlation (SRC) of the proposed method are increased by 6.8% and 5.7%, respectively, as compared with the suboptimal method.

Key words: patent phrase, similarity computation, knowledge injection, prompt learning, prompt text

摘要：

专利是授予发明者在一定时期内保护其发明的法定权利, 在当今的社会活动中发挥着重要作用。然而现有研究并未针对专利相似度数据进行适配优化, 导致其应用在专利短语相似度匹配任务中效果不佳。已有研究表明, 在低资源的场景下, 提示学习将文本片段(模板)作为输入, 将分类问题转换为掩码语言建模问题, 其关键的一步是在标签空间和标签词空间之间构造一个投影。提出一种基于知识注入的提示学习方法, 将其应用于专利短语相似度匹配计算任务。为解决专利短语信息不足的问题, 利用专利短语中的相似度标签信息, 使用知识增强专利短语与标签信息。首先通过实体链接技术建立专利短语与外部知识的关联关系; 然后设计一种基于实体影响度的邻域信息过滤机制, 用于缓解专利短语信息不足的问题; 最后考虑不同外部知识对专利短语相似度计算的影响, 设计应用于专利短语的多种增强提示文本。实验结果表明, 该方法的Pearson相关系数(PCC)和Spearman相关系数(SRC)相较次优对比方法分别提升6.8%和5.7%。

关键词: 专利短语, 相似度计算, 知识注入, 提示学习, 提示文本

Yuanfei DENG, Jiawei LI, Yuncheng JIANG. Similarity Computation of Patent Phrases Based on Knowledge Injection Prompt Learning[J]. Computer Engineering, 2024, 50(4): 294-302.

邓远飞, 李加伟, 蒋运承. 基于知识注入提示学习的专利短语相似度计算[J]. 计算机工程, 2024, 50(4): 294-302.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0067595

http://www.ecice06.com/EN/Y2024/V50/I4/294

Figures/Tables 7

References 36

1	刘小蝶, 朱筠, 晋耀红. 中文专利中有标记并列结构的自动识别研究. 计算机工程, 2018, 44(6): 162-168, 175. URL
	LIU X D, ZHU Y, JIN Y H. Research on automatic identification of marked parallel structures in Chinese patent. Computer Engineering, 2018, 44(6): 162-168, 175. URL
2	SOUSA M J, JAMIL G, WALTER C E, et al. Big data analytics on patents for innovation public policies. Expert Systems, 2023, 40(1): e12673. doi: 10.1111/exsy.12673
3	罗艺雄, 吕学强, 游新冬. 融合多特征的专利功效短语识别. 中文信息学报, 2022, 36(12): 139- 148. URL
	LUO Y X, LÜ X Q, YOU X D. Patent efficacy phrase recognition based on multiple features. Journal of Chinese Information Processing, 2022, 36(12): 139- 148. URL
4	殷亚珏, 高晓雅, 王晶晶, 等. 基于多视角注意力机制的专利匹配方法. 中文信息学报, 2022, 36(7): 106- 113. URL
	YIN Y J, GAO X Y, WANG J J, et al. Patent matching with multi-view attentive network. Journal of Chinese Information Processing, 2022, 36(7): 106- 113. URL
5	KIM J, YOON J, PARK E, et al. Patent document clustering with deep embeddings. Scientometrics, 2020, 123(2): 563- 577. doi: 10.1007/s11192-020-03396-7
6	MASE H, MATSUBAYASHI T, OGAWA Y, et al. Proposal of two-stage patent retrieval method considering the claim structure. ACM Transactions on Asian Language Information Processing, 2005, 4(2): 190- 206. doi: 10.1145/1105696.1105702
7	DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of Conference of the North American Chapter of the Association for Computational Linguistics. [S.l.]: Association for Computational Linguistics, 2019: 4171-4186.
8	BINZ M, SCHULZ E. Using cognitive psychology to understand GPT-3. Proceedings of the National Academy of Sciences of the United States of America, 2023, 120(6): e2218.
9	ALT C, GABRYSZAK A, HENNIG L. TACRED revisited: a thorough evaluation of the TACRED relation extraction task[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. [S.l.]: Association for Computational Linguistics, 2020: 1558-1569.
10	张宁豫, 谢辛, 陈想, 等. 基于知识协同微调的低资源知识图谱补全方法. 软件学报, 2022, 33(10): 3531- 3545. URL
	ZHANG N Y, XIE X, CHEN X, et al. Knowledge collaborative fine-tuning for low-resource knowledge graph completion. Journal of Software, 2022, 33(10): 3531- 3545. URL
11	BHARGAVA P, NG V. Commonsense knowledge reasoning and generation with pre-trained language models: a survey. Proceedings of the AAAI Conference on Artificial Intelligence, 2022, 36(11): 12317- 12325. doi: 10.1609/aaai.v36i11.21496
12	SINGH G V, FIRDAUS M, MISHRA S, et al. Knowing what to say: towards knowledge grounded code-mixed response generation for open-domain conversations. Knowledge-Based Systems, 2022, 249, 108900. doi: 10.1016/j.knosys.2022.108900
13	HU S D, DING N, WANG H D, et al. Knowledgeable prompt-tuning: incorporating knowledge into prompt verbalizer for text classification[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. [S.l.]: Association for Computational Linguistics, 2022: 2225-2240.
14	YE H B, ZHANG N Y, DENG S M, et al. Ontology-enhanced prompt-tuning for few-shot learning[C]//Proceedings of the ACM Web Conference. New York, USA: ACM Press, 2022: 778-787.
15	FUJII A. Enhancing patent retrieval by citation analysis[C]//Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, USA: ACM Press, 2007: 793-794.
16	KANG I S, NA S H, KIM J, et al. Cluster-based patent retrieval. Information Processing & Management, 2007, 43(5): 1173- 1182.
17	LI S B, HU J, CUI Y X, et al. DeepPatent: patent classification with convolutional neural networks and word embedding. Scientometrics, 2018, 117(2): 721- 744. doi: 10.1007/s11192-018-2905-5
18	FANG L T, ZHANG L, WU H, et al. Patent2Vec: multi-view representation learning on patent-graphs for patent classification. World Wide Web, 2021, 24(5): 1791- 1812. doi: 10.1007/s11280-021-00885-4
19	GUU K, LEE K, TUNG Z, et al. Retrieval augmented language model pre-training[C]//Proceedings of International Conference on Machine Learning. New York, USA: ACM Press, 2020: 3929-3938.
20	LIU B, LIN T, LI M. Enhancing aspect-category sentiment analysis via syntactic data augmentation and knowledge enhancement. Knowledge-Based Systems, 2023, 264, 110339. doi: 10.1016/j.knosys.2023.110339
21	INCITTI F, URLI F, SNIDARO L. Beyond word embeddings: a survey. Information Fusion, 2023, 89, 418- 436. doi: 10.1016/j.inffus.2022.08.024
22	陈彦桦, 李剑. 一种基于结构特征的树相似度计算方法. 计算机工程, 2018, 44(11): 197-201, 208. URL
	CHEN Y H, LI J. A tree similarity computation method based on structrue feature. Computer Engineering, 2018, 44(11): 197-201, 208. URL
23	ROGERS A, GARDNER M, AUGENSTEIN I. QA dataset explosion: a taxonomy of NLP resources for question answering and reading comprehension. ACM Computing Surveys, 2023, 55(10): 1- 45.
24	陆晓蕾, 倪斌. 基于预训练语言模型的BERT-CNN多层级专利分类研究. 中文信息学报, 2021, 35(11): 70- 79. URL
	LU X L, NI B. BERT-CNN: a hierarchical patent classifier based on pre-trained language model. Journal of Chinese Information Processing, 2021, 35(11): 70- 79. URL
25	亢晓勉, 宗成庆. 基于篇章结构多任务学习的神经机器翻译. 软件学报, 2022, 33(10): 3806- 3818. URL
	KANG X M, ZONG C Q. Neural machine translation based on multi-task learning of discourse structure. Journal of Software, 2022, 33(10): 3806- 3818. URL
26	LUND B D, WANG T. Chatting about ChatGPT: how may AI and GPT impact academia and libraries?. Library Hi Tech News, 2023, 40(3): 26- 29. doi: 10.1108/LHTN-01-2023-0009
27	SONG C Y, CAI F, WANG M R, et al. TaxonPrompt: Taxonomy-aware curriculum prompt learning for few-shot event classification. Knowledge-Based Systems, 2023, 264, 110290. doi: 10.1016/j.knosys.2023.110290
28	LIU X, JI K X, FU Y C, et al. P-tuning: prompt tuning can be comparable to fine-tuning across scales and tasks[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. [S.l.]: Association for Computational Linguistics, 2022: 61-68.
29	SCHICK T, SCHÜTZE H. Exploiting cloze-questions for few-shot text classification and natural language inference[C]//Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics. [S.l.]: Association for Computational Linguistics, 2021: 255-269.
30	LI X L, LIANG P. Prefix-tuning: optimizing continuous prompts for generation[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. [S.l.]: Association for Computational Linguistics, 2021: 4582-4597.
31	SUN T X, LIU X Y, QIU X P, et al. Paradigm shift in natural language processing. Machine Intelligence Research, 2022, 19(3): 169- 183. doi: 10.1007/s11633-022-1331-6
32	GROVER A, LESKOVEC J. node2vec: scalable feature learning for networks[C]//Proceedings of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM Press, 2016: 855-864.
33	MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality[EB/OL]. [2023-04-05]. https://arxiv.org/abs/1310.4546.
34	PENNINGTON J, SOCHER R, MANNING C. GloVe: global vectors for word representation[C]// Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing. [S.l.]: Association for Computational Linguistics, 2014: 1532-1543.
35	JOULIN A, GRAVE E, BOJANOWSKI P, et al. Bag of tricks for efficient text classification[C]// Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics. [S.l.]: Association for Computational Linguistics, 2017: 427-431.
36	REIMERS N, GUREVYCH I. Sentence-BERT: sentence embeddings using siamese BERT-networks[C]//Proceedings of 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. [S.l.]: Association for Computational Linguistics, 2019: 3982-3992.

[1]	Hongpeng LI, Bo MA, Yating YANG, Lei WANG, Zhen WANG, Xiao LI. Document-level Event Extraction Method Based on Slot Semantic Enhanced Prompt Learning [J]. Computer Engineering, 2023, 49(9): 23-31.
[2]	ZHANG Boxu, PU Zhi, CHENG Xi. Research on Uyghur Text Classification Based on Prompt Learning [J]. Computer Engineering, 2023, 49(6): 292-299,313.
[3]	LI Zhixin,LAN Danmei,ZHANG Canlong,TANG Suqin. Recognition Method of Microblogging Spam Comment Based on Co-Training [J]. Computer Engineering, 2018, 44(7): 212-218.
[4]	CHEN Yanhua,LI Jian. A Tree Similarity Computation Method Based on Structrue Feature [J]. Computer Engineering, 2018, 44(11): 197-201,208.
[5]	CHENG Xi,CHEN Jun. Collaborative Filtering Algorithm Based on MapReduce and Item Classification [J]. Computer Engineering, 2016, 42(7): 194-198.
[6]	CHENG Xiao-lin, XIONG Yan, LIU Qing-wen, LU Qi-wei. A Collaborative Filtering Method Based on Adaptive Local Fusion-parameter [J]. Computer Engineering, 2014, 40(1): 39-44.
[7]	DENG Shao-Wei, LUO Ze, LI Shu-Ren, YAN Bao-Beng. Scholar Recommendation System Based on Academic Relationship of Thesis Co-authors [J]. Computer Engineering, 2013, 39(2): 12-17.
[8]	FAN Hai-Xiong, LIU Fu-Xian, JIA Lu. Case Similarity Computation Method of Fusing Margin and Veto Power [J]. Computer Engineering, 2012, 38(12): 261-264.
[9]	QIN Hua-Yong, ZHANG Run-Mei. Application of Two Level Similarity Computation in Subjective Machine Marking [J]. Computer Engineering, 2012, 38(11): 274-276,280.
[10]	WANG Feng-Ling. Research of Chinese Report Link Recognition Based on Abbreviation Analysis [J]. Computer Engineering, 2011, 37(18): 164-166.
[11]	LIU Shuang-Yi, XU Long-Qin. Case Retrieval Based on Improved Covering Algorithm and Grey Relational Degree [J]. Computer Engineering, 2011, 37(13): 169-171.
[12]	ZHANG Fan, ZHONG Jin-Hong, HUANG Ling. Improved Method for Similarity Computation of Concepts in Domain Ontology [J]. Computer Engineering, 2010, 36(23): 66-68.
[13]	HUANG Wen-tao; XU Ling-yu; LI Yan; WU Zao-liang. Method of Multi-text Fusion and Extraction Based on Flexible Intervals [J]. Computer Engineering, 2007, 33(24): 217-219.

Please choose a citation manager

Content to export