Graph Neural Network Enhancement Based on Personalized PageRank Higher Order Neighborhood Aggregation

doi:10.19678/j.issn.1000-3428.0068976

Abstract

Abstract:

The key idea behind Graph Neural Network (GNN) is to learn the information representation of a target node by aggregating neighborhood information through the topology of a graph; however, edges that are not relevant to a downstream task or nodes with limited neighbors may limit the representation of the neural network. Existing enhancement methods seldom focus on both structure and features simultaneously when enhancing graph data. Among them, existing local area enhancement methods use generative models to generate features through first-order neighborhoods and cannot obtain more relevant higher-order neighborhood information for nodes. To address this phenomenon, this study presents an effective data enhancement strategy. First, an edge prediction model is used to adjust the topology of a graph to improve the Signal-to-Noise Ratio (SNR) and facilitate the message transfer between nodes. Second, a Personalized PageRank (PPR) algorithm is used to aggregate the effective information in multiorder neighborhoods from a global perspective for global feature enhancement. Finally, the generative model is used to generate more features for local enhancement, which enriches node expression, especially for low-degree nodes. Experiments show that the accuracies of Graph Convolutional Network (GCN) and Graph Attention Network (GAT) models are improved by 3.1 and 1.3 percentage points on average, respectively, on the Cora, CiteSeer, and PubMed datasets with this data enhancement strategy. This result shows that performance improves to an extent when this strategy is applied to neural network architectures with different benchmark sets.

Key words: data augmentation, Personalized PageRank(PPR), generative model, neural network, global aggregation, multi-order neighborhood

摘要：

图神经网络(GNN)的关键思想是通过图的拓扑结构来聚合邻域信息学习目标节点的信息表征，当图中存在与下游任务无关的边，或者节点的邻居有限时，都会限制神经网络的表达。现有的增强方法很少从结构和特征两方面出发来同时增强图数据，其中现有的局域增强方法运用生成模型通过一阶邻域来生成特征，无法为节点获得更多相关高阶邻域信息。针对这种现象，提出一种有效的数据增强策略。首先运用边预测模型来调整图的拓扑结构，提高信噪比(SNR)，促进节点之间的消息传递；然后运用个性化PageRank(PPR)算法从全局角度聚合多阶邻域中的有效信息进行全局特征增强；最后运用生成模型来生成更多特征进行局域增强，丰富节点表达，尤其是低度节点。实验结果表明，在Cora、CiteSeer和PubMed数据集上，在图卷积网络(GCN)和图注意力网络(GAT)模型上运用该数据增强策略，在测试精度方面模型准确率平均提高3.1和1.3百分点，证明当应用于不同的基准集的各种神经网络架构时，该数据增强策略都能产生一定程度上的性能提升。

关键词: 数据增强, 个性化PageRank, 生成模型, 神经网络, 全局聚合, 多阶邻域

SHANG Yaming, WU Anbiao, YUAN Ye, WANG Yishu. Graph Neural Network Enhancement Based on Personalized PageRank Higher Order Neighborhood Aggregation[J]. Computer Engineering, 2025, 51(6): 38-48.

商雅名, 吴安彪, 袁野, 王一舒. 基于个性化PageRank高阶邻域聚合的图神经网络增强[J]. 计算机工程, 2025, 51(6): 38-48.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0068976

https://www.ecice06.com/EN/Y2025/V51/I6/38

Figures/Tables 10

Fig.1 Data augmentation framework

Fig.2 Topology change of the graph

Fig.3 PPR global aggregation process

Fig.4 classification performance using global enhancement on different models

Fig.5 Accuracy of GCN with different parameters

Fig.6 Accuracy of GCN with different thresholds

References 41

1	YUAN Y, WANG G R, WANG H X, et al. Efficient subgraph search over large uncertain graphs. Proceedings of the VLDB Endowment, 2011, 4(11): 876- 886. doi: 10.14778/3402707.3402726
2	ZHANG M, CHEN Y. Link prediction based on graph neural networks[C]//Proceedings of Annual Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2018: 5171-5181.
3	YUAN Y, CHEN L, WANG G R. Efficiently answering probability threshold-based shortest path queries over uncertain graphs[C]//Proceedings of the 15th International Conference on Database Systems for Advanced Applications. Berlin, Germany: Springer, 2010: 155-170.
4	YUAN Y, WANG G R, CHEN L, et al. Efficient subgraph similarity search on large probabilistic graph databases. Proceedings of the VLDB Endowment, 2012, 5(9): 800- 811. doi: 10.14778/2311906.2311908
5	SHORTEN C, KHOSHGOFTAAR T M. A survey on image data augmentation for deep learning. Journal of Big Data, 2019, 6(1): 60. doi: 10.1186/s40537-019-0197-0
6	KAFLE K, YOUSEFHUSSIEN M, KANAN C. Data augmentation for visual question answering[C]//Proceedings of the 10th International Conference on Natural Language Generation. Stroudsburg, USA: ACL, 2017: 198-202.
7	ZHANG X, LECUN Y. Character-level convolutional networks for text classification[C]//Proceedings of Annual Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2015: 1-9.
8	吴量, 张方方, 程超, 等. 基于双层数据增强的监督对比学习文本分类模型. 吉林大学学报(理学版), 2024, 62(1): 144- 151.
	WU L, ZHANG F F, CHENG C, et al. A text classification model of supervised contrastive learning based on double-layer data augmentation. Journal of Jilin University (Science Edition), 2024, 62(1): 144- 151.
9	CHEN D L, LIN Y K, LI W, et al. Measuring and relieving the over-smoothing problem for graph neural networks from the topological view. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(4): 3438- 3445. doi: 10.1609/aaai.v34i04.5747
10	ZHAO T, LIU Y, NEVES L, et al. Data augmentation for graph neural networks. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(12): 11015- 11023. doi: 10.1609/aaai.v35i12.17315
11	LIU S T, YING R, DONG H, et al. Local augmentation for graph neural networks[C]//Proceedings of the 39th International Conference on Machine Learning. Baltimore, USA: PMLR Press, 2022: 14054-14072.
12	ZHU D H, DAI X Y, CHEN J J. Pre-train and learn: preserving global information for graph neural networks. Journal of Computer Science and Technology, 2021, 36(6): 1420- 1430. doi: 10.1007/s11390-020-0142-x
13	RONG Y, HUANG W, XU T, et al. DropEdge: towards deep graph convolutional networks on node classification[C]//Proceedings of International Conference on Learning Representations. Addis Ababa, Ethiopia: ICLR Press, 2020: 1-17.
14	BRUNA J, ZAREMBA W, SZLAM A, et al. Spectral networks and locally connected networks on graphs[C]//Proceedings of International Conference on Learning Representations. Banff, Canada: ICLR Press, 2014: 1015-11023.
15	肖国庆, 李雪琪, 陈玥丹, 等. 大规模图神经网络研究综述. 计算机学报, 2024, 47(1): 148- 171.
	XIAO G Q, LI X Q, CHEN Y D, et al. A survey of large-scale graph neural networks. Chinese Journal of Computers, 2024, 47(1): 148- 171.
16	KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[C]//Proceedings of International Conference on Learning Representations. Toulon, France: ICLR Press, 2017: 1-14.
17	CHEN M, WEI Z W. Simple and deep graph convolutional networks[C]//Proceedings of the 37th International Conference on Machine Learning. [S. l. ]: ICML Press, 2020: 1725-1735.
18	VELICKOVIC P, CUCURULL G, CASANOVA A. Graph attention networks[C]//Proceedings of International Conference on Learning Representations. Vancouver, Canada: ICLR Press, 2018: 1-12.
19	缪昊洋, 高谭芮, 汤影. 基于生成模型的联邦学习隐私保护算法. 电子设计工程, 2023, 31(24): 81-84, 89.
	MIAO H Y, GAO T R, TANG Y. Federated learning privacy protection algorithm based on generative model. Electronic Design Engineering, 2023, 31(24): 81-84, 89.
20	杨盛春, 贾林祥. 神经网络内监督学习和无监督学习之比较. 徐州建筑职业技术学院学报, 2006, 6(3): 55- 58.
	YANG S C, JIA L X. Comparison between supervised learning and unsupervised learning in neural networks. Journal of Xuzhou Institute of Architectural Technology, 2006, 6(3): 55- 58.
21	KIPF T N, WELLING M. Variational graph auto-encoders[EB/OL]. [2024-09-10]. https://arxiv.org/abs/1611.07308.
22	SALHA G, HENNEQUIN R, VAZIRGIANNIS M. Keep it simple: graph autoencoders without graph convolutional networks[EB/OL]. [2024-09-10]. https://arxiv.org/abs/2006.03545.
23	XU D, RUAN C W, MOTWANI K, et al. Generative graph convolutional network for growing graphs[C]//Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Washington D. C., USA: IEEE Press, 2019: 3167-3171.
24	YANG C, ZHUANG P, SHI W, et al. Conditional structure generation through graph variational generative adversarial nets[C]//Proceedings of Annual Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2019: 1338-1349.
25	SIMONOVSKY M, KOMODAKIS N. GraphVAE: towards generation of small graphs using variational autoencoders[C]//Proceedings of International Conference on Advanced Nanomaterials and Nanodevices. Berlin, Germany: Springer, 2018: 412-422.
26	LIU Q, ALLAMANI M, BROCKSCHMIDT M, et al. Constrained graph variational autoencoders for molecule design[C]//Proceedings of Annual Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2018: 7806-7815.
27	PAGE L, BRIN S, MOTWANI R, et al. The PageRank citation ranking: bringing order to the web[EB/OL]. [2024-09-10]. http://www-db.stanford.edu/~backrub/pageranksub.ps.
28	HAVELIWALA T H. Topic-sensitive PageRank: a context-sensitive ranking algorithm for web search. IEEE Transactions on Knowledge and Data Engineering, 2003, 15(4): 784- 796. doi: 10.1109/TKDE.2003.1208999
29	JEH G, WIDOM J, JEH G, et al. Scaling personalized web search[C]//Proceedings of the 12th International Conference on World Wide Web. New York, USA: ACM Press, 2003: 271-279.
30	BAHMANI B, CHOWDHURY A, GOEL A. Fast incremental and personalized PageRank. VLDB Endowment, 2010, 4(3): 173- 184. doi: 10.14778/1929861.1929864
31	WEI Z W, HE X D, XIAO X K, et al. TopPPR: top-k personalized PageRank queries with precision guarantees on large graphs[C]//Proceedings of International Conference on Management of Data. New York, USA: ACM Press, 2018: 2748-2758.
32	CHIEN E, PENG J H, LI P, et al. Adaptive universal generalized PageRank graph neural network[C]//Proceedings of International Conference on Learning Representations. [S. l. ]: ICLR Press, 2021: 1-24.
33	KIPF T N, WELLING M. Variational graph autoencoders[C]//Proceedings of the 30th Annual Conference on Neural Information Processing Systems. Barcelona, Spain: NIPS Press, 2016: 1-3.
34	SEN P, NAMATA G, BILGIC M, et al. Collective classification in network data. AI Magazine, 2008, 29(3): 93- 106. doi: 10.1609/aimag.v29i3.2157
35	SOHN K, LEE H, YAN X. Learning structured output representation using deep conditional generative models[C]//Proceedings of Annual Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2015: 3483-3491.
36	DEFFERRARD M, BRESSON X, VANDERGHEYNST P. Convolutional neural networks on graphs with fast localized spectral filtering[C]//Proceedings of Annual Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2016: 3837-3845.
37	KLICPERA J, BOJCHEVSKI A, GÜNNEMANN S. Predict then propagate: graph neural networks meet personalized pagerank[C]//Proceedings of International Conference on Learning Representation. New Orleans, USA: ICLR Press, 2019: 1-15.
38	WANG H, ZHOU C, CHEN X, et al. Graph stochastic neural networks for semi-supervised learning[C]//Proceedings of Annual Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2020: 1-10.
39	FENG W, ZHANG J, DONG Y, et al. Graph random neural network for semi-supervised learning on graphs[C]//Proceedings of Annual Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2020: 1-12.
40	WIJESINGHE A, WANG Q. A new perspective on "how graph neural networks go beyond weisfeiler-lehman?"[C]//Proceedings of International Conference on Learning Representations. [S. l. ]: ICLR Press, 2022: 1-23.
41	YANG Z, COHEN W, SALAKHUDINOV R. Revisiting semi-supervised learning with graph embeddings[C]//Proceedings of International Conference on Machine Learning. New York, USA: ICML Press, 2016: 40-48.

[1]	QIN Yongwang, ZHANG Yang, HU Xing, LIU Sheng, LI Shaoqing. Gate-level Netlist Function Recognition Based on Graph Attention Networks [J]. Computer Engineering, 2025, 51(6): 29-37.
[2]	LIU Dage, YOU Jinguo, GENG Qiqi. Cross-Domain Aspect Term Extraction Fusing Global and Local Semantics [J]. Computer Engineering, 2025, 51(6): 116-126.
[3]	PANG Xin, GE Fengpei, LI Yanling. Soundscape Recognition: Explorations and Frontiers of Acoustic Scene Classification in the Digital Era [J]. Computer Engineering, 2025, 51(6): 1-19.
[4]	HAO Zhifeng, LI Yanglin, XU Boyan, CAI Ruichu. Hypergraph Neural Networks for Cross-domain Text-to-SQL [J]. Computer Engineering, 2025, 51(5): 114-123.
[5]	HUANG Yao, CHAI Zhilei. Communication and Topology-Aware Partitioning and Mapping Algorithm for SNN [J]. Computer Engineering, 2025, 51(5): 219-228.
[6]	WU Kaifeng, LIU Lei, LIU Chen, LIANG Chengqing. Unmanned Aerial Vehicle Formation Control Based on MADDPG with Integrated Curriculum Learning [J]. Computer Engineering, 2025, 51(5): 73-82.
[7]	YAO Xun, WANG Haipeng, HU Xinrong, YANG Jie. Multi-view Contrastive Recommendation Algorithm Based on Adaptive Enhancement [J]. Computer Engineering, 2025, 51(5): 103-113.
[8]	LIU Wenjie, CHEN Liang, REN Zhijie. Few-shot Relation Reasoning Model Based on Graph Neural Network and Meta-Learning [J]. Computer Engineering, 2025, 51(5): 124-132.
[9]	GUO Peilin, ZHANG De, WANG Huaixiu. Exploring the Impact of Skip Connection Structures on the Deep Neural Networks Feature Extraction Based on Feature Visualization [J]. Computer Engineering, 2025, 51(4): 149-157.
[10]	YANG Ping, ZHANG Xi. Improved DeepLabv3+ Road Surface Crack Detection Method [J]. Computer Engineering, 2025, 51(4): 261-270.
[11]	LIU Yunxiang, LIANG Zhichao. A Highly Efficient Traffic Prediction Model for Continuous Time-series Graph Attention Networks [J]. Computer Engineering, 2025, 51(4): 350-359.
[12]	LI Siyuan, ZHONG Xingyu, LI Kaiyin, XU Qingzhen. Strategy Teaching Research Based on Multilayer Graph Relationship and Reinforcement Learning [J]. Computer Engineering, 2025, 51(3): 122-130.
[13]	CAI Ruichu, XU Zunhong, CHEN Daoxin, YANG Zhenhui, LI Zijian, HAO Zhifeng. Causal Mechanism-Based Molecular Property Prediction [J]. Computer Engineering, 2025, 51(3): 105-112.
[14]	LIU Chunyu, CHEN Qingfeng, MO Shaocong, XIE Ze. Knowledge Graph Completion Based on Logical Rules and Graph Neural Network [J]. Computer Engineering, 2025, 51(3): 131-143.
[15]	ZHANG Zhaoxin, HUANG Shize, ZHANG Bingjie, SHEN Tuo. Camouflaged Adversarial Example Generation Method for the Form of Motion Blur in Traffic Scenes [J]. Computer Engineering, 2025, 51(3): 45-53.

Please choose a citation manager

Content to export