基于对抗训练的伪标签约束自编码器

doi:10.19678/j.issn.1000-3428.0065936

摘要/Abstract

摘要：

社会网络的节点经常存在标注缺失、标注错误和人工标注成本高的现象，这种现象对监督或半监督的网络表示学习效果产生了不利的影响。提出一个自监督学习的网络表示学习模型——基于对抗训练的伪标签约束自编码器（AT-PLCAE）。设计一个伪标签约束自编码器，通过缩短原始图的伪标签和网络表示的伪标签之间的距离，减少编码过程中产生的信息损失，约束和引导模型有效学习。同时，设计与伪标签约束自编码器相适应的对抗网络，组织表示的潜在空间结构。将潜在表示后验分布与输入的特定先验分布相匹配后，该模型能够缓解过拟合问题并提升模型的泛化能力。在Cora、Citeseer、Wiki和Pubmed这4个公开数据集上进行节点分类实验，结果表明，AT-PLCAE模型在分类准确率方面学习效果优于基准方法，与基准方法的最高分类准确率相比，在Cora数据集上提升0.018，在Citeseer和Pubmed数据集上均提升0.011。同时消融实验结果表明，针对伪标签约束自编码器的对抗训练增强了模型的泛化能力。

关键词: 自监督学习, 网络表示学习, 伪标签, 自编码器, 对抗训练, 泛化

Abstract:

Nodes in social networks often suffer from missing labels, labeling errors, and high manual labeling costs, which negatively affect supervised and semi-supervised network representation learning. In this paper, a self-supervised network representation learning model, the Adversarial Training based Pseudo Label Constraint Auto-Encoder(AT-PLCAE) is proposed, whereby an autoencoder constrained by pseudo-label is designed. By reducing the distance between the pseudo-label of the original graph and that of the network representation, the model reduces information loss during the coding process, thereby learning effectively. Adversarial networks are designed so that the pseudo-label-constrained auto-encoder can organize the potential spatial structure of the representation layer. By forcing the posterior distribution of the latent representation to match the specific prior distribution of the input, the model alleviates the overfitting problem and improves generalization. Node classification experiments on four public datasets: Cora, Citeseer, Wiki, and PubMed, showed that the AT-PLCAE model is superior to the benchmark method. Compared with the benchmark method, AT-PLCAE improved the highest classification accuracy value by 0.018 on Cora, 0.011 on CiteSeer, and PubMed, respectively. The ablation experiment results showed that adversarial networks designed for the pseudo-label-constrained auto-encoder enhance the generalization of the model.

Key words: self-supervised learning, network representation learning, pseudo label, auto-encoder, adversarial training, generalization

富坤, 孙明磊, 郝玉涵, 刘赢华. 基于对抗训练的伪标签约束自编码器[J]. 计算机工程, 2023, 49(11): 123-130.

Kun FU, Minglei SUN, Yuhan HAO, Yinghua LIU. Adversarial Training Based Pseudo Label Constraint Auto-Encoder[J]. Computer Engineering, 2023, 49(11): 123-130.

https://www.ecice06.com/CN/Y2023/V49/I11/123

图/表 8

图1 模型框架

Fig.1 Model framework

图2 Cora数据集的表示层可视化图

Fig.2 Visualization diagrams of presentation layer of Cora dataset

图3 表示隐藏层维度对算法性能的影响

Fig.3 Infulence of representation hidden layer dimension on algorithm performance

图4 伪标签约束项权重对算法性能的影响

Fig.4 Influence of pseudo label constraint weight on algorithm performance

参考文献 25

1	LIU X Y , TANG J . Network representation learning: a macro and micro view. AI Open, 2021, 2, 43- 64. doi: 10.1016/j.aiopen.2021.02.001
2	HU Z N, DONG Y X, WANG K S, et al. GPT-GNN: generative pre-training of graph neural networks[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM Press, 2020: 1857-1867.
3	VELIČKOVIĆ P, FEDUS W, HAMILTON W L, et al. Deep graph infomax[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1809.10341.pdf.
4	ZHU Q K, DU B, YAN P K. Self-supervised training of graph convolutional networks[EB/OL]. [2022-09-10]. https://arxiv.org/abs/2006.02380.pdf.
5	LIU Y X , JIN M , PAN S R , et al. Graph self-supervised learning: a survey. IEEE Transactions on Knowledge and Data Engineering, 2023, 35 (6): 5879- 5900.
6	MAKHZANI A, SHLENS J, JAITLY N, et al. Adversarial autoencoders[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1511.05644.pdf.
7	DONAHUE J, KRÄHENBÜHL P, DARRELL T. Adversarial feature learning[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1605.09782.pdf.
8	DUMOULIN V, BELGHAZI I, POOLE B, et al. Adversarially learned inference[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1606.00704.pdf.
9	PAN S R , HU R Q , FUNG S F , et al. Learning graph embedding with adversarial training methods. IEEE Transactions on Cybernetics, 2020, 50 (6): 2475- 2487. doi: 10.1109/TCYB.2019.2932096
10	YOU Y N, CHEN T L, WANG Z Y, et al. When does self-supervision help graph convolutional networks?[EB/OL]. [2022-09-10]. https://arxiv.org/abs/2006.09136.pdf.
11	WANG C, PAN S R, LONG G D, et al. MGAE: marginalized graph autoencoder for graph clustering[C]//Proceedings of 2017 ACM Conference on Information and Knowledge Management. New York, USA: ACM Press, 2017: 889-898.
12	PARK J, LEE M, CHANG H J, et al. Symmetric graph convolutional autoencoder for unsupervised graph representation learning[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2020: 6518-6527.
13	KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1609.02907.pdf.
14	WANG D X, CUI P, ZHU W W. Structural deep network embedding[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM Press, 2016: 1225-1234.
15	KIPF T N, WELLING M. Variational graph auto-encoders[EB/OL]. [EB/OL]. [2022-09-10]. https://arxiv.org/abs/1611.07308.pdf.
16	VELIČKOVIĆ P, CUCURULL G, CASANOVA A, et al. Graph attention networks[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1710.10903.pdf.
17	XU K, HU W H, LESKOVEC J, et al. How powerful are graph neural networks?[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1810.00826.pdf.
18	KINGMA D P, WELLING M. Auto-encoding variational Bayes[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1312.6114.pdf.
19	富坤, 赵晓梦, 付紫桐, 等. 基于不完全信息的深度网络表示学习方法. 计算机科学, 2021, 48 (12): 212- 218. URL
	FU K , ZHAO X M , FU Z T , et al. Deep network representation learning method on incomplete information networks. Computer Science, 2021, 48 (12): 212- 218. URL
20	CAO S S, LU W, XU Q K. Deep neural networks for learning graph representations[C]//Proceedings of the 30th AAAI Conference on Artificial Intelligence. New York, USA: ACM Press, 2016: 1145-1152.
21	GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2014: 2672-2680.
22	ARJOVSKY M, CHINTALA S, BOTTOU L. Wasserstein GAN[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1701.07875.pdf.
23	PEROZZI B, AL-RFOU R, SKIENA S. DeepWalk: online learning of social representations[C]//Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM Press, 2014: 701-710.
24	GROVER A, LESKOVEC J. node2vec: scalable feature learning for networks[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM Press, 2016: 855-864.
25	HUANG X A, LI J D, HU X A. Accelerated attributed network embedding[C]//Proceedings of 2017 SIAM International Conference on Data Mining. Philadelphia, USA: Society for Industrial and Applied Mathematics, 2017: 633-641.

[1]	张正康, 杨丹, 聂铁铮, 寇月. 基于图结构聚类的自监督学习疾病诊断方法[J]. 计算机工程, 2024, 50(7): 360-371.
[2]	余长宏, 许孔豪, 张泽, 高明. 基于分割点改进孤立森林的网络入侵检测方法[J]. 计算机工程, 2024, 50(6): 148-156.
[3]	魏琢艺, 罗迈, 李文兵, 曾远松, 余伟江, 杨跃东. 基于多源域适应的单细胞智能分类[J]. 计算机工程, 2024, 50(6): 48-55.
[4]	李晶, 李健, 陈海丰, 张倩, 王丽燕, 裴二成. 基于关键区域遮挡与重建的人脸表情识别[J]. 计算机工程, 2024, 50(5): 241-249.
[5]	周昭辰, 方清茂, 吴晓红, 胡平, 何小海. 基于MacBERT与对抗训练的机器阅读理解模型[J]. 计算机工程, 2024, 50(5): 41-50.
[6]	张宝鑫, 杨丹, 聂铁铮, 寇月. 基于自监督的多视角图协同过滤推荐方法[J]. 计算机工程, 2024, 50(5): 100-110.
[7]	王琳, 黄浩. 引入预训练表示混合矢量量化和CTC的语音转换[J]. 计算机工程, 2024, 50(4): 313-320.
[8]	陈琳, 范元凯, 何震瀛, 刘晓清, 杨阳, 汤路民. SQL-to-text模型的组合泛化能力评估方法[J]. 计算机工程, 2024, 50(3): 326-335.
[9]	陈虹, 王瀚文, 金海波. 融合改进自编码器和残差网络的入侵检测模型[J]. 计算机工程, 2024, 50(2): 188-195.
[10]	谢兆贤, 邹兴敏, 张文静. 面向大型数据集的高效决策树参数剪枝算法[J]. 计算机工程, 2024, 50(1): 156-165.
[11]	朱孟栩, 张文豪, 李国洪, 顾行发, 余涛, 郑逢杰, 张丽丽, 吴俣, 邴芳飞, 唐健雄. 基于卷积神经网络的高分六号卫星多光谱图像压缩[J]. 计算机工程, 2023, 49(9): 287-294.
[12]	沈志东, 岳恒宪. 基于分布式扰动的文本对抗训练方法[J]. 计算机工程, 2023, 49(9): 16-22.
[13]	刘晓黎, 王轶彤. 基于自监督学习的多密度图会话推荐[J]. 计算机工程, 2023, 49(9): 60-68, 78.
[14]	邵良杉, 赵松泽. 基于多模型融合的不完整数据分数插补算法[J]. 计算机工程, 2023, 49(9): 79-88, 98.
[15]	李军侠, 王星驰, 殷梓, 石德硕. 边缘深度挖掘的弱监督显著性目标检测[J]. 计算机工程, 2023, 49(7): 169-178.

选择文件类型/文献管理软件名称

选择包含的内容