基于对抗训练的伪标签约束自编码器

doi:10.19678/j.issn.1000-3428.0065936

摘要/Abstract

摘要：

社会网络的节点经常存在标注缺失、标注错误和人工标注成本高的现象，这种现象对监督或半监督的网络表示学习效果产生了不利的影响。提出一个自监督学习的网络表示学习模型——基于对抗训练的伪标签约束自编码器（AT-PLCAE）。设计一个伪标签约束自编码器，通过缩短原始图的伪标签和网络表示的伪标签之间的距离，减少编码过程中产生的信息损失，约束和引导模型有效学习。同时，设计与伪标签约束自编码器相适应的对抗网络，组织表示的潜在空间结构。将潜在表示后验分布与输入的特定先验分布相匹配后，该模型能够缓解过拟合问题并提升模型的泛化能力。在Cora、Citeseer、Wiki和Pubmed这4个公开数据集上进行节点分类实验，结果表明，AT-PLCAE模型在分类准确率方面学习效果优于基准方法，与基准方法的最高分类准确率相比，在Cora数据集上提升0.018，在Citeseer和Pubmed数据集上均提升0.011。同时消融实验结果表明，针对伪标签约束自编码器的对抗训练增强了模型的泛化能力。

关键词: 自监督学习, 网络表示学习, 伪标签, 自编码器, 对抗训练, 泛化

Abstract:

Nodes in social networks often suffer from missing labels, labeling errors, and high manual labeling costs, which negatively affect supervised and semi-supervised network representation learning. In this paper, a self-supervised network representation learning model, the Adversarial Training based Pseudo Label Constraint Auto-Encoder(AT-PLCAE) is proposed, whereby an autoencoder constrained by pseudo-label is designed. By reducing the distance between the pseudo-label of the original graph and that of the network representation, the model reduces information loss during the coding process, thereby learning effectively. Adversarial networks are designed so that the pseudo-label-constrained auto-encoder can organize the potential spatial structure of the representation layer. By forcing the posterior distribution of the latent representation to match the specific prior distribution of the input, the model alleviates the overfitting problem and improves generalization. Node classification experiments on four public datasets: Cora, Citeseer, Wiki, and PubMed, showed that the AT-PLCAE model is superior to the benchmark method. Compared with the benchmark method, AT-PLCAE improved the highest classification accuracy value by 0.018 on Cora, 0.011 on CiteSeer, and PubMed, respectively. The ablation experiment results showed that adversarial networks designed for the pseudo-label-constrained auto-encoder enhance the generalization of the model.

Key words: self-supervised learning, network representation learning, pseudo label, auto-encoder, adversarial training, generalization

富坤, 孙明磊, 郝玉涵, 刘赢华. 基于对抗训练的伪标签约束自编码器[J]. 计算机工程, 2023, 49(11): 123-130.

Kun FU, Minglei SUN, Yuhan HAO, Yinghua LIU. Adversarial Training Based Pseudo Label Constraint Auto-Encoder[J]. Computer Engineering, 2023, 49(11): 123-130.

http://www.ecice06.com/CN/Y2023/V49/I11/123

图/表 8

图1 模型框架

Fig.1 Model framework

图2 Cora数据集的表示层可视化图

Fig.2 Visualization diagrams of presentation layer of Cora dataset

图3 表示隐藏层维度对算法性能的影响

Fig.3 Infulence of representation hidden layer dimension on algorithm performance

图4 伪标签约束项权重对算法性能的影响

Fig.4 Influence of pseudo label constraint weight on algorithm performance

参考文献 25

1	LIU X Y , TANG J . Network representation learning: a macro and micro view. AI Open, 2021, 2, 43- 64. doi: 10.1016/j.aiopen.2021.02.001
2	HU Z N, DONG Y X, WANG K S, et al. GPT-GNN: generative pre-training of graph neural networks[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM Press, 2020: 1857-1867.
3	VELIČKOVIĆ P, FEDUS W, HAMILTON W L, et al. Deep graph infomax[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1809.10341.pdf.
4	ZHU Q K, DU B, YAN P K. Self-supervised training of graph convolutional networks[EB/OL]. [2022-09-10]. https://arxiv.org/abs/2006.02380.pdf.
5	LIU Y X , JIN M , PAN S R , et al. Graph self-supervised learning: a survey. IEEE Transactions on Knowledge and Data Engineering, 2023, 35 (6): 5879- 5900.
6	MAKHZANI A, SHLENS J, JAITLY N, et al. Adversarial autoencoders[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1511.05644.pdf.
7	DONAHUE J, KRÄHENBÜHL P, DARRELL T. Adversarial feature learning[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1605.09782.pdf.
8	DUMOULIN V, BELGHAZI I, POOLE B, et al. Adversarially learned inference[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1606.00704.pdf.
9	PAN S R , HU R Q , FUNG S F , et al. Learning graph embedding with adversarial training methods. IEEE Transactions on Cybernetics, 2020, 50 (6): 2475- 2487. doi: 10.1109/TCYB.2019.2932096
10	YOU Y N, CHEN T L, WANG Z Y, et al. When does self-supervision help graph convolutional networks?[EB/OL]. [2022-09-10]. https://arxiv.org/abs/2006.09136.pdf.
11	WANG C, PAN S R, LONG G D, et al. MGAE: marginalized graph autoencoder for graph clustering[C]//Proceedings of 2017 ACM Conference on Information and Knowledge Management. New York, USA: ACM Press, 2017: 889-898.
12	PARK J, LEE M, CHANG H J, et al. Symmetric graph convolutional autoencoder for unsupervised graph representation learning[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2020: 6518-6527.
13	KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1609.02907.pdf.
14	WANG D X, CUI P, ZHU W W. Structural deep network embedding[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM Press, 2016: 1225-1234.
15	KIPF T N, WELLING M. Variational graph auto-encoders[EB/OL]. [EB/OL]. [2022-09-10]. https://arxiv.org/abs/1611.07308.pdf.
16	VELIČKOVIĆ P, CUCURULL G, CASANOVA A, et al. Graph attention networks[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1710.10903.pdf.
17	XU K, HU W H, LESKOVEC J, et al. How powerful are graph neural networks?[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1810.00826.pdf.
18	KINGMA D P, WELLING M. Auto-encoding variational Bayes[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1312.6114.pdf.
19	富坤, 赵晓梦, 付紫桐, 等. 基于不完全信息的深度网络表示学习方法. 计算机科学, 2021, 48 (12): 212- 218. URL
	FU K , ZHAO X M , FU Z T , et al. Deep network representation learning method on incomplete information networks. Computer Science, 2021, 48 (12): 212- 218. URL
20	CAO S S, LU W, XU Q K. Deep neural networks for learning graph representations[C]//Proceedings of the 30th AAAI Conference on Artificial Intelligence. New York, USA: ACM Press, 2016: 1145-1152.
21	GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2014: 2672-2680.
22	ARJOVSKY M, CHINTALA S, BOTTOU L. Wasserstein GAN[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1701.07875.pdf.
23	PEROZZI B, AL-RFOU R, SKIENA S. DeepWalk: online learning of social representations[C]//Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM Press, 2014: 701-710.
24	GROVER A, LESKOVEC J. node2vec: scalable feature learning for networks[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM Press, 2016: 855-864.
25	HUANG X A, LI J D, HU X A. Accelerated attributed network embedding[C]//Proceedings of 2017 SIAM International Conference on Data Mining. Philadelphia, USA: Society for Industrial and Applied Mathematics, 2017: 633-641.

[1]	朱孟栩, 张文豪, 李国洪, 顾行发, 余涛, 郑逢杰, 张丽丽, 吴俣, 邴芳飞, 唐健雄. 基于卷积神经网络的高分六号卫星多光谱图像压缩[J]. 计算机工程, 2023, 49(9): 287-294.
[2]	沈志东, 岳恒宪. 基于分布式扰动的文本对抗训练方法[J]. 计算机工程, 2023, 49(9): 16-22.
[3]	刘晓黎, 王轶彤. 基于自监督学习的多密度图会话推荐[J]. 计算机工程, 2023, 49(9): 60-68, 78.
[4]	邵良杉, 赵松泽. 基于多模型融合的不完整数据分数插补算法[J]. 计算机工程, 2023, 49(9): 79-88, 98.
[5]	李军侠, 王星驰, 殷梓, 石德硕. 边缘深度挖掘的弱监督显著性目标检测[J]. 计算机工程, 2023, 49(7): 169-178.
[6]	朱红, 牛浩然, 朱彤. 基于字词融合与对抗训练的行业人物实体识别[J]. 计算机工程, 2023, 49(5): 56-62.
[7]	张子宣, 宗学军, 何戡, 连莲. 基于CVAE-CatBoost的工业控制网络异常流量检测研究[J]. 计算机工程, 2023, 49(5): 173-180.
[8]	许凤, 杨兴耀, 于炯, 李梓杨, 李晨瑜, 张君. 小波卷积增强的对比学习推荐算法[J]. 计算机工程, 2023, 49(5): 105-111,121.
[9]	何悦, 陈广胜, 景维鹏, 徐泽堃. 基于深度多相似性哈希方法的遥感图像检索[J]. 计算机工程, 2023, 49(2): 206-212.
[10]	刘强, 张颖, 周卫祥, 蒋先涛, 周薇娜, 周谋国. 自适应类增量学习的物联网入侵检测系统[J]. 计算机工程, 2023, 49(2): 169-174.
[11]	袁立宁, 胡皓, 刘钊. 基于多通道图卷积自编码器的图表示学习[J]. 计算机工程, 2023, 49(2): 150-160,174.
[12]	李惠森, 侯进, 党辉, 周宇航. 基于渐进式训练的多判别器域适应目标检测[J]. 计算机工程, 2023, 49(10): 202-211, 221.
[13]	王曙燕, 郭睿涵, 孙家泽. 基于图对比学习的MOOC推荐方法[J]. 计算机工程, 2023, 49(1): 57-64,72.
[14]	雷洁, 饶文碧, 杨焱超, 熊盛武. 基于分类不确定性的伪标签目标检测算法[J]. 计算机工程, 2023, 49(1): 49-56.
[15]	闫静, 张雪英, 李凤莲, 陈桂军, 黄丽霞. 结合栈式监督AE与可变加权ELM的回归预测模型[J]. 计算机工程, 2022, 48(8): 62-69,76.

选择文件类型/文献管理软件名称

选择包含的内容