Self-Supervised Learning Based on Graph Structural Clustering for Disease Diagnosis Method

doi:10.19678/j.issn.1000-3428.0068187

Abstract

Abstract:

Recently, graph self-supervised learning has been applied to disease diagnosis to alleviate the lack of medical labeling information and manual labeling problems. However, the performance of existing graph self-supervised learning heavily relies on high-quality positive and negative samples, which limits the flexibility and generalizability of disease diagnosis. Moreover, the full potential of patients' multi-modal data is not adequately utilized in constructing medical heterogeneous attributed graphs, which affects the performance of disease diagnosis. Therefore, this study proposes a framework called self-supervised learning based on the Structural Clustering of a medical heterogeneous attributed graph for Disease Diagnosis (SC4DD). This framework uses medical structured data and unstructured medical text to construct a medical heterogeneous attributed graph, and generates pseudo-labels for nodes using a structural clustering algorithm on the graph. Considering the different levels of importance of the different meta-paths for learning patient representations and the different impacts of different model medical data on the diagnosis results, a heterogeneous Graph Neural Network (GNN) with an attention mechanism is introduced as an encoder. Pseudo-labels are used as self-supervised signals to assist the encoder in learning the attention coefficients and patient representations. Experimental results on the MIMIC-Ⅲ dataset show that SC4DD outperforms other baselines and effectively improves the disease-diagnosis performance. In particular, compared to the optimal performance baseline method (HeCo), SC4DD achieves improvements of 1.46%, 0.97%, and 0.94% in the Macro-F1 scores, along with improvements of 0.91%, 0.84%, and 0.52% in the Micro-F1 scores, for 2%, 3%, and 4% of labeled nodes.

Key words: disease diagnosis, Electronic Medical Records (EMR), graph self-supervised learning, Graph Neural Network (GNN), medical heterogeneous attributed graph

摘要：

图自监督学习方法近年来被应用于疾病诊断任务中以缓解医疗标签信息缺乏和人工标注问题。然而, 图自监督学习的性能主要依赖于高质量的正样本和负样本, 这限制了疾病诊断的灵活性和泛用性。此外, 在构建医疗异构属性图时没有充分利用病人的多模态数据, 影响了疾病诊断的性能。提出一个基于医疗异构属性图结构聚类的自监督学习疾病诊断框架SC4DD。该框架利用病人的结构化数据和非结构化临床文本摘要构建医疗异构属性图, 通过图上的结构聚类算法生成节点的伪标签。考虑到不同元路径对学习病人嵌入表示的重要性以及不同模态医疗数据对疾病诊断结果的影响程度, 引入注意力机制的异构图神经网络作为编码器, 伪标签作为自监督信号辅助编码器学习注意力系数和病人嵌入表示。在MIMIC-Ⅲ数据集上的实验结果表明, SC4DD优于传统基线方法, 能够有效提高疾病诊断的性能。其中, 相较于性能最优的基线方法HeCo, SC4DD在2%、3%、4%标记节点下的宏平均F1值分别提高了1.46%、0.97%、0.94%, 微平均F1值分别提高了0.91%、0.84%、0.52%。

关键词: 疾病诊断, 电子病历, 图自监督学习, 图神经网络, 医疗异构属性图

Zhengkang ZHANG, Dan YANG, Tiezheng NIE, Yue KOU. Self-Supervised Learning Based on Graph Structural Clustering for Disease Diagnosis Method[J]. Computer Engineering, 2024, 50(7): 360-371.

张正康, 杨丹, 聂铁铮, 寇月. 基于图结构聚类的自监督学习疾病诊断方法[J]. 计算机工程, 2024, 50(7): 360-371.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0068187

https://www.ecice06.com/EN/Y2024/V50/I7/360

Figures/Tables 12

Fig.1 Example of medical heterogeneous attributed graph and relative illustrations of network schema and meta-path

Fig.2 The overall architecture of SC4DD

Fig.3 Patient clinical text summary generation based on BART

Fig.4 Performance comparison of SC4DD and its variant

Fig.5 Visualization of patient nodes embedding representation

Fig.6 Performance variation of SC4DD under different number of pseudo-label classes

Fig.7 Performance variation of SC4DD under different number of attention coefficient dimension

References 33

1	潘嘉诚, 董一鸿, 陈华辉. 基于图神经网络的自闭症辅助诊断研究综述. 计算机工程, 2022, 48(9): 1- 11. URL
	PAN J C, DONG Y H, CHEN H H. Review of research on auxiliary diagnosis of autism based on graph neural networks. Computer Engineering, 2022, 48(9): 1- 11. URL
2	贺煜航, 刘棪, 陈刚. 基于自适应图卷积网络的心电图多标签分类模型. 计算机工程, 2022, 48(12): 261- 269. URL
	HE Y H, LIU Y, CHEN G. Multi-label classification model of electrocardiogram based on adaptive graph convolutional network. Computer Engineering, 2022, 48(12): 261- 269. URL
3	QIU J Z, CHEN Q B, DONG Y X, et al. GCC: graph contrastive coding for graph neural network pre-training[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York, USA: ACM Press, 2020: 1150-1160.
4	PENG Z, HUANG W B, LUO M N, et al. Graph representation learning via graphical mutual information maximization[C]//Proceedings of World Wide Web Conference. New York, USA: ACM Press, 2020: 259-270.
5	ZHU Y Q, XU Y C, YU F, et al. Graph contrastive learning with adaptive augmentation[C]//Proceedings of World Wide Web Conference. New York, USA: ACM Press, 2021: 2069-2080.
6	YU J X, LI X. Heterogeneous graph contrastive learning with meta-path contexts and weighted negative samples[C]//Proceedings of 2023 SIAM International Conference on Data Mining. Washington D. C., USA: IEEE Press, 2023: 37-45.
7	WANG Z H, LI Q, YU D H, et al. Heterogeneous graph contrastive multi-view learning[C]//Proceedings of 2023 SIAM International Conference on Data Mining. Washington D. C., USA: IEEE Press, 2023: 136-144.
8	WANG X, JI H Y, SHI C, et al. Heterogeneous graph attention network[C]//Proceedings of the 28th World Wide Web Conference. New York, USA: ACM Press, 2019: 2022-2032.
9	FU X Y, ZHANG J N, MENG Z Q, et al. MAGNN: metapath aggregated graph neural network for heterogeneous graph embedding[C]//Proceedings of the 29th World Wide Web Conference. New York, USA: ACM Press, 2020: 2331-2341.
10	ZHU S C, ZHOU C, PAN S R, et al. Relation structure-aware heterogeneous graph neural network[C]//Proceedings of the 19th IEEE International Conference on Data Mining. Piscataway, USA: IEEE Press, 2019: 1534-1539.
11	ZHANG C X, SONG D J, HUANG C, et al. Heterogeneous graph neural network[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York, USA: ACM Press, 2019: 793-803.
12	HU Z N, DONG Y X, WANG K S, et al. Heterogeneous graph transformer[C]//Proceedings of World Wide Web Conference. New York, USA: ACM Press, 2020: 2704-2710.
13	RAGHAVAN U N, ALBERT R, KUMARA S. Near linear time algorithm to detect community structures in large-scale networks. Physical Review E, 2007, 76(3): 036106. doi: 10.1103/PhysRevE.76.036106
14	BELLEIC, ALATTAS H, KAANICHE N. Label-GCN: an effective method for adding label propagation to graph convolutional networks[EB/OL]. [2023-07-01]. https://arxiv.org/pdf/2104.02153.
15	SHI Y S, HUANG Z J, FENG S K, et al. Masked label prediction: unified message passing model for semi-supervised classification[C]//Proceedings of the 30th International Joint Conference on Artificial Intelligence. Washington D. C., USA: IEEE Press, 2021: 358-367.
16	WANG H W, LESKOVEC J. Unifying graph convolutional neural networks and label propagation[EB/OL]. [2023-07-01]. https://arxiv.org/pdf/2002.06755.
17	LIU X, ZHANG F J, HOU Z Y, et al. Self-supervised learning: generative or contrastive[EB/OL]. [2023-07-01]. https://arxiv.org/abs/2006.08218v3.
18	KIPF T, WELLING M. Variational graph auto-encoders[EB/OL]. [2023-07-01]. https://arxiv.org/abs/1611.07308.
19	KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[EB/OL]. [2023-07-01]. https://arxiv.org/abs/1609.02907.
20	HU Z N, DONG Y X, WANG K S, et al. GPT-GNN: generative pre-training of graph neural networks[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York, USA: ACM Press, 2020: 1857-1867.
21	HASSANI K, AHMADI A H K. Contrastive multi-view representation learning on graphs[C]//Proceedings of the 37th International Conference on Machine Learning. Washington D. C., USA: IEEE Press, 2020: 4116-4126.
22	WANG X, LIU N, HAN H, et al. Self-supervised heterogeneous graph neural network with co-contrastive learning[C]//Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. New York, USA: ACM Press, 2021: 1726-1736.
23	DAI Q Y, LI Q, TANG J, et al. Adversarial network embedding. Artificial Intelligence, 2018, 32(1): 2167- 2174.
24	SUN Z C, YIN H Z, CHEN H X, et al. Disease prediction via graph neural networks. IEEE Journal of Biomedical and Health Informatics, 2021, 25(3): 818- 826. doi: 10.1109/JBHI.2020.3004143
25	ZHENG S, ZHU Z F, LIU Z Z, et al. Multi-modal graph learning for disease prediction. IEEE Transactions on Medical Imaging, 2022, 41(9): 2207- 2216. doi: 10.1109/TMI.2022.3159264
26	LEWIS M, LIU Y H, GOYAL N, et al. BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: Association for Computational Linguistics, 2020: 7871-7880.
27	YANG Y M, GUAN Z Y, LI J X, et al. Interpretable and efficient heterogeneous graph convolutional network. IEEE Transactions on Knowledge and Data Engineering, 2021, 23(1): 1637- 1650.
28	BARRON J T. Continuously differentiable exponential linear units[EB/OL]. [2023-07-01]. https://arxiv.org/abs/1704.07483.
29	JOHNSON A E W, POLLARD T J, SHEN L, et al. MIMIC-Ⅲ, a freely accessible critical care database. Scientific Data, 2016, 3(1): 160035. doi: 10.1038/sdata.2016.35
30	VELICKOVIC P, CUCURULL G, CASANOVA A, et al. Graph attention networks[C]//Proceedings of IEEE International Conference on Learning Representations. Washington D. C., USA: IEEE Press, 2017: 581-596.
31	DONG Y X, CHAWLA N V, SWAMI A. Metapath2vec: scalable representation learning for heterogeneous networks[C]//Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM Press, 2017: 381-398.
32	KINGMA D P, BA J. Adam: a method for stochastic optimization[C]//Proceedings of IEEE International Conference on Learning Representations. Washington D. C., USA: IEEE Press, 2014: 235-246.
33	VEN DER MAATEN V, HINTON G. 2008. Visualizing Data using t-SNE. Journal of Machine Learning Research, 2008, 9, 2579- 2605.

[1]	XI Jin-ju; TAN Wen-xue; LI Shu-hong. Research on Disease Pattern Resemblance Recognition Model [J]. Computer Engineering, 2010, 36(8): 200-202.
[2]	XI Jin-Ju, TAN Wen-Hua, BI Xu-Tong, HE Jin-Zhou, LI Chu-Gong. Research on Diagnostic Method Based on Multi-mode Composite Reasoning Mechanism [J]. Computer Engineering, 2010, 36(11): 192-194.

Please choose a citation manager

Content to export