基于联合熵的多视图集成聚类分析

doi:10.19678/j.issn.1000-3428.0066233

摘要/Abstract

摘要：

多视图方法可使问题分析的角度更加全面，并且能有效利用各个视图间的相关信息和互补信息，因而多视图聚类分析已成为机器学习与模式识别等领域的研究热点之一。但在多视图集成聚类分析中，基聚类簇作为基聚类中的一个类簇，包含若干相似数据对象，其疏密程度仅能体现数据自身分布特性，并不能体现基聚类簇质量。利用联合熵来评估基聚类簇的不确定性及质量，提出一种多视图集成聚类分析方法。利用联合熵评估基聚类簇的质量，通过基聚类簇不确定性指数体现基聚类簇的重要程度与质量优劣。利用基聚类簇不确定性指数构造一种加权共协矩阵，提出一种多视图集成聚类算法(MVECJE)，改善多视图集成聚类分析的性能。通过实验验证聚类簇权重在多视图集成聚类分析中的重要程度，表明其能改善集成聚类性能。在MSRC-v1、Caltech101-7、Handwritten numerals(HW)图像数据集和Reuters文本数据集上，采用CoregSC、AWGL、MMSC、DIMSC、COMVSC、MVKKM和CW$ {\mathrm{K}}^{2} $M作为对比算法进行实验，结果表明，在NMI、ACC、ARI评价指标上，MVECJE算法具有明显的优势，其中在HW数据集上3个评价指标均高于0.93。

关键词: 多视图集成聚类, 基聚类簇, 权重, 加权共协矩阵, 联合熵

Abstract:

Multi-view clustering analysis has become a research hotspot in machine learning and pattern recognition as a more comprehensive perspective, and the relevant and complementary information between various views are provided. However, in multi-view ensemble clustering analysis, the base clustering cluster, as a cluster in base clustering, contains several similar data objects. Its density can only reflect the distribution characteristics of the data and not the quality of the base cluster. In this study, joint entropy is used to evaluate the uncertainty and quality of the base cluster, and a multi-view ensemble clustering analysis method is proposed. First, the quality of the base clusters is evaluated using joint entropy and expressed using an uncertainty index.Second, a weighted co-occurrence matrix is constructed using the uncertainty index of the base cluster, and a Multi-View Ensemble Clustering algorithm based on Joint Entropy(MVECJE) is established, which effectively improves the performance of the multi-view ensemble clustering analysis. Finally, through experiments, the importance of cluster weights in multi-view ensemble clustering analysis is verified and the performance of ensemble clustering is improved. In addition, on the MSRC-v1, Caltech101-7, and Handwritten numerals(HWs) image datasets, and Reuters text datasets, CoregSC, AWGL, MMSC, DIMSC, COMVSC, MVKKM, and CW$ {\mathrm{K}}^{2} $M are considered as comparison algorithms. From the experimental results, the MVECJE algorithm presents advantages regarding the NMI, ACC, and ARI evaluation indices, the three evaluation indices in the HW dataset are higher than 0.93.

Key words: multi-view ensemble clustering, base cluster, weight, weighted co-occurrence matrix, joint entropy

赵晓杰, 牛雪莹, 张继福. 基于联合熵的多视图集成聚类分析[J]. 计算机工程, 2023, 49(10): 112-119.

Xiaojie ZHAO, Xueying NIU, Jifu ZHANG. Multi-View Ensemble Clustering Analysis Based on Joint Entropy[J]. Computer Engineering, 2023, 49(10): 112-119.

http://www.ecice06.com/CN/Y2023/V49/I10/112

图/表 5

参考文献 30

1	CHAO G Q, SUN S L, BI J B. A survey on multiview clustering. IEEE Transactions on Artificial Intelligence, 2021, 2(2): 146- 168. doi: 10.1109/TAI.2021.3065894
2	HAN J, KAMBER M, PEI J. Data mining: concepts and techniques[M]. [S. l. ]: Morgan Kaufmann Publishers Inc., 2006.
3	钱雪忠, 姚琳燕. 面向稀疏高维大数据的扩展增量模糊聚类算法. 计算机工程, 2019, 45(6): 75-81, 88.. URL
	QIAN X Z, YAO L Y. Extended incremental fuzzy clustering algorithm for sparse high-dimensional big data. Computer Engineering, 2019, 45(6): 75-81, 88.. URL
4	YAO Y Q, LI Y, JIANG B B, et al. Multiple kernel k-means clustering by selecting representative kernels. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(11): 4983- 4996. doi: 10.1109/TNNLS.2020.3026532
5	FENG L, CAI L, LIU Y, et al. Multi-view spectral clustering via robust local subspace learning. Soft Computing, 2017, 21(8): 1937- 1948. doi: 10.1007/s00500-016-2120-3
6	TZORTZIS G, LIKAS A. Kernel-based weighted multi-view clustering[C]//Proceedings of the 12th International Conference on Data Mining. Washington D. C., USA: IEEE Press, 2013: 675-684.
7	XU Y M, WANG C D, LAI J H. Weighted multi-view clustering with feature selection. Pattern Recognition: The Journal of the Pattern Recognition Society, 2016, 53, 25- 35. doi: 10.1016/j.patcog.2015.12.007
8	LIU J, CAO F Y, GAO X Z, et al. A cluster-weighted kernel K-means method for multi-view clustering. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(4): 4860- 4867. doi: 10.1609/aaai.v34i04.5922
9	HU J T, LI M M, ZHU E, et al. Consensus multiple kernel K-means clustering with late fusion alignment and matrix-induced regularization. IEEE Access, 2019, 7, 136322- 136331. doi: 10.1109/ACCESS.2019.2940896
10	WANG H, YANG Y, LIU B, et al. A study of graph-based system for multi-view clustering. Knowledge-Based Systems, 2019, 163, 1009- 1019. doi: 10.1016/j.knosys.2018.10.022
11	陶洋, 鲍灵浪, 胡昊. 结构约束的对称低秩表示子空间聚类算法. 计算机工程, 2021, 47(4): 56-61, 67. URL
	TAO Y, BAO L L, HU H. Structure-constrained symmetric low-rank representation algorithm for subspace clustering. Computer Engineering, 2021, 47(4): 56-61, 67. URL
12	WANG Y, ZHANG W, WU L, et al. Iterative views agreement: an iterative low-rank based structured optimization method to multi-view spectral clustering[C]//Proceedings of the 25th International Joint Conference on Artificial Intelligence. New York, USA: AAAI Press, 2016: 2153-2159.
13	CAO X C, ZHANG C Q, FU H Z, et al. Diversity-induced multi-view subspace clustering[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2015: 586-594.
14	EL HAJJAR S, DORNAIKA F, ABDALLAH F. One-step multi-view spectral clustering with cluster label correlation graph. Information Sciences, 2022, 592, 97- 111. doi: 10.1016/j.ins.2022.01.017
15	LI M, LIU X, LEI W, et al. Multiple kernel clustering with local kernel alignment maximization[C]//Proceedings of the 25th International Joint Conference on Artificial Intelligence. New York, USA: AAAI Press, 2016: 1704-1710.
16	CAI X, NIE F P, HUANG H, et al. Heterogeneous image feature integration via multi-modal spectral clustering[C]//Proceedings of CVPR'11. Washington D. C., USA: IEEE Press, 2011: 1977-1984.
17	NIE F P, LI J, LI X L. Parameter-free auto-weighted multiple graph learning: a framework for multiview clustering and semi-supervised classification[C]//Proceedings of the 25th International Joint Conference on Artificial Intelligence. New York, USA: AAAI Press, 2016: 1881-1887.
18	LIN S X, ZHONG G, SHU T. Simultaneously learning feature-wise weights and local structures for multi-view subspace clustering. Knowledge-Based Systems, 2020, 205, 106280. doi: 10.1016/j.knosys.2020.106280
19	NIE F P, TIAN L, LI X L. Multiview clustering via adaptively weighted procrustes[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM Press, 2018: 2022-2030.
20	CHECHIK G, HEITZ G, ELIDAN G, et al. Max-margin classification of data with absent features. Journal of Machine Learning Research, 2008, 9, 1- 21.
21	ZHOU D Y, BURGES C J C. Spectral clustering and transductive learning with multiple views[C]//Proceedings of the 24th International Conference on Machine Learning. New York, USA: ACM Press, 2007: 1159-1166.
22	YU S, TRANCHEVENT L, LIU X H, et al. Optimized data fusion for kernel k-means clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(5): 1031- 1039. doi: 10.1109/TPAMI.2011.255
23	LIU X W, DOU Y, YIN J P, et al. Multiple kernel k-means clustering with matrix-induced regularization[C]//Proceedings of the 30th AAAI Conference on Artificial Intelligence. New York, USA: AAAI Press, 2016: 1888-1894.
24	YIN Q Y, WU S, HE R, et al. Multi-view clustering via pairwise sparse subspace representation. Neurocomputing, 2015, 156, 12- 21. doi: 10.1016/j.neucom.2015.01.017
25	ZHANG X C, ZHAO L, ZONG L L, et al. Multi-view clustering via multi-manifold regularized nonnegative matrix factorization[C]//Proceedings of IEEE International Conference on Data Mining. Washington D. C., USA: IEEE Press, 2015: 1103-1108.
26	HUANG D, WANG C D, LAI J H. Locally weighted ensemble clustering. IEEE Transactions on Cybernetics, 2018, 48(5): 1460- 1473. doi: 10.1109/TCYB.2017.2702343
27	FRED A L N, JAIN A K. Combining multiple clusterings using evidence accumulation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(6): 835- 850. doi: 10.1109/TPAMI.2005.113
28	STREHL A, GHOSH J. Cluster ensembles―a knowledge reuse framework for combining multiple partitions. Journal of Machine Learning Research, 2003, 3(3): 583- 617.
29	KUMAR A, RAI P, DAUMÉ H. Co-regularized multi-view spectral clustering[C]//Proceedings of the 24th International Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2011: 1413-1421.
30	ZHANG P, LIU X W, XIONG J, et al. Consensus one-step multi-view subspace clustering[C]//Proceedings of the 39th International Conference on Data Engineering. Washington D. C., USA: IEEE Press, 2023: 3761-3762.

[1]	张天骐, 闻斌, 熊天, 吴超. 基于张量分解与场景分割的鲁棒视频水印算法[J]. 计算机工程, 2023, 49(8): 250-256, 264.
[2]	马建红, 龚天, 姚爽. 基于证据句与图卷积网络的文档级关系抽取[J]. 计算机工程, 2023, 49(8): 104-110.
[3]	汤卫芬, 高翠芳. 极值点自适应加权的动态时间规整算法[J]. 计算机工程, 2023, 49(7): 150-160.
[4]	江雨燕, 邵金, 李平. 融合自动权重学习的深度子空间聚类[J]. 计算机工程, 2022, 48(8): 77-84,97.
[5]	贺娜, 马盈仓. 融合KL信息的多视图模糊聚类算法[J]. 计算机工程, 2022, 48(7): 114-121,150.
[6]	邢彤彤, 孙仁诚, 邵峰晶, 隋毅. 深度学习中的权重初始化方法研究[J]. 计算机工程, 2022, 48(7): 104-113.
[7]	艾成豪, 高建华, 黄子杰. 混合特征选择和集成学习驱动的代码异味检测[J]. 计算机工程, 2022, 48(7): 168-176,198.
[8]	谢斌红, 秦耀龙, 张英俊. 基于学习主动中心轮廓模型的场景文本检测[J]. 计算机工程, 2022, 48(3): 244-252,262.
[9]	黄帅, 张毅. 基于梯形跨尺度特征耦合网络的SAR图像舰船检测[J]. 计算机工程, 2022, 48(12): 270-280.
[10]	张立恒, 王浩, 薛博维, 何立明, 吕悦. 基于改进D-LinkNet模型的高分遥感影像道路提取研究[J]. 计算机工程, 2021, 47(9): 288-296.
[11]	李伟, 黄鹤鸣, 武风英, 张会云. 基于深度多特征融合的自适应CNN图像分类算法[J]. 计算机工程, 2021, 47(9): 235-239,251.
[12]	刘苗苗, 周从华, 张婷. 基于分段特征及自适应加权的DTW相似性度量[J]. 计算机工程, 2021, 47(8): 62-68,77.
[13]	阎馨, 朱永浩, 屠乃威, 吴书文, 王雨虹. 基于PCA与权重贝叶斯的工作面煤与瓦斯突出预测[J]. 计算机工程, 2021, 47(8): 315-320.
[14]	马佳佳, 张志强, 曹素珍, 窦凤鸽, 丁晓晖, 王彩芬. 基于雾节点的分布式属性基加密方案[J]. 计算机工程, 2021, 47(6): 38-43.
[15]	黄彬, 胡立坤, 张宇. 基于自适应权重的改进Census立体匹配算法[J]. 计算机工程, 2021, 47(5): 189-196.

选择文件类型/文献管理软件名称

选择包含的内容