«上一篇 下一篇»
  计算机工程  2022, Vol. 48 Issue (7): 114-121, 150  DOI: 10.19678/j.issn.1000-3428.0061852
0

引用本文  

贺娜, 马盈仓. 融合KL信息的多视图模糊聚类算法[J]. 计算机工程, 2022, 48(7), 114-121, 150. DOI: 10.19678/j.issn.1000-3428.0061852.
HE Na, MA Yingcang. Multi-View Fuzzy Clustering Algorithm Fused with KL Information[J]. Computer Engineering, 2022, 48(7), 114-121, 150. DOI: 10.19678/j.issn.1000-3428.0061852.

基金项目

国家自然科学基金(61976130);陕西省重点研发计划(2018KW-021);陕西省自然科学基金(2020JQ-923)

通信作者

马盈仓(通信作者),教授、博士

作者简介

作者简介:贺娜(1995—),女,硕士研究生,主研方向为机器学习

文章历史

收稿日期:2021-06-04
修回日期:2021-08-01
融合KL信息的多视图模糊聚类算法
贺娜 , 马盈仓     
西安工程大学 理学院, 西安 710600
摘要:现有多视图模糊C均值聚类(FCM)算法通常将一个多视图分解为多个单视图进行数据处理,导致视图数据聚类精度降低,从而影响全局数据划分结果。为实现高维数据和多视图数据的高效聚类,提出一种基于KL信息的多视图自加权模糊聚类算法。将多个视图信息及其权重进行拟合融入标准FCM算法,求解多个隶属度矩阵和质心矩阵。在此基础上,通过附加KL信息作为模糊正则项进一步修正共识隶属度矩阵并保持权重分布的平滑性,其中KL信息是视图隶属度与其共识隶属度的比值,最小化KL信息会使每个视图的隶属度偏向于共识隶属度以得到更好的聚类结果。实验结果表明,该算法相比于传统聚类算法具有更好的聚类效果和更快的收敛速度,尤其在3-Sources数据集上相比于MVASM算法的聚类精度、标准化互信息和纯度分别提升了7.46、15.34和5.48个百分点。
关键词多视图聚类    模糊C均值    权重    KL信息    共识隶属度矩阵    
Multi-View Fuzzy Clustering Algorithm Fused with KL Information
HE Na , MA Yingcang     
School of Science, Xi'an Polytechnic University, Xi'an 710600, China
Abstract: Existing multi-view Fuzzy C-Means(FCM) clustering algorithms usually artificially decompose multi-view data into multiple single-view data for processing, reducing the clustering accuracy of view data and affecting the results of global data division.To achieve efficient clustering of high-dimensional and multi-view data, a multi-view self-weighted fuzzy clustering algorithm based on Kullback-Leibler(KL) information is proposed, fitting multiple view information and their weights into the standard FCM algorithm to solve multiple membership matrices and centroid matrices.On this basis, additional KL information is used as a fuzzy regular term to further correct the consensus membership matrix and maintain the smoothness of the weight distribution, where the KL information is the ratio of a view's membership to its consensus membership, and minimizing the KL information biases each view's membership towards consensus membership, resulting in improved clustering results.The results show that the proposed algorithm has an improved clustering effect and faster convergence speed than traditional clustering algorithms.In particular, the clustering Accuracy(ACC), Normalized Mutual Information(NMI), and Purity of the MVASM algorithm on the 3-Sources dataset increased by 7.46, 15.34, and 5.48 percentage points respectively.
Key words: multi-view clustering    Fuzzy C-Means(FCM)    weight    Kullback-Leibler(KL) information    consensus membership matrix    

开放科学(资源服务)标志码(OSID):

0 概述

聚类是一种无监督的机器学习任务,根据数据自身的距离或相似度将它们划分为若干组,划分原则是组内距离最小化而组间距离最大化[1-3]。常见的聚类算法主要包括基于层次[4-5]、基于模糊C均值(Fuzzy C-Means,FCM)[6-7]、基于迭代[8-9]、基于协作[10-11],基于分解[12-14]、基于谱聚类集群[15-17]等,其中基于$ \mathrm{F}\mathrm{C}\mathrm{M} $的聚类算法由于具有较高的聚类准确率且易于处理、空间复杂度低,受到学者们的广泛关注。但是基于$ \mathrm{F}\mathrm{C}\mathrm{M} $的聚类算法主要是针对单一视图数据的聚类算法,当其面对多视图数据时只能对各视图样本进行独立的聚类分析以得到每个视图下的聚类结果,然后使用集成学习机制[18-19]将每个视图下的聚类结果进行统一,最终获取全局意义下的聚类结果。然而人为地将多视图数据分解为多个单视图数据进行处理会因不同视图聚类结果存在明显差异而影响最终获取的全局划分结果。

近些年来,学者们在多视图聚类算法研究方面取得了较大进展。文献[20]通过典型相关分析使用数据的多个视图来构建投影。文献[21]学习了同时为原始问题提供多个视图的非冗余子空间,并在每个视图中找到一个聚类方法。文献[22]提出一种类似将异构图像特征与图相结合的多视角光谱聚类算法。文献[23]引入共正则化谱聚类,采用共正则化框架来解决多视图聚类问题。文献[24]提出解决大规模多视图数据的多视图K-means聚类算法。文献[25]提出同时进行特征选择和多视图聚类的结构化稀疏学习方法。文献[26]分析并研究了基于采样的主动式初始中心选择方法对K-means型多视图聚类算法的影响。虽然上述算法取得了较好的性能,但其中一些算法忽略了视图多样性。文献[27]通过人为干预或先验知识实现多视图加权,但不能保证最终结果与每个视图的贡献相一致。文献[28]提出一种基于FCM的多视角模糊聚类方法CoFKM,但由于每个视图被平等对待,因此没有考虑每个视图的权重。文献[29]指出在某些视图有噪声或存在干扰的情况下,算法聚类精度可能会降低。受现有研究的启发,本文提出一种新的多视图自加权聚类算法KMFC。该算法通过学习共识隶属度矩阵进行聚类表示来挖掘多个视图的潜在共识信息,求解多个隶属度矩阵和质心矩阵,并利用视图的特定信息进一步修正共识隶属度矩阵。

1 模型建立与求解 1.1 模型建立

建立多视图聚类模型,通过引入Kullback-Leibler(KL)信息使学习的矩阵$ {\mathit{\boldsymbol{U}}}^{\mathrm{*}} $尽可能与每个$ {\mathit{\boldsymbol{U}}}^{\left(p\right)} $一致,从而得到如下目标函数:

$ \begin{array}{l}\mathrm{m}\mathrm{i}\mathrm{n}\sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{‖{\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}-{\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}‖}_{2}^{2}\right)+\\ \lambda \sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}\mathrm{l}\mathrm{g}\frac{{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}{{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}\right)\end{array} $ (1)

在提高视图内聚类结果时,需要考虑不同视图间的聚类一致性。若直接将多个视图拼接在一起,则不利于提高聚类性能。更合理的方法是将这些视图与合适的权重$ {\mathit{\boldsymbol{\alpha}}}_{p}\left(p=\mathrm{1, 2}, \cdots , m\right) $进行整合,并增加一个参数$ q $来保持权重分布的平滑性。如果在式(1)中添加这些参数,则调整如下:

$ \begin{array}{l}\mathrm{m}\mathrm{i}\mathrm{n}\sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}{‖{\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}-{\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}‖}_{2}^{2}\right)+\\ \lambda \sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}\mathrm{l}\mathrm{g}\frac{{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}{{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}\right)\\ \mathrm{s}.\mathrm{t}.\sum\limits_{k=1}^{c}{\mathit{\boldsymbol{u}}}_{ik}=1, {\mathit{\boldsymbol{u}}}_{ik}\ge 0, \sum\limits_{p=1}^{m}{\mathit{\boldsymbol{\alpha}}}_{p}=1, {\mathit{\boldsymbol{\alpha}}}_{p}\ge 0, \\ \sum\limits_{k=1}^{c}{\mathit{\boldsymbol{u}}}_{ik}^{*}=1, {\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}\ge 0, i=\mathrm{1, 2}, \cdots , n\end{array} $ (2)

其中:$ {\mathit{\boldsymbol{X}}}^{\left(p\right)}=\left[{\mathit{\boldsymbol{x}}}_{1}^{\left(p\right)}, {\mathit{\boldsymbol{x}}}_{2}^{\left(p\right)}, \cdots , {\mathit{\boldsymbol{x}}}_{n}^{\left(p\right)}\right] $$ {\mathit{\boldsymbol{V}}}^{\left(p\right)}=\left[{\mathit{\boldsymbol{v}}}_{1}^{\left(p\right)}, {\mathit{\boldsymbol{v}}}_{2}^{\left(p\right)}, \cdots , {\mathit{\boldsymbol{v}}}_{c}^{\left(p\right)}\right] $$ {\mathit{\boldsymbol{U}}}^{\left(p\right)}=\left[{\mathit{\boldsymbol{u}}}_{1}^{\left(p\right)}, {\mathit{\boldsymbol{u}}}_{2}^{\left(p\right)}, \cdots , {\mathit{\boldsymbol{u}}}_{c}^{\left(p\right)}\right] $$ {\mathit{\boldsymbol{X}}}^{\left(p\right)}\in {\mathbb{R}}^{{d}_{p}\times n} $是在$ p $个视图中具有$ {d}_{p} $维和$ n $个样本的数据矩阵,第$ i $个数据点记为$ {\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}\in {\mathbb{Z}}^{{d}_{p}} $$ {\mathit{\boldsymbol{U}}}^{\left(p\right)}\in {\mathbb{R}}^{{d}_{p}\times c} $$ p $个视图的隶属度,它的第$ k $个隶属度向量为$ {\mathit{\boldsymbol{u}}}_{k}^{\left(p\right)}\in {\mathbb{R}}^{{d}_{p}} $;矩阵$ {\mathit{\boldsymbol{V}}}^{\left(p\right)}\in {\mathbb{R}}^{{d}_{p}\times c} $$ p $个视图的中心矩阵,它的第$ k $个中心向量为$ {\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}\in {\mathbb{R}}^{{d}_{p}} $$ {\mathit{\boldsymbol{U}}}^{\mathrm{*}}\in {\mathbb{R}}^{n\times c} $是不同视图上的公共隶属度矩阵;$ {\mathit{\boldsymbol{\alpha}}}_{p} $是第$ p $个视图的权值;$ q $为各权重的幂指数;$ \lambda $为正则化系数。

由于每个视图都有各自的属性信息,因此KMFC算法可以使视图内信息与公共信息进行较好的拟合,其中视图内信息来自$ {\mathit{\boldsymbol{v}}}^{\left(p\right)} $,不同视图之间的公共信息来自共识隶属度矩阵$ {\mathit{\boldsymbol{U}}}^{\mathrm{*}} $

1.2 模型求解

将多视图聚类模型的目标函数分为4个子问题,通过迭代交替优化方法进行求解。

1)固定$ \mathit{\boldsymbol{V}} $$ \mathit{\boldsymbol{\alpha}} $$ {\mathit{\boldsymbol{U}}}^{\mathit{*}} $,更新$ \mathit{\boldsymbol{U}} $,问题式(2)可以改写如下:

$ \begin{array}{l}\underset{\sum\limits_{k=1}^{c}{\mathit{\boldsymbol{u}}}_{ik}=1, {\mathit{\boldsymbol{u}}}_{ik}\ge 0}{\mathrm{m}\mathrm{i}\mathrm{n}}\sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}{‖{\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}-{\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}‖}_{2}^{2}\right)+\\ \lambda \sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}\mathrm{l}\mathrm{g}\frac{{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}{{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}\right)\iff \\  \underset{\sum\limits_{k=1}^{c}{\mathit{\boldsymbol{u}}}_{ik}=1, {\mathit{\boldsymbol{u}}}_{ik}\ge 0}{\mathrm{m}\mathrm{i}\mathrm{n}}\sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{h}}}_{ik}^{\left(p\right)}\right)+\\ \lambda \sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}\mathrm{l}\mathrm{g}\frac{{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}{{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}\right)\end{array} $ (3)

其中:$ {\mathit{\boldsymbol{h}}}_{ik}^{\left(p\right)}={\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}{‖{\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}-{\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}‖}_{2}^{2} $

式(3)的拉格朗日函数表示如下:

$ \begin{array}{l}L\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}, \eta \right)=\sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{h}}}_{ik}^{\left(p\right)}\right)+\\ \lambda \sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}\mathrm{l}\mathrm{g}\frac{{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}{{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}\right)+\sum\limits_{i=1}^{n}{\eta }_{i}\left(\sum\limits_{k=1}^{c}{\mathit{\boldsymbol{u}}}_{ik}-1\right)\end{array} $ (4)

其中:$ \eta \ge 0 $为约束条件下的拉格朗日乘子。

对式(4)中的$ {\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)} $求导,设为0,得到:

$ \frac{\partial L}{\partial {\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}=\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{h}}}_{ik}^{\left(p\right)}+{\eta }_{i}+\lambda \sum\limits_{p=1}^{m}\left(\mathrm{l}\mathrm{g}\frac{{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}{{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}+1\right)=0 $ (5)
$ {\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}=\mathrm{e}\mathrm{x}\mathrm{p}\left(\frac{-\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{h}}}_{ik}^{\left(p\right)}}{\lambda }\right){\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}\mathrm{e}\mathrm{x}\mathrm{p}\left(\frac{-{\eta }_{i}-\lambda }{\lambda }\right) $ (6)

考虑等式约束$ \sum\limits_{k=1}^{c}{\mathit{\boldsymbol{u}}}_{ik}=1 $,得到:

$ \begin{array}{l} \sum\limits_{k=1}^{c}\left[\mathrm{e}\mathrm{x}\mathrm{p}\left(\frac{-\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{h}}}_{ik}^{\left(p\right)}}{\lambda }\right){\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}\mathrm{e}\mathrm{x}\mathrm{p}\left(\frac{-{\eta }_{i}-\lambda }{\lambda }\right)\right]=1\Rightarrow \\ \mathrm{e}\mathrm{x}\mathrm{p}\left(\frac{-{\eta }_{i}-\lambda }{\lambda }\right)=\frac{1}{\sum\limits_{k=1}^{c}\left[\mathrm{e}\mathrm{x}\mathrm{p}\left(\frac{-\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{h}}}_{ik}^{\left(p\right)}}{\lambda }\right){\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}\right]} \end{array} $ (7)

结合式(6)和式(7)得到:

$ {\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}=\frac{\mathrm{e}\mathrm{x}\mathrm{p}\left(\frac{-\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{h}}}_{ik}^{\left(p\right)}}{\lambda }\right){\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}{\sum\limits_{k=1}^{c}\left[\mathrm{e}\mathrm{x}\mathrm{p}\left(\frac{-\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{h}}}_{ik}^{\left(p\right)}}{\lambda }\right){\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}\right]} $ (8)

2)固定$ \mathit{\boldsymbol{U}} $$ \mathit{\boldsymbol{\alpha}} $$ {\mathit{\boldsymbol{U}}}^{\mathit{*}} $,更新$ \mathit{\boldsymbol{V}} $,省略与$ \mathit{\boldsymbol{V}} $无关的正则化项:

$ \begin{array}{l}\mathrm{m}\mathrm{i}\mathrm{n}\sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}{‖{\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}-{\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}‖}_{2}^{2}\right)+\\ \lambda \sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}\mathrm{l}\mathrm{g}\frac{{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}{{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}\right)\iff \\ \underset{V}{\mathrm{m}\mathrm{i}\mathrm{n}}\sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}{‖{\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}-{\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}‖}_{2}^{2}\right)\end{array} $ (9)

使用J表示问题式(9)的目标函数,得到:

$ J=\sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left(\begin{array}{l}{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}{\left({\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}\right)}^{\mathrm{T}}{\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}+{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}{\left({\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}\right)}^{\mathrm{T}}{\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}-\\ 2{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}{\left({\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}\right)}^{\mathrm{T}}{\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}\end{array}\right) $ (10)

利用式(10)对$ {\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)} $求导,使其为0,得到:

$ \frac{\partial J\left({\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}\right)}{\partial {\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}}=\sum\limits_{i=1}^{n}\left(2{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}{\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}-2{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}{\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}\right)=0 $ (11)
$ {\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}=\frac{\sum\limits_{i=1}^{n}{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}}{\sum\limits_{i=1}^{n}{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}} $ (12)

3)固定$ \mathit{\boldsymbol{U}} $$ \mathit{\boldsymbol{V}} $$ {\mathit{\boldsymbol{U}}}^{\mathit{*}} $,更新α,省略与α无关的正则化项:

$ \begin{array}{l}\underset{\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{\alpha}}}_{p}=1, {\mathit{\boldsymbol{\alpha}}}_{p}\ge 0}{\mathrm{m}\mathrm{i}\mathrm{n}}\sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}{‖{\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}-{\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}‖}_{2}^{2}\right)+\\ \lambda \sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}\mathrm{l}\mathrm{g}\frac{{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}{{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}\right)\iff \\ \underset{\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{\alpha}}}_{p}=1, {\mathit{\boldsymbol{\alpha}}}_{p}\ge 0}{\mathrm{m}\mathrm{i}\mathrm{n}}\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}\left(\sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{‖{\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}-{\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}‖}_{2}^{2}\right)\iff \\ \underset{\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{\alpha}}}_{p}=1, {\mathit{\boldsymbol{\alpha}}}_{p}\ge 0}{\mathrm{m}\mathrm{i}\mathrm{n}}\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}{\mathit{\pmb{\Phi}}}_{p}\end{array} $ (13)

其中:$ {\mathit{\pmb{\Phi}}}_{p}=\sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{‖{\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}-{\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}‖}_{2}^{2} $

式(13)的拉格朗日函数表示如下:

$ L\left(\mathit{\boldsymbol{\alpha}}, \mathit{\pmb{\gamma}}, \mathit{\pmb{\beta}}\right)=\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}{\mathit{\pmb{\Phi}}}_{p}-\mathit{\pmb{\gamma}}\left({\mathit{\boldsymbol{\alpha}}}^{\mathrm{{\rm T}}}1-1\right)-{\mathit{\pmb{\beta}}}^{\mathrm{{\rm T}}}\mathit{\boldsymbol{\alpha}} $ (14)

其中:$ \mathit{\pmb{\gamma}} $$ \mathit{\pmb{\beta}} $是拉格朗日乘子。

利用式(14)对$ {\mathit{\boldsymbol{\alpha}}}_{p} $求导,使其为0,得到:

$ {\mathit{\boldsymbol{\alpha}}}_{p}^{q-1}=\frac{\mathit{\pmb{\gamma}}}{q{\mathit{\pmb{\Phi}}}_{p}}+\frac{{\mathit{\pmb{\beta}}}_{p}}{q{\mathit{\pmb{\Phi}}}_{p}} $ (15)

问题式(13)的KKT条件表示如下:

$ \mathit{\pmb{\beta}}\ge 0 $ (16)
$ \mathit{\boldsymbol{\alpha}}\ge 0 $ (17)
$ {\mathit{\boldsymbol{\alpha}}}^{\mathrm{{\rm T}}}\mathit{\pmb{\beta}}=0 $ (18)

结合式(16)~式(18)得到$ {\mathit{\pmb{\beta}}}_{p}{\mathit{\boldsymbol{\alpha}}}_{p}=0 $,并将式(15)乘以$ {\mathit{\boldsymbol{\alpha}}}_{p} $得到:

$ {\mathit{\boldsymbol{\alpha}}}_{p}={\left(\frac{\mathit{\pmb{\gamma}}}{q{\mathit{\pmb{\Phi}}}_{p}}\right)}^{\frac{1}{q-1}} $ (19)

考虑等式约束$ \sum\limits_{p=1}^{m}{\mathit{\boldsymbol{\alpha}}}_{p}=1 $,得到:

$ \sum\limits_{p=1}^{m}{\left(\frac{\mathit{\pmb{\gamma}}}{q{\mathit{\pmb{\Phi}}}_{p}}\right)}^{\frac{1}{q-1}}=1\Rightarrow {\left(\frac{\mathit{\pmb{\gamma}}}{q}\right)}^{\frac{1}{q-1}}=\frac{1}{\sum\limits_{p=1}^{m}{\mathit{\pmb{\Phi}}}_{p}^{\frac{1}{1-q}}} $ (20)

结合式(19)与式(20)得到:

$ {\mathit{\boldsymbol{\alpha}}}_{p}=\frac{{\mathit{\pmb{\Phi}}}_{p}^{\frac{1}{1-q}}}{\sum\limits_{s=1}^{m}{\mathit{\pmb{\Phi}}}_{s}^{\frac{1}{1-q}}} $ (21)

4)固定$ \mathit{\boldsymbol{U}} $$ \mathit{\boldsymbol{V}} $α,更新$ {\mathit{\boldsymbol{U}}}^{\mathit{*}} $,省略与$ {\mathit{\boldsymbol{U}}}^{\mathit{*}} $无关的项:

$ \begin{array}{l}\underset{\sum\limits_{k=1}^{c}{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}=1, {\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}\ge 0}{\mathrm{m}\mathrm{i}\mathrm{n}}\sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}{‖{\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}-{\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}‖}_{2}^{2}\right)+\\ \lambda \sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}\mathrm{l}\mathrm{g}\frac{{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}{{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}\right)\iff \\ \underset{\sum\limits_{k=1}^{c}{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}=1, {\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}\ge 0}{\mathrm{m}\mathrm{i}\mathrm{n}}\sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}\mathrm{l}\mathrm{g}\frac{{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}{{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}\right)\end{array} $ (22)

式(22)的拉格朗日函数表示如下:

$ \begin{array}{l}L\left({\mathit{\boldsymbol{u}}}_{ik}, \mathit{\pmb{\xi}}\right)=\sum\limits_{p=1}^{m}\sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}\mathrm{l}\mathrm{g}\frac{{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}{{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}\right)+\\ \sum\limits_{i=1}^{n}{\mathit{\pmb{\xi}}}_{i}\left(\sum\limits_{k=1}^{c}{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}-1\right)\end{array} $ (23)

其中:$ \mathit{\pmb{\xi}}\ge 0 $为约束条件下的拉格朗日乘子。

对式(23)中的$ {\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}} $求导,设为0,得到:

$ \frac{\partial L}{\partial {\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}=\sum\limits_{p=1}^{m}\left(-\frac{{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}{{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}+1\right)+{\mathit{\pmb{\xi}}}_{i}=0 $ (24)
$ {\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}=\frac{1}{{\mathit{\pmb{\xi}}}_{i}}\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)} $ (25)

考虑等式约束$ \sum\limits_{k=1}^{c}{\mathit{\boldsymbol{u}}}_{ik}^{*}=1 $,得到:

$ \sum\limits_{k=1}^{c}\frac{1}{{\mathit{\pmb{\xi}}}_{i}}\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}=1\Rightarrow {\mathit{\pmb{\xi}}}_{i}=\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)} $ (26)

结合式(25)和式(26)得到:

$ {\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}=\frac{\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}{\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}} $ (27)

综上所述,通过问题1的求解可更新$ p $个视图的隶属度矩阵$ {\mathit{\boldsymbol{U}}}^{\left(p\right)} $。通过问题2的求解更新$ {\mathit{\boldsymbol{v}}}^{\left(p\right)} $可得到$ p $个视图的聚类中心矩阵。通过问题3的求解可学习权值$ {\mathit{\boldsymbol{\alpha}}}_{p} $来协调不同的视图。通过问题4的求解可学习一个共识隶属度矩阵$ {\mathit{\boldsymbol{U}}}^{\mathrm{*}} $来表示不同视图之间的聚类。重复上述过程,直到目标函数收敛。

1.3 算法流程

算法1  KMFC算法

输入  $ m $个视图的数据$ \left\{{\mathit{\boldsymbol{X}}}^{\left(p\right)}\left|p=\mathrm{1, 2}, \cdots , m\right.\right\} $$ {\mathit{\boldsymbol{X}}}^{\left(p\right)}\in {\mathbb{R}}^{{d}_{p}\times n} $,聚类个数为$ c $,参数$ \lambda $$ q $

输出  共识隶属度矩阵$ {\mathit{\boldsymbol{U}}}^{\mathrm{*}}\in {\mathbb{R}}^{n\times c} $,聚类中心矩阵$ {\mathit{\boldsymbol{V}}}^{\left(p\right)}\in {\mathbb{R}}^{{d}_{p}\times c} $和每个视图的权重$ {\alpha }_{p} $

1.初始化:通过K-Means聚类方法初始化$ {\mathrm{U}}^{\left(\mathrm{p}\right)} $和V(p),对p个视图初始化αp=1/m;

2.while不收敛do

3.对于每个数据点i,通过求解式(8)更新第p个视图的隶属度矩阵$ {\mathrm{U}}^{\left(\mathrm{p}\right)} $的第i行;

4.根据式(12)更新第p个视图的第k个聚类中心向量$ {\mathrm{v}}_{\mathrm{k}}^{\left(\mathrm{p}\right)} $

5.根据式(21)更新权重$ {\mathrm{\alpha }}_{\mathrm{p}} $

6.对于每个数据点i,通过求解式(27)计算共识隶属度矩阵$ {\mathrm{U}}^{\mathrm{*}} $的第i行;

7.输出$ {\mathrm{U}}^{\mathrm{*}} $$ {\mathrm{V}}^{\left(\mathrm{p}\right)} $$ {\mathrm{\alpha }}_{\mathrm{p}} $

2 理论分析 2.1 幂指数q

利用参数q来调整权值分布,根据式(21)可得出存在两种极端情况:1)当$ q\to \mathrm{\infty } $时,KMFC算法得到相等的权值,即$ \frac{1}{c} $;2)当$ q\to {1}^{+} $时,设$ {\mathit{\pmb{\Phi}}}_{o} $$ \left\{{\mathit{\pmb{\Phi}}}_{1}, {\mathit{\pmb{\Phi}}}_{2}, \cdots , {\mathit{\pmb{\Phi}}}_{o}, \cdots , {\mathit{\pmb{\Phi}}}_{m}\right\} $中的最小值,将$ {\mathit{\pmb{\Phi}}}_{o} $代入式(21)中得到权值:

$ \underset{q\to {1}^{+}}{\mathrm{l}\mathrm{i}\mathrm{m}}{\mathit{\boldsymbol{\alpha}}}_{0}=\underset{q\to {1}^{+}}{\mathrm{l}\mathrm{i}\mathrm{m}}\frac{1}{1+\sum\limits_{s\ne o}{\left(\frac{{\mathit{\pmb{\Phi}}}_{s}}{{\mathit{\pmb{\Phi}}}_{o}}\right)}^{\frac{1}{1-q}}}=1 $ (28)

由此可见,KMFC算法将权重1赋给$ {\mathit{\pmb{\Phi}}}_{o} $最小的视图,将权重0赋给其他视图。通过该方法可以保证KMFC算法在$ q > 1 $时不存在平凡解。

2.2 收敛性分析

定理1  在每次迭代中,问题式(2)的目标函数值不断减小,直到算法收敛。

证明  假设经过第$ t $次迭代得到$ {\mathit{\boldsymbol{U}}}^{\left(t\right)} $$ {\mathit{\boldsymbol{V}}}^{\left(t\right)} $$ {\mathit{\boldsymbol{\alpha}}}^{\left(t\right)} $$ {\mathit{\boldsymbol{U}}}^{\mathrm{*}\left(t\right)} $。在第$ t+1 $次迭代中,首先将$ \mathit{\boldsymbol{V}} $$ \mathit{\boldsymbol{\alpha}} $$ {\mathit{\boldsymbol{U}}}^{\mathrm{*}} $分别固定为$ {\mathit{\boldsymbol{V}}}^{\left(t\right)} $$ {\mathit{\boldsymbol{\alpha}}}^{\left(t\right)} $$ {\mathit{\boldsymbol{U}}}^{\mathrm{*}\left(t\right)} $,然后在不同的视图之间求解$ {\mathit{\boldsymbol{U}}}^{\left(t+1\right)} $。根据问题式(8),$ {\mathit{\boldsymbol{U}}}^{\left(t+1\right)} $可由式(29)求解:

$ {\mathit{\boldsymbol{u}}}_{ik}^{\left(t+1\right)\left(p\right)}=\frac{\mathrm{e}\mathrm{x}\mathrm{p}\left(\frac{-\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{h}}}_{ik}^{\left(t\right)\left(p\right)}}{\lambda }\right){\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}\left(t\right)}}{\sum\limits_{k=1}^{c}\left[\mathrm{e}\mathrm{x}\mathrm{p}\left(\frac{-\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{h}}}_{ik}^{\left(t\right)\left(p\right)}}{\lambda }\right){\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}\left(t\right)}\right]} $ (29)

其中:上述目标函数和约束在$ {\mathit{\boldsymbol{u}}}_{i}^{\left(p\right)} $域中是凸的。这样问题式(29)就会收敛到全局最优解。

同样地,在第$ t+1 $次迭代中,将$ \mathit{\boldsymbol{U}} $$ \mathit{\boldsymbol{\alpha}} $$ {\mathit{\boldsymbol{U}}}^{\mathrm{*}} $分别固定为$ {\mathit{\boldsymbol{U}}}^{\left(t\right)} $$ {\mathit{\boldsymbol{\alpha}}}^{\left(t\right)} $$ {\mathit{\boldsymbol{U}}}^{\mathrm{*}\left(t\right)} $,根据问题式(12),$ {\mathit{\boldsymbol{V}}}^{\left(t+1\right)} $可由式(30)求解:

$ {\mathit{\boldsymbol{v}}}_{k}^{\left(t+1\right)\left(p\right)}=\frac{\sum\limits_{i=1}^{n}{\mathit{\boldsymbol{u}}}_{ik}^{\left(t\right)\left(p\right)}{\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}}{\sum\limits_{i=1}^{n}{\mathit{\boldsymbol{u}}}_{ik}^{\left(t\right)\left(p\right)}} $ (30)

其中:上述目标函数和约束在$ {\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)} $域中是凸的。这样问题式(30)就会收敛到全局最优解。

在第$ t+1 $次迭代中,将$ \mathit{\boldsymbol{U}} $$ \mathit{\boldsymbol{V}} $$ {\mathit{\boldsymbol{U}}}^{\mathrm{*}} $分别固定为$ {\mathit{\boldsymbol{U}}}^{\left(t\right)} $$ {\mathit{\boldsymbol{V}}}^{\left(t\right)} $$ {\mathit{\boldsymbol{U}}}^{\mathrm{*}\left(t\right)} $,根据问题式(21),$ {\mathit{\boldsymbol{\alpha}}}^{\left(t+1\right)} $可由式(31)求解:

$ {\mathit{\boldsymbol{\alpha}}}_{p}^{\left(t+1\right)}=\frac{{\mathit{\pmb{\Phi}}}_{p}^{\left(t\right)\frac{1}{1-q}}}{\sum\limits_{s=1}^{m}{\mathit{\pmb{\Phi}}}_{s}^{\left(t\right)\frac{1}{1-q}}} $ (31)

其中:上述目标函数和约束在$ {\mathit{\boldsymbol{\alpha}}}_{p} $域中是凸的。这样问题式(31)就会收敛到全局最优解。

在第$ t+1 $次迭代中,将$ \mathit{\boldsymbol{U}} $$ \mathit{\boldsymbol{V}} $$ \mathit{\boldsymbol{\alpha}} $分别固定为$ {\mathit{\boldsymbol{U}}}^{\left(t\right)} $$ {\mathit{\boldsymbol{V}}}^{\left(t\right)} $$ {\mathit{\boldsymbol{\alpha}}}^{\left(t\right)} $,根据问题式(27),$ {\mathit{\boldsymbol{U}}}^{\mathrm{*}\left(t+1\right)} $可由式(32)求解:

$ {\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}\left(t+1\right)}=\frac{\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{u}}}_{ik}^{\left(t\right)\left(p\right)}}{\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{u}}}_{ik}^{\left(t\right)\left(p\right)}} $ (32)

其中:上述目标函数和约束在$ {\mathit{\boldsymbol{u}}}_{i}^{\mathrm{*}} $域中是凸的。这样问题式(32)就会收敛到全局最优解。

显然,问题式(2)可以分为4个子问题,每个子问题都是一个凸优化问题。因此,上述证明过程验证了算法1的收敛性。

3 实验与结果分析

根据聚类精度(Accuracy,ACC)[30]、标准化互信息(Normalized Mutual Information,NMI)[30]、相似性(Jaccard)[31]、纯度(Purity)[31]4个评价指标对KMFC算法进行性能评价。

3.1 数据集

在5个真实数据集上评估KMFC算法的性能:

1)COIL20数据集[32]具有1 440张标准化图像,包含20个对象,每个对象对应72张图像。每张图像可以用30个等距投影(Isometric,ISO)、19个线性判别分析(Linear Discriminant Analysis,LDA)和30个邻域保持嵌入(Neighborhood Preserving Embedding,NPE)这3类异构特征来表示。

2)YALE数据集[33]由15名受试者的165张图像组成,每个受试者有11张图像,对应不同的面部表情或形态。每张图像可以用30个ISO、14个LDA和30个NPE这3类异构特征来表示。

3)3-Sources数据集[34]包含294篇新闻文章,涵盖商业、娱乐、健康、政治、体育和技术6个方面。第一视图有3 068个特征,第二视图有3 631个特征,第三视图有3 560个特征。

4)NUS-WIDE数据集[35]由包含81个对象的269 648张图像组成。在实验中选择猫、牛、狗、麋鹿、鹰、马、狮子、松鼠、老虎、鲸鱼、狼、斑马12种动物类别。每张图像可以用64种颜色直方图、144种颜色相关图、73种边缘方向直方图、128个小波纹理、225个块状颜色矩和500袋基于SIFT描述的单词这6类低级特征来表示。

5)Prokaryotic phyla数据集[36]包含551个原核生物种类,包括文本数据和不同的基因组表达。本文选用的1个视图数据为由描述原核生物种类的文档词袋表示组成的文本数据,另外2个视图数据为2种基因组表示[17]。与文献[17]一样,为降低数据集维数,对3个视图分别应用主成分分析(Principal Component Analysis,PCA)并保留解释90%方差的主成分。

数据集统计信息如表 1所示。

下载CSV 表 1 数据集统计信息 Table 1 Dataset statistics
3.2 评价指标

对于ACC、NMI、Jaccard、Purity这些广泛使用的指标,值越大表示集群性能越好。

ACC表示聚类精度,定义如下:

$ {A}_{\mathrm{A}\mathrm{C}\mathrm{C}}=\frac{\sum\limits_{i=1}^{n}\delta \left({\tau }_{i}, \mathrm{m}\mathrm{a}\mathrm{p}\left({r}_{i}\right)\right)}{n} $ (33)

其中:$ n $为样本点的个数;$ {\tau }_{i} $为第$ i $个样本点真实的类标签;$ {r}_{i} $为学习到的第$ i $个样本点对应的类标签;$ \delta \left(x, y\right) $定义为一个函数,当$ x=y $$ \delta \left(x, y\right)=1 $,否则为0;$ \mathrm{m}\mathrm{a}\mathrm{p}\left({r}_{i}\right) $是一个映射函数,它将学习到的标签$ {r}_{i} $与真实标签$ {\tau }_{i} $进行匹配。

NMI表示$ {\tau }_{i} $$ {r}_{i} $之间的相似程度,定义如下:

$ {N}_{\mathrm{N}\mathrm{M}\mathrm{I}}\left({\tau }_{i}, {r}_{i}\right)=\frac{\sum\limits_{i=1}^{c}\sum\limits_{j=1}^{c}\left({n}_{i, j}\mathrm{l}\mathrm{g}\frac{{n}_{i, j}}{{n}_{i}{n}_{j}}\right)}{\sqrt{\left(\sum\limits_{i=1}^{c}{n}_{i}\mathrm{l}\mathrm{g}\frac{{n}_{i}}{n}\right)\left(\sum\limits_{j=1}^{c}{\widehat{n}}_{j}\mathrm{l}\mathrm{g}\frac{{\widehat{n}}_{j}}{n}\right)}} $ (34)

其中:$ {n}_{i} $表示算法中每一类$ {r}_{i}\left(1\le i\le c\right) $包含的样本点的个数;$ {\widehat{n}}_{j} $表示算法中每一类$ {\tau }_{j}\left(1\le j\le c\right) $包含的样本点的个数;$ {n}_{i, j} $表示学习到的第$ i $个样本点对应的类标签$ {r}_{i} $和真实的类标签$ {\tau }_{j} $的交集中所包含的样本点的个数。

Jaccard度量有限样本集N1N2之间的相似性,定义如下:

$ {J}_{\mathrm{J}\mathrm{a}\mathrm{c}\mathrm{c}\mathrm{a}\mathrm{r}\mathrm{d}}=\frac{\left|{N}_{1}\bigcap {N}_{2}\right|}{\left|{N}_{1}\bigcup {N}_{2}\right|}=\frac{{T}_{\mathrm{T}\mathrm{P}}}{{T}_{\mathrm{T}\mathrm{P}}+{F}_{\mathrm{F}\mathrm{P}}+{F}_{\mathrm{F}\mathrm{N}}} $ (35)

其中:$ {T}_{\mathrm{T}\mathrm{P}} $是真阳性的数目;$ {F}_{\mathrm{F}\mathrm{P}} $是假阳性的数目;$ {F}_{\mathrm{F}\mathrm{N}} $是假阴性的数目。

Purity是正确类标签的百分比,定义如下:

$ {P}_{\mathrm{P}\mathrm{u}\mathrm{r}\mathrm{i}\mathrm{t}\mathrm{y}}=\frac{1}{n}\sum\limits_{i=1}^{c}\underset{1\le j\le c}{\mathrm{m}\mathrm{a}\mathrm{x}}\left|\mathrm{m}\mathrm{a}\mathrm{p}\left({r}_{i}\right)\bigcap {\tau }_{j}\right| $ (36)

Purity的取值范围为$ \left[\mathrm{0, 1}\right] $,值越靠近1,性能越好。

3.3 实验设置

首先,比较KMFC算法与FCM算法和熵模糊C均值(Entropy Fuzzy C-Means,EFCM)算法的性能,以证明KMFC算法体现了多视图算法的优势。其次,将KMFC算法与SWMC[37]、MVASM[38]、MVGL[39]3种算法进行比较,其中比较算法中涉及的参数调整参照原文献进行设置并实验以显示最优结果。在KMFC算法中,参数$ q $控制不同视图上权值的分配,参数$ \lambda $调节共识隶属度矩阵的稀疏性。根据式(21)在$ \left[\mathrm{1, 30}\right] $内以步长为0.01来调节$ q $,在$ \left[\mathrm{0, 10}\right] $内以步长0.1来调节$ \lambda $,在3-Source、NUS-WIDE和Prokaryotic phyla数据集上设置$ \left(q, \lambda \right) $分别为(1.22,0.9)、(1.30,0.4)和(1.02,0.3)时KMFC算法的评价指标变化,如图 1~图 3所示。

Download:
图 1 KMFC算法在3-Source、NUS-WIDE和Prokaryotic phyla数据集上的ACC比较 Fig. 1 ACC comparation of KMFC algorithm on 3-Source, NUS-WIDE and Prokaryotic phyla datasets
Download:
图 2 KMFC算法在3-Source、NUS-WIDE和Prokaryotic phyla数据集上的NMI比较 Fig. 2 NMI comparation of KMFC algorithm on 3-Source, NUS-WID and Prokaryotic phyla datasets
Download:
图 3 KMFC算法在3-Source、NUS-WIDE和Prokaryotic phyla数据集上的Purity比较 Fig. 3 Purity comparation of KMFC on 3-Source, NUS-WID and Prokaryotic phyla datasets
3.4 实验结果

在COIL20和YALE32数据集上,KMFC和2种单视图聚类算法FCM、EFCM的ACC、NMI、Jaccard和Purity比较结果如表 2表 3所示,其中括号中的数字表示算法排名。从表 2表 3可以看出:在ACC、NMI、Jaccard和Purity 4个评价指标上,KMFC算法在COIL20数据集上相比FCM(2)算法分别提高0.584 9、0.375 5、0.692 4、0.687 4,相比EFCM(2)算法分别提高0.589 9、0.383 6、0.702 0、0.698 5;在YALE32数据集上,KMFC算法相比FCM(2)算法分别提高0.471 6、303 4、617 5、610 8,相比EFCM(2)算法分别提高0.506 7、0.313 4、0.628 4、0.625 3。上述结果证明了多视图学习的优越性和有效性,并且其有利于提高聚类精度。

下载CSV 表 2 在COIL20数据集上KMFC与2种单视图聚类算法的ACC、NMI、Jaccard和Purity比较 Table 2 Comparison of ACC, NMI, Jaccard and Purity between KMFC and two single-view clustering algorithms on COIL20 dataset
下载CSV 表 3 在YALE32数据集上KMFC与2种单视图聚类算法的ACC、NMI、Jaccard和Purity比较 Table 3 Comparison of ACC, NMI, Jaccard and Purity between KMFC and two single-view clustering algorithms on YALE32 dataset

在3-Source、NUS-WIDE和Prokaryotic phyla数据集上不同多视图聚类算法的ACC、NMI和Purity比较结果如表 4~表 6所示。从表 4~表 6可以看出:在数据集3-Sources上,KMFC算法的ACC、NMI、Purity相比次优的MVASM算法分别提高0.074 6、0.153 4、0.054 8;在数据集NUS-WIDE上,KMFC算法相比次优的MVASM算法分别提高0.057 4、0.002 4、0.006 7;在数据集Prokaryotic phyla上,KMFC算法相比次优的MVGL算法分别提高0.060 9、0.062 6、0.000 7。KMFC算法在3个数据集上的收敛曲线如图 4所示。可见,KMFC算法的聚类性能明显优于对比算法。

下载CSV 表 4 在3-Source数据集上不同多视图聚类算法的ACC、NMI和Purity比较 Table 4 Comparison of ACC, NMI and Purity of different multi-view clustering algorithms on 3-Source dataset
下载CSV 表 5 在NUS-WIDE数据集上不同多视图聚类算法的ACC、NMI和Purity比较 Table 5 Comparison of ACC, NMI and Purity of different multi-view clustering algorithms on NUS-WIDE dataset
下载CSV 表 6 在Prokaryotic phyla数据集上不同多视图聚类算法的ACC、NMI和Purity比较 Table 6 Comparison of ACC, NMI and Purity of different multi-view clustering algorithms on Prokaryotic phyla dataset
Download:
图 4 在3-Sources、NUS-WIDE、Prokaryotic phyla数据集上KMFC算法的收敛曲线 Fig. 4 Convergence curves of KMFC algorithm on 3-Sources, NUS-WIDE, Prokaryotic phyla datasets
4 结束语

本文提出一种新的多视图加权聚类算法,将每个视图信息及其权重进行拟合融入标准模糊C均值聚类算法,再附加一个KL信息作为模糊正则项,其中KL信息是一个视图的隶属度与其共识隶属度的比值,因此最小化KL信息会使每个视图的隶属度偏向于共识隶属度,最终实现对共识隶属度矩阵的聚类。在多个数据集上的实验结果证明了该算法的有效性。但该算法中的权重需要引入幂指数$ q $来进行调节,其细微变化即可影响算法性能,因此下一步将设计并实现融合KL信息的多视图完全自加权模糊聚类算法。

参考文献
[1]
XU C, TAO D C, XU C. A survey on multi-view learning[EB/OL]. [2021-05-11]. https://arxiv.org/abs/1304.5634.
[2]
WEN J, ZHANG Z, XU Y, et al. Unified embedding alignment with missing views inferring for incomplete multi-view clustering[C]//Proceedings of the 33rd AAAI Conference on Artificial Intelligence and the 31st Innovative Applications of Artificial Intelligence Conference and the 9th AAAI Symposium on Educational Advances in Artificial Intelligence. Palo Alto, USA: AAAI Press, 2019: 5393-5400.
[3]
ZONG L L, ZHANG X C, LIU X Y, et al. Weighted multi-view spectral clustering based on spectral perturbation[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2018: 4621-4629.
[4]
JOHNSON S C. Hierarchical clustering schemes[J]. Psychometrika, 1967, 32(3): 241-254. DOI:10.1007/BF02289588
[5]
LEE J W T, YEUNG D S, TSANG E C C. Hierarchical clustering based on ordinal consistency[J]. Pattern Recognition, 2005, 38(11): 1913-1925. DOI:10.1016/j.patcog.2005.05.008
[6]
WU K L, YANG M S. Alternative c-means clustering algorithms[J]. Pattern Recognition, 2002, 35(10): 2267-2278. DOI:10.1016/S0031-3203(01)00197-2
[7]
ZHANG D Q, CHEN S C. A comment on "alternative c-means clustering algorithms"[J]. Pattern Recognition, 2004, 37(2): 173-174. DOI:10.1016/j.patcog.2003.08.001
[8]
TSENG P. Nearest q-flat to m points[J]. Journal of Optimization Theory and Applications, 2000, 105(1): 249-252. DOI:10.1023/A:1004678431677
[9]
WANG Y, ZHANG W J, WU L, et al. Iterative views agreement: an iterative low-rank based structured optimization method to multi-view spectral clustering[C]//Proceedings of the 25th International Joint Conference on Artificial Intelligence. Washington D. C., USA: IEEE Press, 2016: 2153-2159.
[10]
PEDRYCZ W. Collaborative fuzzy clustering[J]. Pattern Recognition Letters, 2002, 23(14): 1675-1686. DOI:10.1016/S0167-8655(02)00130-7
[11]
CORNUÉJOLS A, WEMMERT C, GANÇARSKI P, et al. Collaborative clustering: why, when, what and how[J]. Information Fusion, 2018, 39: 81-95. DOI:10.1016/j.inffus.2017.04.008
[12]
COSTEIRA J P, KANADE T. A multi-body factorization method for independently moving objects[J]. International Journal of Computer Vision, 1998, 29(3): 159-179. DOI:10.1023/A:1008000628999
[13]
LIU Y Y, JIAO L C, SHANG F H. An efficient matrix factorization based low-rank representation for subspace clustering[J]. Pattern Recognition, 2013, 46(1): 284-292. DOI:10.1016/j.patcog.2012.06.011
[14]
WANG X B, LEI Z, SHI H L, et al. Co-referenced subspace clustering[C]//Proceedings of IEEE International Conference on Multimedia and Expo. Washington D. C., USA: IEEE Press, 2018: 1-6.
[15]
GUO X J. Robust subspace segmentation by simultaneously learning data representations and their affinity matrix[C]//Proceedings of the 24th International Conference on Artificial Intelligence. Washington D. C., USA: IEEE Press, 2015: 3547-3553.
[16]
WANG X B, GUO X J, LEI Z, et al. Exclusivity-consistency regularized multi-view subspace clustering[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 1-9.
[17]
BRBIĆ M, KOPRIVA I. Multi-view low-rank sparse subspace clustering[J]. Pattern Recognition, 2018, 73: 247-258. DOI:10.1016/j.patcog.2017.08.024
[18]
ASUR S, UCAR D, PARTHASARATHY S. An ensemble framework for clustering protein-protein interaction networks[J]. Bioinformatics, 2007, 23(13): 29-40. DOI:10.1093/bioinformatics/btm212
[19]
WANG H J, SHAN H H, BANERJEE A. Bayesian cluster ensembles[J]. Statistical Analysis and Data Mining, 2011, 4(1): 54-70. DOI:10.1002/sam.10098
[20]
CHAUDHURI K, KAKADE S M, LIVESCU K, et al. Multi-view clustering via canonical correlation analysis[C]//Proceedings of the 26th Annual International Conference on Machine Learning. New York, USA: ACM Press, 2009: 129-136.
[21]
NIU D L, JENNIFER G D, JORDAN M I. Multiple non-redundant spectral clustering views[C]//Proceedings of the 27th International Conference on Machine Learning. New York, USA: ACM Press, 2010: 831-838.
[22]
CAI X, NIE F P, HUANG H, et al. Heterogeneous image feature integration via multi-modal spectral clustering[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2011: 1977-1984.
[23]
KUMAR A, RAI P, DAUME H. Co-regularized multi-view spectral clustering[C]//Proceedings of the 24th International Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2011, 24: 1413-1421.
[24]
CAI X, NIE F P, HUANG H. Multi-view k-means clustering on big data[C]//Proceedings of the 23rd International Joint Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2013: 2598-2604.
[25]
WANG H, NIE F P, HUANG H. Multi-view clustering and feature learning via structured sparsity[J]. Journal of Machine Learning Research, 2013, 28(3): 352-360.
[26]
洪敏, 贾彩燕, 王晓阳. K-means型多视图聚类中的初始化问题研究[J]. 计算机科学与探索, 2019, 13(4): 574-585.
HONG M, JIA C Y, WANG X Y. Research on initialization of K-means type multi-view clustering[J]. Journal of Frontiers of Computer Science and Technology, 2019, 13(4): 574-585. (in Chinese)
[27]
XIA R, PAN Y, DU L, et al. Robust multi-view spectral clustering via low-rank and sparse decomposition[C]//Proceedings of the 28th AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2014: 2149-2155.
[28]
CLEUZIOU G, EXBRAYAT M, MARTIN L, et al. CoFKM: a centralized method for multiple-view clustering[C]//Proceedings of the 9th IEEE International Conference on Data Mining. Washington D. C., USA: IEEE Press, 2009: 752-757.
[29]
JIANG Y Z, CHUNG F L, WANG S T, et al. Collaborative fuzzy clustering from multiple weighted views[J]. IEEE Transactions on Cybernetics, 2015, 45(4): 688-701. DOI:10.1109/TCYB.2014.2334595
[30]
CAI D, HE X, HAN J, et al. Document clustering using locality preserving indexing[J]. IEEE Transactions on Knowledge and Data Engineering, 2005, 17(12): 1624-1637. DOI:10.1109/TKDE.2005.198
[31]
CHEN G H, PAN Y, GUO M Y, et al. COMPACT: a comparative package for clustering assessment[C]//Proceedings of 2005 International Conference on Parallel and Distributed Processing and Applications. Berlin, Germany: Springer, 2005: 159-167.
[32]
NAYAR S. Columbia Object Image Library(COIL20)[EB/OL]. [2021-05-11]. https://www.researchgate.net/publication/2784735_Columbia_Object_Image_Library_COIL-100.
[33]
CAI D, HE X F, HAN J W. Using graph model for face analysis[EB/OL]. [2021-05-11]. https://www.semanticscholar.org/paper/Using-Graph-Model-for-Face-Analysis-Cai-He/19c889f2b26b785f0ac45c427339f0335b9cc514?p2df.
[34]
GREENE D, CUNNINGHAM P. A matrix factorization approach for integrating multiple data views[M]. Berlin, Germany: Springer, 2009.
[35]
CHUA T S, TANG J H, HONG R C, et al. NUS-WIDE: a real-world Web image database from National University of Singapore[C]//Proceedings of the ACM International Conference on Image and Video Retrieval. New York, USA: ACM Press, 2009: 1-9.
[36]
BRBIĆ M, PIŠKOREC M, VIDULIN V, et al. The landscape of microbial phenotypic traits and associated genes[J]. Nucleic Acids Research, 2016, 44(21): 10074-10090.
[37]
NIE F P, LI J, LI X L. Self-weighted multiview clustering with multiple graphs[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence. Melbourne, Australia: International Joint Conferences on Artificial Intelligence Organization, 2017: 2564-2570.
[38]
HAN J W, XU J L, NIE F P, et al. Multi-view K-means clustering with adaptive sparse memberships and weight allocation[EB/OL]. [2021-05-11]. https://www.researchgate.net/publication/340572521_Multi-view_K-Means_Clustering_with_Adaptive_Sparse_Memberships_and_Weight_Allocation.
[39]
ZHAN K, ZHANG C Q, GUAN J P, et al. Graph learning for multiview clustering[J]. IEEE Transactions on Cybernetics, 2018, 48(10): 2887-2895. DOI:10.1109/TCYB.2017.2751646