开放科学(资源服务)标志码(OSID):
聚类是一种无监督的机器学习任务,根据数据自身的距离或相似度将它们划分为若干组,划分原则是组内距离最小化而组间距离最大化[1-3]。常见的聚类算法主要包括基于层次[4-5]、基于模糊C均值(Fuzzy C-Means,FCM)[6-7]、基于迭代[8-9]、基于协作[10-11],基于分解[12-14]、基于谱聚类集群[15-17]等,其中基于
近些年来,学者们在多视图聚类算法研究方面取得了较大进展。文献[20]通过典型相关分析使用数据的多个视图来构建投影。文献[21]学习了同时为原始问题提供多个视图的非冗余子空间,并在每个视图中找到一个聚类方法。文献[22]提出一种类似将异构图像特征与图相结合的多视角光谱聚类算法。文献[23]引入共正则化谱聚类,采用共正则化框架来解决多视图聚类问题。文献[24]提出解决大规模多视图数据的多视图K-means聚类算法。文献[25]提出同时进行特征选择和多视图聚类的结构化稀疏学习方法。文献[26]分析并研究了基于采样的主动式初始中心选择方法对K-means型多视图聚类算法的影响。虽然上述算法取得了较好的性能,但其中一些算法忽略了视图多样性。文献[27]通过人为干预或先验知识实现多视图加权,但不能保证最终结果与每个视图的贡献相一致。文献[28]提出一种基于FCM的多视角模糊聚类方法CoFKM,但由于每个视图被平等对待,因此没有考虑每个视图的权重。文献[29]指出在某些视图有噪声或存在干扰的情况下,算法聚类精度可能会降低。受现有研究的启发,本文提出一种新的多视图自加权聚类算法KMFC。该算法通过学习共识隶属度矩阵进行聚类表示来挖掘多个视图的潜在共识信息,求解多个隶属度矩阵和质心矩阵,并利用视图的特定信息进一步修正共识隶属度矩阵。
1 模型建立与求解 1.1 模型建立建立多视图聚类模型,通过引入Kullback-Leibler(KL)信息使学习的矩阵
$ \begin{array}{l}\mathrm{m}\mathrm{i}\mathrm{n}\sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{‖{\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}-{\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}‖}_{2}^{2}\right)+\\ \lambda \sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}\mathrm{l}\mathrm{g}\frac{{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}{{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}\right)\end{array} $ | (1) |
在提高视图内聚类结果时,需要考虑不同视图间的聚类一致性。若直接将多个视图拼接在一起,则不利于提高聚类性能。更合理的方法是将这些视图与合适的权重
$ \begin{array}{l}\mathrm{m}\mathrm{i}\mathrm{n}\sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}{‖{\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}-{\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}‖}_{2}^{2}\right)+\\ \lambda \sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}\mathrm{l}\mathrm{g}\frac{{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}{{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}\right)\\ \mathrm{s}.\mathrm{t}.\sum\limits_{k=1}^{c}{\mathit{\boldsymbol{u}}}_{ik}=1, {\mathit{\boldsymbol{u}}}_{ik}\ge 0, \sum\limits_{p=1}^{m}{\mathit{\boldsymbol{\alpha}}}_{p}=1, {\mathit{\boldsymbol{\alpha}}}_{p}\ge 0, \\ \sum\limits_{k=1}^{c}{\mathit{\boldsymbol{u}}}_{ik}^{*}=1, {\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}\ge 0, i=\mathrm{1, 2}, \cdots , n\end{array} $ | (2) |
其中:
由于每个视图都有各自的属性信息,因此KMFC算法可以使视图内信息与公共信息进行较好的拟合,其中视图内信息来自
将多视图聚类模型的目标函数分为4个子问题,通过迭代交替优化方法进行求解。
1)固定
$ \begin{array}{l}\underset{\sum\limits_{k=1}^{c}{\mathit{\boldsymbol{u}}}_{ik}=1, {\mathit{\boldsymbol{u}}}_{ik}\ge 0}{\mathrm{m}\mathrm{i}\mathrm{n}}\sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}{‖{\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}-{\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}‖}_{2}^{2}\right)+\\ \lambda \sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}\mathrm{l}\mathrm{g}\frac{{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}{{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}\right)\iff \\ \underset{\sum\limits_{k=1}^{c}{\mathit{\boldsymbol{u}}}_{ik}=1, {\mathit{\boldsymbol{u}}}_{ik}\ge 0}{\mathrm{m}\mathrm{i}\mathrm{n}}\sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{h}}}_{ik}^{\left(p\right)}\right)+\\ \lambda \sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}\mathrm{l}\mathrm{g}\frac{{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}{{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}\right)\end{array} $ | (3) |
其中:
式(3)的拉格朗日函数表示如下:
$ \begin{array}{l}L\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}, \eta \right)=\sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{h}}}_{ik}^{\left(p\right)}\right)+\\ \lambda \sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}\mathrm{l}\mathrm{g}\frac{{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}{{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}\right)+\sum\limits_{i=1}^{n}{\eta }_{i}\left(\sum\limits_{k=1}^{c}{\mathit{\boldsymbol{u}}}_{ik}-1\right)\end{array} $ | (4) |
其中:
对式(4)中的
$ \frac{\partial L}{\partial {\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}=\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{h}}}_{ik}^{\left(p\right)}+{\eta }_{i}+\lambda \sum\limits_{p=1}^{m}\left(\mathrm{l}\mathrm{g}\frac{{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}{{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}+1\right)=0 $ | (5) |
$ {\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}=\mathrm{e}\mathrm{x}\mathrm{p}\left(\frac{-\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{h}}}_{ik}^{\left(p\right)}}{\lambda }\right){\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}\mathrm{e}\mathrm{x}\mathrm{p}\left(\frac{-{\eta }_{i}-\lambda }{\lambda }\right) $ | (6) |
考虑等式约束
$ \begin{array}{l} \sum\limits_{k=1}^{c}\left[\mathrm{e}\mathrm{x}\mathrm{p}\left(\frac{-\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{h}}}_{ik}^{\left(p\right)}}{\lambda }\right){\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}\mathrm{e}\mathrm{x}\mathrm{p}\left(\frac{-{\eta }_{i}-\lambda }{\lambda }\right)\right]=1\Rightarrow \\ \mathrm{e}\mathrm{x}\mathrm{p}\left(\frac{-{\eta }_{i}-\lambda }{\lambda }\right)=\frac{1}{\sum\limits_{k=1}^{c}\left[\mathrm{e}\mathrm{x}\mathrm{p}\left(\frac{-\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{h}}}_{ik}^{\left(p\right)}}{\lambda }\right){\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}\right]} \end{array} $ | (7) |
结合式(6)和式(7)得到:
$ {\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}=\frac{\mathrm{e}\mathrm{x}\mathrm{p}\left(\frac{-\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{h}}}_{ik}^{\left(p\right)}}{\lambda }\right){\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}{\sum\limits_{k=1}^{c}\left[\mathrm{e}\mathrm{x}\mathrm{p}\left(\frac{-\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{h}}}_{ik}^{\left(p\right)}}{\lambda }\right){\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}\right]} $ | (8) |
2)固定
$ \begin{array}{l}\mathrm{m}\mathrm{i}\mathrm{n}\sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}{‖{\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}-{\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}‖}_{2}^{2}\right)+\\ \lambda \sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}\mathrm{l}\mathrm{g}\frac{{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}{{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}\right)\iff \\ \underset{V}{\mathrm{m}\mathrm{i}\mathrm{n}}\sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}{‖{\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}-{\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}‖}_{2}^{2}\right)\end{array} $ | (9) |
使用J表示问题式(9)的目标函数,得到:
$ J=\sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left(\begin{array}{l}{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}{\left({\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}\right)}^{\mathrm{T}}{\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}+{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}{\left({\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}\right)}^{\mathrm{T}}{\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}-\\ 2{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}{\left({\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}\right)}^{\mathrm{T}}{\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}\end{array}\right) $ | (10) |
利用式(10)对
$ \frac{\partial J\left({\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}\right)}{\partial {\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}}=\sum\limits_{i=1}^{n}\left(2{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}{\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}-2{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}{\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}\right)=0 $ | (11) |
$ {\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}=\frac{\sum\limits_{i=1}^{n}{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}}{\sum\limits_{i=1}^{n}{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}} $ | (12) |
3)固定
$ \begin{array}{l}\underset{\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{\alpha}}}_{p}=1, {\mathit{\boldsymbol{\alpha}}}_{p}\ge 0}{\mathrm{m}\mathrm{i}\mathrm{n}}\sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}{‖{\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}-{\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}‖}_{2}^{2}\right)+\\ \lambda \sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}\mathrm{l}\mathrm{g}\frac{{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}{{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}\right)\iff \\ \underset{\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{\alpha}}}_{p}=1, {\mathit{\boldsymbol{\alpha}}}_{p}\ge 0}{\mathrm{m}\mathrm{i}\mathrm{n}}\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}\left(\sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{‖{\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}-{\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}‖}_{2}^{2}\right)\iff \\ \underset{\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{\alpha}}}_{p}=1, {\mathit{\boldsymbol{\alpha}}}_{p}\ge 0}{\mathrm{m}\mathrm{i}\mathrm{n}}\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}{\mathit{\pmb{\Phi}}}_{p}\end{array} $ | (13) |
其中:
式(13)的拉格朗日函数表示如下:
$ L\left(\mathit{\boldsymbol{\alpha}}, \mathit{\pmb{\gamma}}, \mathit{\pmb{\beta}}\right)=\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}{\mathit{\pmb{\Phi}}}_{p}-\mathit{\pmb{\gamma}}\left({\mathit{\boldsymbol{\alpha}}}^{\mathrm{{\rm T}}}1-1\right)-{\mathit{\pmb{\beta}}}^{\mathrm{{\rm T}}}\mathit{\boldsymbol{\alpha}} $ | (14) |
其中:
利用式(14)对
$ {\mathit{\boldsymbol{\alpha}}}_{p}^{q-1}=\frac{\mathit{\pmb{\gamma}}}{q{\mathit{\pmb{\Phi}}}_{p}}+\frac{{\mathit{\pmb{\beta}}}_{p}}{q{\mathit{\pmb{\Phi}}}_{p}} $ | (15) |
问题式(13)的KKT条件表示如下:
$ \mathit{\pmb{\beta}}\ge 0 $ | (16) |
$ \mathit{\boldsymbol{\alpha}}\ge 0 $ | (17) |
$ {\mathit{\boldsymbol{\alpha}}}^{\mathrm{{\rm T}}}\mathit{\pmb{\beta}}=0 $ | (18) |
结合式(16)~式(18)得到
$ {\mathit{\boldsymbol{\alpha}}}_{p}={\left(\frac{\mathit{\pmb{\gamma}}}{q{\mathit{\pmb{\Phi}}}_{p}}\right)}^{\frac{1}{q-1}} $ | (19) |
考虑等式约束
$ \sum\limits_{p=1}^{m}{\left(\frac{\mathit{\pmb{\gamma}}}{q{\mathit{\pmb{\Phi}}}_{p}}\right)}^{\frac{1}{q-1}}=1\Rightarrow {\left(\frac{\mathit{\pmb{\gamma}}}{q}\right)}^{\frac{1}{q-1}}=\frac{1}{\sum\limits_{p=1}^{m}{\mathit{\pmb{\Phi}}}_{p}^{\frac{1}{1-q}}} $ | (20) |
结合式(19)与式(20)得到:
$ {\mathit{\boldsymbol{\alpha}}}_{p}=\frac{{\mathit{\pmb{\Phi}}}_{p}^{\frac{1}{1-q}}}{\sum\limits_{s=1}^{m}{\mathit{\pmb{\Phi}}}_{s}^{\frac{1}{1-q}}} $ | (21) |
4)固定
$ \begin{array}{l}\underset{\sum\limits_{k=1}^{c}{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}=1, {\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}\ge 0}{\mathrm{m}\mathrm{i}\mathrm{n}}\sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}{\mathit{\boldsymbol{\alpha}}}_{p}^{\left(q\right)}{‖{\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}-{\mathit{\boldsymbol{v}}}_{k}^{\left(p\right)}‖}_{2}^{2}\right)+\\ \lambda \sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}\mathrm{l}\mathrm{g}\frac{{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}{{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}\right)\iff \\ \underset{\sum\limits_{k=1}^{c}{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}=1, {\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}\ge 0}{\mathrm{m}\mathrm{i}\mathrm{n}}\sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}\mathrm{l}\mathrm{g}\frac{{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}{{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}\right)\end{array} $ | (22) |
式(22)的拉格朗日函数表示如下:
$ \begin{array}{l}L\left({\mathit{\boldsymbol{u}}}_{ik}, \mathit{\pmb{\xi}}\right)=\sum\limits_{p=1}^{m}\sum\limits_{i=1}^{n}\sum\limits_{k=1}^{c}\left({\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}\mathrm{l}\mathrm{g}\frac{{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}{{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}\right)+\\ \sum\limits_{i=1}^{n}{\mathit{\pmb{\xi}}}_{i}\left(\sum\limits_{k=1}^{c}{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}-1\right)\end{array} $ | (23) |
其中:
对式(23)中的
$ \frac{\partial L}{\partial {\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}=\sum\limits_{p=1}^{m}\left(-\frac{{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}{{\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}}+1\right)+{\mathit{\pmb{\xi}}}_{i}=0 $ | (24) |
$ {\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}=\frac{1}{{\mathit{\pmb{\xi}}}_{i}}\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)} $ | (25) |
考虑等式约束
$ \sum\limits_{k=1}^{c}\frac{1}{{\mathit{\pmb{\xi}}}_{i}}\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}=1\Rightarrow {\mathit{\pmb{\xi}}}_{i}=\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)} $ | (26) |
结合式(25)和式(26)得到:
$ {\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}}=\frac{\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}}{\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{u}}}_{ik}^{\left(p\right)}} $ | (27) |
综上所述,通过问题1的求解可更新
算法1 KMFC算法
输入
输出 共识隶属度矩阵
1.初始化:通过K-Means聚类方法初始化
2.while不收敛do
3.对于每个数据点i,通过求解式(8)更新第p个视图的隶属度矩阵
4.根据式(12)更新第p个视图的第k个聚类中心向量
5.根据式(21)更新权重
6.对于每个数据点i,通过求解式(27)计算共识隶属度矩阵
7.输出
利用参数q来调整权值分布,根据式(21)可得出存在两种极端情况:1)当
$ \underset{q\to {1}^{+}}{\mathrm{l}\mathrm{i}\mathrm{m}}{\mathit{\boldsymbol{\alpha}}}_{0}=\underset{q\to {1}^{+}}{\mathrm{l}\mathrm{i}\mathrm{m}}\frac{1}{1+\sum\limits_{s\ne o}{\left(\frac{{\mathit{\pmb{\Phi}}}_{s}}{{\mathit{\pmb{\Phi}}}_{o}}\right)}^{\frac{1}{1-q}}}=1 $ | (28) |
由此可见,KMFC算法将权重1赋给
定理1 在每次迭代中,问题式(2)的目标函数值不断减小,直到算法收敛。
证明 假设经过第
$ {\mathit{\boldsymbol{u}}}_{ik}^{\left(t+1\right)\left(p\right)}=\frac{\mathrm{e}\mathrm{x}\mathrm{p}\left(\frac{-\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{h}}}_{ik}^{\left(t\right)\left(p\right)}}{\lambda }\right){\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}\left(t\right)}}{\sum\limits_{k=1}^{c}\left[\mathrm{e}\mathrm{x}\mathrm{p}\left(\frac{-\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{h}}}_{ik}^{\left(t\right)\left(p\right)}}{\lambda }\right){\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}\left(t\right)}\right]} $ | (29) |
其中:上述目标函数和约束在
同样地,在第
$ {\mathit{\boldsymbol{v}}}_{k}^{\left(t+1\right)\left(p\right)}=\frac{\sum\limits_{i=1}^{n}{\mathit{\boldsymbol{u}}}_{ik}^{\left(t\right)\left(p\right)}{\mathit{\boldsymbol{x}}}_{i}^{\left(p\right)}}{\sum\limits_{i=1}^{n}{\mathit{\boldsymbol{u}}}_{ik}^{\left(t\right)\left(p\right)}} $ | (30) |
其中:上述目标函数和约束在
在第
$ {\mathit{\boldsymbol{\alpha}}}_{p}^{\left(t+1\right)}=\frac{{\mathit{\pmb{\Phi}}}_{p}^{\left(t\right)\frac{1}{1-q}}}{\sum\limits_{s=1}^{m}{\mathit{\pmb{\Phi}}}_{s}^{\left(t\right)\frac{1}{1-q}}} $ | (31) |
其中:上述目标函数和约束在
在第
$ {\mathit{\boldsymbol{u}}}_{ik}^{\mathrm{*}\left(t+1\right)}=\frac{\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{u}}}_{ik}^{\left(t\right)\left(p\right)}}{\sum\limits_{k=1}^{c}\sum\limits_{p=1}^{m}{\mathit{\boldsymbol{u}}}_{ik}^{\left(t\right)\left(p\right)}} $ | (32) |
其中:上述目标函数和约束在
显然,问题式(2)可以分为4个子问题,每个子问题都是一个凸优化问题。因此,上述证明过程验证了算法1的收敛性。
3 实验与结果分析根据聚类精度(Accuracy,ACC)[30]、标准化互信息(Normalized Mutual Information,NMI)[30]、相似性(Jaccard)[31]、纯度(Purity)[31]4个评价指标对KMFC算法进行性能评价。
3.1 数据集在5个真实数据集上评估KMFC算法的性能:
1)COIL20数据集[32]具有1 440张标准化图像,包含20个对象,每个对象对应72张图像。每张图像可以用30个等距投影(Isometric,ISO)、19个线性判别分析(Linear Discriminant Analysis,LDA)和30个邻域保持嵌入(Neighborhood Preserving Embedding,NPE)这3类异构特征来表示。
2)YALE数据集[33]由15名受试者的165张图像组成,每个受试者有11张图像,对应不同的面部表情或形态。每张图像可以用30个ISO、14个LDA和30个NPE这3类异构特征来表示。
3)3-Sources数据集[34]包含294篇新闻文章,涵盖商业、娱乐、健康、政治、体育和技术6个方面。第一视图有3 068个特征,第二视图有3 631个特征,第三视图有3 560个特征。
4)NUS-WIDE数据集[35]由包含81个对象的269 648张图像组成。在实验中选择猫、牛、狗、麋鹿、鹰、马、狮子、松鼠、老虎、鲸鱼、狼、斑马12种动物类别。每张图像可以用64种颜色直方图、144种颜色相关图、73种边缘方向直方图、128个小波纹理、225个块状颜色矩和500袋基于SIFT描述的单词这6类低级特征来表示。
5)Prokaryotic phyla数据集[36]包含551个原核生物种类,包括文本数据和不同的基因组表达。本文选用的1个视图数据为由描述原核生物种类的文档词袋表示组成的文本数据,另外2个视图数据为2种基因组表示[17]。与文献[17]一样,为降低数据集维数,对3个视图分别应用主成分分析(Principal Component Analysis,PCA)并保留解释90%方差的主成分。
数据集统计信息如表 1所示。
![]() |
下载CSV 表 1 数据集统计信息 Table 1 Dataset statistics |
对于ACC、NMI、Jaccard、Purity这些广泛使用的指标,值越大表示集群性能越好。
ACC表示聚类精度,定义如下:
$ {A}_{\mathrm{A}\mathrm{C}\mathrm{C}}=\frac{\sum\limits_{i=1}^{n}\delta \left({\tau }_{i}, \mathrm{m}\mathrm{a}\mathrm{p}\left({r}_{i}\right)\right)}{n} $ | (33) |
其中:
NMI表示
$ {N}_{\mathrm{N}\mathrm{M}\mathrm{I}}\left({\tau }_{i}, {r}_{i}\right)=\frac{\sum\limits_{i=1}^{c}\sum\limits_{j=1}^{c}\left({n}_{i, j}\mathrm{l}\mathrm{g}\frac{{n}_{i, j}}{{n}_{i}{n}_{j}}\right)}{\sqrt{\left(\sum\limits_{i=1}^{c}{n}_{i}\mathrm{l}\mathrm{g}\frac{{n}_{i}}{n}\right)\left(\sum\limits_{j=1}^{c}{\widehat{n}}_{j}\mathrm{l}\mathrm{g}\frac{{\widehat{n}}_{j}}{n}\right)}} $ | (34) |
其中:
Jaccard度量有限样本集N1和N2之间的相似性,定义如下:
$ {J}_{\mathrm{J}\mathrm{a}\mathrm{c}\mathrm{c}\mathrm{a}\mathrm{r}\mathrm{d}}=\frac{\left|{N}_{1}\bigcap {N}_{2}\right|}{\left|{N}_{1}\bigcup {N}_{2}\right|}=\frac{{T}_{\mathrm{T}\mathrm{P}}}{{T}_{\mathrm{T}\mathrm{P}}+{F}_{\mathrm{F}\mathrm{P}}+{F}_{\mathrm{F}\mathrm{N}}} $ | (35) |
其中:
Purity是正确类标签的百分比,定义如下:
$ {P}_{\mathrm{P}\mathrm{u}\mathrm{r}\mathrm{i}\mathrm{t}\mathrm{y}}=\frac{1}{n}\sum\limits_{i=1}^{c}\underset{1\le j\le c}{\mathrm{m}\mathrm{a}\mathrm{x}}\left|\mathrm{m}\mathrm{a}\mathrm{p}\left({r}_{i}\right)\bigcap {\tau }_{j}\right| $ | (36) |
Purity的取值范围为
首先,比较KMFC算法与FCM算法和熵模糊C均值(Entropy Fuzzy C-Means,EFCM)算法的性能,以证明KMFC算法体现了多视图算法的优势。其次,将KMFC算法与SWMC[37]、MVASM[38]、MVGL[39]3种算法进行比较,其中比较算法中涉及的参数调整参照原文献进行设置并实验以显示最优结果。在KMFC算法中,参数
![]() |
Download:
|
图 1 KMFC算法在3-Source、NUS-WIDE和Prokaryotic phyla数据集上的ACC比较 Fig. 1 ACC comparation of KMFC algorithm on 3-Source, NUS-WIDE and Prokaryotic phyla datasets |
![]() |
Download:
|
图 2 KMFC算法在3-Source、NUS-WIDE和Prokaryotic phyla数据集上的NMI比较 Fig. 2 NMI comparation of KMFC algorithm on 3-Source, NUS-WID and Prokaryotic phyla datasets |
![]() |
Download:
|
图 3 KMFC算法在3-Source、NUS-WIDE和Prokaryotic phyla数据集上的Purity比较 Fig. 3 Purity comparation of KMFC on 3-Source, NUS-WID and Prokaryotic phyla datasets |
在COIL20和YALE32数据集上,KMFC和2种单视图聚类算法FCM、EFCM的ACC、NMI、Jaccard和Purity比较结果如表 2、表 3所示,其中括号中的数字表示算法排名。从表 2、表 3可以看出:在ACC、NMI、Jaccard和Purity 4个评价指标上,KMFC算法在COIL20数据集上相比FCM(2)算法分别提高0.584 9、0.375 5、0.692 4、0.687 4,相比EFCM(2)算法分别提高0.589 9、0.383 6、0.702 0、0.698 5;在YALE32数据集上,KMFC算法相比FCM(2)算法分别提高0.471 6、303 4、617 5、610 8,相比EFCM(2)算法分别提高0.506 7、0.313 4、0.628 4、0.625 3。上述结果证明了多视图学习的优越性和有效性,并且其有利于提高聚类精度。
![]() |
下载CSV 表 2 在COIL20数据集上KMFC与2种单视图聚类算法的ACC、NMI、Jaccard和Purity比较 Table 2 Comparison of ACC, NMI, Jaccard and Purity between KMFC and two single-view clustering algorithms on COIL20 dataset |
![]() |
下载CSV 表 3 在YALE32数据集上KMFC与2种单视图聚类算法的ACC、NMI、Jaccard和Purity比较 Table 3 Comparison of ACC, NMI, Jaccard and Purity between KMFC and two single-view clustering algorithms on YALE32 dataset |
在3-Source、NUS-WIDE和Prokaryotic phyla数据集上不同多视图聚类算法的ACC、NMI和Purity比较结果如表 4~表 6所示。从表 4~表 6可以看出:在数据集3-Sources上,KMFC算法的ACC、NMI、Purity相比次优的MVASM算法分别提高0.074 6、0.153 4、0.054 8;在数据集NUS-WIDE上,KMFC算法相比次优的MVASM算法分别提高0.057 4、0.002 4、0.006 7;在数据集Prokaryotic phyla上,KMFC算法相比次优的MVGL算法分别提高0.060 9、0.062 6、0.000 7。KMFC算法在3个数据集上的收敛曲线如图 4所示。可见,KMFC算法的聚类性能明显优于对比算法。
![]() |
下载CSV 表 4 在3-Source数据集上不同多视图聚类算法的ACC、NMI和Purity比较 Table 4 Comparison of ACC, NMI and Purity of different multi-view clustering algorithms on 3-Source dataset |
![]() |
下载CSV 表 5 在NUS-WIDE数据集上不同多视图聚类算法的ACC、NMI和Purity比较 Table 5 Comparison of ACC, NMI and Purity of different multi-view clustering algorithms on NUS-WIDE dataset |
![]() |
下载CSV 表 6 在Prokaryotic phyla数据集上不同多视图聚类算法的ACC、NMI和Purity比较 Table 6 Comparison of ACC, NMI and Purity of different multi-view clustering algorithms on Prokaryotic phyla dataset |
![]() |
Download:
|
图 4 在3-Sources、NUS-WIDE、Prokaryotic phyla数据集上KMFC算法的收敛曲线 Fig. 4 Convergence curves of KMFC algorithm on 3-Sources, NUS-WIDE, Prokaryotic phyla datasets |
本文提出一种新的多视图加权聚类算法,将每个视图信息及其权重进行拟合融入标准模糊C均值聚类算法,再附加一个KL信息作为模糊正则项,其中KL信息是一个视图的隶属度与其共识隶属度的比值,因此最小化KL信息会使每个视图的隶属度偏向于共识隶属度,最终实现对共识隶属度矩阵的聚类。在多个数据集上的实验结果证明了该算法的有效性。但该算法中的权重需要引入幂指数
[1] |
XU C, TAO D C, XU C. A survey on multi-view learning[EB/OL]. [2021-05-11]. https://arxiv.org/abs/1304.5634.
|
[2] |
WEN J, ZHANG Z, XU Y, et al. Unified embedding alignment with missing views inferring for incomplete multi-view clustering[C]//Proceedings of the 33rd AAAI Conference on Artificial Intelligence and the 31st Innovative Applications of Artificial Intelligence Conference and the 9th AAAI Symposium on Educational Advances in Artificial Intelligence. Palo Alto, USA: AAAI Press, 2019: 5393-5400.
|
[3] |
ZONG L L, ZHANG X C, LIU X Y, et al. Weighted multi-view spectral clustering based on spectral perturbation[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2018: 4621-4629.
|
[4] |
JOHNSON S C. Hierarchical clustering schemes[J]. Psychometrika, 1967, 32(3): 241-254. DOI:10.1007/BF02289588 |
[5] |
LEE J W T, YEUNG D S, TSANG E C C. Hierarchical clustering based on ordinal consistency[J]. Pattern Recognition, 2005, 38(11): 1913-1925. DOI:10.1016/j.patcog.2005.05.008 |
[6] |
WU K L, YANG M S. Alternative c-means clustering algorithms[J]. Pattern Recognition, 2002, 35(10): 2267-2278. DOI:10.1016/S0031-3203(01)00197-2 |
[7] |
ZHANG D Q, CHEN S C. A comment on "alternative c-means clustering algorithms"[J]. Pattern Recognition, 2004, 37(2): 173-174. DOI:10.1016/j.patcog.2003.08.001 |
[8] |
TSENG P. Nearest q-flat to m points[J]. Journal of Optimization Theory and Applications, 2000, 105(1): 249-252. DOI:10.1023/A:1004678431677 |
[9] |
WANG Y, ZHANG W J, WU L, et al. Iterative views agreement: an iterative low-rank based structured optimization method to multi-view spectral clustering[C]//Proceedings of the 25th International Joint Conference on Artificial Intelligence. Washington D. C., USA: IEEE Press, 2016: 2153-2159.
|
[10] |
PEDRYCZ W. Collaborative fuzzy clustering[J]. Pattern Recognition Letters, 2002, 23(14): 1675-1686. DOI:10.1016/S0167-8655(02)00130-7 |
[11] |
CORNUÉJOLS A, WEMMERT C, GANÇARSKI P, et al. Collaborative clustering: why, when, what and how[J]. Information Fusion, 2018, 39: 81-95. DOI:10.1016/j.inffus.2017.04.008 |
[12] |
COSTEIRA J P, KANADE T. A multi-body factorization method for independently moving objects[J]. International Journal of Computer Vision, 1998, 29(3): 159-179. DOI:10.1023/A:1008000628999 |
[13] |
LIU Y Y, JIAO L C, SHANG F H. An efficient matrix factorization based low-rank representation for subspace clustering[J]. Pattern Recognition, 2013, 46(1): 284-292. DOI:10.1016/j.patcog.2012.06.011 |
[14] |
WANG X B, LEI Z, SHI H L, et al. Co-referenced subspace clustering[C]//Proceedings of IEEE International Conference on Multimedia and Expo. Washington D. C., USA: IEEE Press, 2018: 1-6.
|
[15] |
GUO X J. Robust subspace segmentation by simultaneously learning data representations and their affinity matrix[C]//Proceedings of the 24th International Conference on Artificial Intelligence. Washington D. C., USA: IEEE Press, 2015: 3547-3553.
|
[16] |
WANG X B, GUO X J, LEI Z, et al. Exclusivity-consistency regularized multi-view subspace clustering[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2017: 1-9.
|
[17] |
BRBIĆ M, KOPRIVA I. Multi-view low-rank sparse subspace clustering[J]. Pattern Recognition, 2018, 73: 247-258. DOI:10.1016/j.patcog.2017.08.024 |
[18] |
ASUR S, UCAR D, PARTHASARATHY S. An ensemble framework for clustering protein-protein interaction networks[J]. Bioinformatics, 2007, 23(13): 29-40. DOI:10.1093/bioinformatics/btm212 |
[19] |
WANG H J, SHAN H H, BANERJEE A. Bayesian cluster ensembles[J]. Statistical Analysis and Data Mining, 2011, 4(1): 54-70. DOI:10.1002/sam.10098 |
[20] |
CHAUDHURI K, KAKADE S M, LIVESCU K, et al. Multi-view clustering via canonical correlation analysis[C]//Proceedings of the 26th Annual International Conference on Machine Learning. New York, USA: ACM Press, 2009: 129-136.
|
[21] |
NIU D L, JENNIFER G D, JORDAN M I. Multiple non-redundant spectral clustering views[C]//Proceedings of the 27th International Conference on Machine Learning. New York, USA: ACM Press, 2010: 831-838.
|
[22] |
CAI X, NIE F P, HUANG H, et al. Heterogeneous image feature integration via multi-modal spectral clustering[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., USA: IEEE Press, 2011: 1977-1984.
|
[23] |
KUMAR A, RAI P, DAUME H. Co-regularized multi-view spectral clustering[C]//Proceedings of the 24th International Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2011, 24: 1413-1421.
|
[24] |
CAI X, NIE F P, HUANG H. Multi-view k-means clustering on big data[C]//Proceedings of the 23rd International Joint Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2013: 2598-2604.
|
[25] |
WANG H, NIE F P, HUANG H. Multi-view clustering and feature learning via structured sparsity[J]. Journal of Machine Learning Research, 2013, 28(3): 352-360. |
[26] |
洪敏, 贾彩燕, 王晓阳. K-means型多视图聚类中的初始化问题研究[J]. 计算机科学与探索, 2019, 13(4): 574-585. HONG M, JIA C Y, WANG X Y. Research on initialization of K-means type multi-view clustering[J]. Journal of Frontiers of Computer Science and Technology, 2019, 13(4): 574-585. (in Chinese) |
[27] |
XIA R, PAN Y, DU L, et al. Robust multi-view spectral clustering via low-rank and sparse decomposition[C]//Proceedings of the 28th AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2014: 2149-2155.
|
[28] |
CLEUZIOU G, EXBRAYAT M, MARTIN L, et al. CoFKM: a centralized method for multiple-view clustering[C]//Proceedings of the 9th IEEE International Conference on Data Mining. Washington D. C., USA: IEEE Press, 2009: 752-757.
|
[29] |
JIANG Y Z, CHUNG F L, WANG S T, et al. Collaborative fuzzy clustering from multiple weighted views[J]. IEEE Transactions on Cybernetics, 2015, 45(4): 688-701. DOI:10.1109/TCYB.2014.2334595 |
[30] |
CAI D, HE X, HAN J, et al. Document clustering using locality preserving indexing[J]. IEEE Transactions on Knowledge and Data Engineering, 2005, 17(12): 1624-1637. DOI:10.1109/TKDE.2005.198 |
[31] |
CHEN G H, PAN Y, GUO M Y, et al. COMPACT: a comparative package for clustering assessment[C]//Proceedings of 2005 International Conference on Parallel and Distributed Processing and Applications. Berlin, Germany: Springer, 2005: 159-167.
|
[32] |
NAYAR S. Columbia Object Image Library(COIL20)[EB/OL]. [2021-05-11]. https://www.researchgate.net/publication/2784735_Columbia_Object_Image_Library_COIL-100.
|
[33] |
CAI D, HE X F, HAN J W. Using graph model for face analysis[EB/OL]. [2021-05-11]. https://www.semanticscholar.org/paper/Using-Graph-Model-for-Face-Analysis-Cai-He/19c889f2b26b785f0ac45c427339f0335b9cc514?p2df.
|
[34] |
GREENE D, CUNNINGHAM P. A matrix factorization approach for integrating multiple data views[M]. Berlin, Germany: Springer, 2009.
|
[35] |
CHUA T S, TANG J H, HONG R C, et al. NUS-WIDE: a real-world Web image database from National University of Singapore[C]//Proceedings of the ACM International Conference on Image and Video Retrieval. New York, USA: ACM Press, 2009: 1-9.
|
[36] |
BRBIĆ M, PIŠKOREC M, VIDULIN V, et al. The landscape of microbial phenotypic traits and associated genes[J]. Nucleic Acids Research, 2016, 44(21): 10074-10090. |
[37] |
NIE F P, LI J, LI X L. Self-weighted multiview clustering with multiple graphs[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence. Melbourne, Australia: International Joint Conferences on Artificial Intelligence Organization, 2017: 2564-2570.
|
[38] |
HAN J W, XU J L, NIE F P, et al. Multi-view K-means clustering with adaptive sparse memberships and weight allocation[EB/OL]. [2021-05-11]. https://www.researchgate.net/publication/340572521_Multi-view_K-Means_Clustering_with_Adaptive_Sparse_Memberships_and_Weight_Allocation.
|
[39] |
ZHAN K, ZHANG C Q, GUAN J P, et al. Graph learning for multiview clustering[J]. IEEE Transactions on Cybernetics, 2018, 48(10): 2887-2895. DOI:10.1109/TCYB.2017.2751646 |