Federated learning is a distributed machine learning technique for collaboratively training machine learning models for multiple clients while protecting the privacy of client data. However, the heterogeneity inherent in client data limits the full application potential of federated learning, for which personalized federated learning is a viable solution. The traditional clustering-based personalized federated learning schemes group clients with the same data distribution into one cluster, exploiting the homogeneous nature of some client data and reducing the impact of data heterogeneity on federated learning; however, this approach fails to account for the possibility of clients belonging to multiple clusters. Based on the concept that client data approximate adhere to multiple data distributions, a personalized Federated learning algorithm is proposed based on Mutual information and Soft clustering(pFedMS).A mutual information formula based on model features is introduced to address the shortcomings of current federated learning client clustering indices, which can not accurately reflect the similarity of model features.This formula serves as a clustering index that effectively distinguishes similar clients. A clustering rationality measurement method based on intra-class and inter-class distances is proposed to dynamically adjust the clustering results. The similarity between clients and clusters is calculated using affiliation, which allows clients to belong to multiple clusters simultaneously and improves the performance of the clustering algorithm. Experimental results on CIFAR-10 and Fashion-MNIST(FMNIST) datasets show that the pFedMS improves the Best Mean Testing Accuracy(BMTA) of clients by 2.4 to 3.0 percentage points compared to the comparison algorithms such as FedAvg, CFL.
personalized federated learning,