pFedGPA: Diffusion-based Generative Parameter Aggregation for Personalized Federated Learning Article Swipe

MAKE IMPACT

Jiahao Lai , Jiaqi Li , Jian Xu , Yanru Wu , Boshi Tang , Siqi Chen , Yongfeng Huang , Wenbo Ding , Yang Li · YOU? · · 2024 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2409.05701

Federated Learning (FL) offers a decentralized approach to model training, where data remains local and only model parameters are shared between the clients and the central server. Traditional methods, such as Federated Averaging (FedAvg), linearly aggregate these parameters which are usually trained on heterogeneous data distributions, potentially overlooking the complex, high-dimensional nature of the parameter space. This can result in degraded performance of the aggregated model. While personalized FL approaches can mitigate the heterogeneous data issue to some extent, the limitation of linear aggregation remains unresolved. To alleviate this issue, we investigate the generative approach of diffusion model and propose a novel generative parameter aggregation framework for personalized FL, \texttt{pFedGPA}. In this framework, we deploy a diffusion model on the server to integrate the diverse parameter distributions and propose a parameter inversion method to efficiently generate a set of personalized parameters for each client. This inversion method transforms the uploaded parameters into a latent code, which is then aggregated through denoising sampling to produce the final personalized parameters. By encoding the dependence of a client's model parameters on the specific data distribution using the high-capacity diffusion model, \texttt{pFedGPA} can effectively decouple the complexity of the overall distribution of all clients' model parameters from the complexity of each individual client's parameter distribution. Our experimental results consistently demonstrate the superior performance of the proposed method across multiple datasets, surpassing baseline approaches.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2409.05701

PDF

https://arxiv.org/pdf/2409.05701

OA Status

green

Related Works

10

OpenAlex ID

https://openalex.org/W4403618774

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4403618774

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2409.05701

Digital Object Identifier

Title

pFedGPA: Diffusion-based Generative Parameter Aggregation for Personalized Federated Learning

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2024

Year of publication

Publication date

2024-09-09

Full publication date if available

Authors

Jiahao Lai, Jiaqi Li, Jian Xu, Yanru Wu, Boshi Tang, Siqi Chen, Yongfeng Huang, Wenbo Ding, Yang Li

List of authors in order

Landing page

https://arxiv.org/abs/2409.05701

Publisher landing page

PDF URL

https://arxiv.org/pdf/2409.05701

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2409.05701

Direct OA link when available

Concepts

Generative grammar, Generative model, Computer science, Diffusion, Federated learning, Artificial intelligence, Physics, Thermodynamics

Top concepts (fields/topics) attached by OpenAlex

Cited by

0

Total citation count in OpenAlex

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload