arXiv (Cornell University)
FwdLLM: Efficient FedLLM using Forward Gradient
August 2023 • Mengwei Xu, Yaozong Wu, Dongqi Cai, Xiang Li, Shangguang Wang
Large Language Models (LLMs) are transforming the landscape of mobile intelligence. Federated Learning (FL), a method to preserve user data privacy, is often employed in fine-tuning LLMs to downstream mobile tasks, an approach known as FedLLM. Though recent efforts have addressed the network issue induced by the vast model size, they have not practically mitigated vital challenges concerning integration with mobile devices, such as significant memory consumption and sluggish model convergence. In response to these…