Caesar: A Low-deviation Compression Approach for Efficient Federated Learning Article Swipe
Jiaming Yan
,
Jianchun Liu
,
Hongli Xu
,
L. Q. Huang
,
Jun Gong
,
Xudong Liu
,
Kun Hou
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2412.19989
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2412.19989
Compression is an efficient way to relieve the tremendous communication overhead of federated learning (FL) systems. However, for the existing works, the information loss under compression will lead to unexpected model/gradient deviation for the FL training, significantly degrading the training performance, especially under the challenges of data heterogeneity and model obsolescence. To strike a delicate trade-off between model accuracy and traffic cost, we propose Caesar, a novel FL framework with a low-deviation compression approach. For the global model download, we design a greedy method to optimize the compression ratio for each device based on the staleness of the local model, ensuring a precise initial model for local training. Regarding the local gradient upload, we utilize the device's local data properties (\ie, sample volume and label distribution) to quantify its local gradient's importance, which then guides the determination of the gradient compression ratio. Besides, with the fine-grained batch size optimization, Caesar can significantly diminish the devices' idle waiting time under the synchronized barrier. We have implemented Caesar on two physical platforms with 40 smartphones and 80 NVIDIA Jetson devices. Extensive results show that Caesar can reduce the traffic costs by about 25.54%$\thicksim$37.88% compared to the compression-based baselines with the same target accuracy, while incurring only a 0.68% degradation in final test accuracy relative to the full-precision communication.
Related Topics
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2412.19989
- https://arxiv.org/pdf/2412.19989
- OA Status
- green
- Cited By
- 1
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4405955380
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4405955380Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2412.19989Digital Object Identifier
- Title
-
Caesar: A Low-deviation Compression Approach for Efficient Federated LearningWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-12-28Full publication date if available
- Authors
-
Jiaming Yan, Jianchun Liu, Hongli Xu, L. Q. Huang, Jun Gong, Xudong Liu, Kun HouList of authors in order
- Landing page
-
https://arxiv.org/abs/2412.19989Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2412.19989Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2412.19989Direct OA link when available
- Concepts
-
Compression (physics), Computer science, Materials science, Composite materialTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
1Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 1Per-year citation counts (last 5 years)
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4405955380 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2412.19989 |
| ids.doi | https://doi.org/10.48550/arxiv.2412.19989 |
| ids.openalex | https://openalex.org/W4405955380 |
| fwci | |
| type | preprint |
| title | Caesar: A Low-deviation Compression Approach for Efficient Federated Learning |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10237 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9990000128746033 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Cryptography and Data Security |
| topics[1].id | https://openalex.org/T10764 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9983000159263611 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Privacy-Preserving Technologies in Data |
| topics[2].id | https://openalex.org/T11612 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9901000261306763 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1702 |
| topics[2].subfield.display_name | Artificial Intelligence |
| topics[2].display_name | Stochastic Gradient Optimization Techniques |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C180016635 |
| concepts[0].level | 2 |
| concepts[0].score | 0.5576062798500061 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q2712821 |
| concepts[0].display_name | Compression (physics) |
| concepts[1].id | https://openalex.org/C41008148 |
| concepts[1].level | 0 |
| concepts[1].score | 0.5538371205329895 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[1].display_name | Computer science |
| concepts[2].id | https://openalex.org/C192562407 |
| concepts[2].level | 0 |
| concepts[2].score | 0.08233970403671265 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q228736 |
| concepts[2].display_name | Materials science |
| concepts[3].id | https://openalex.org/C159985019 |
| concepts[3].level | 1 |
| concepts[3].score | 0.0 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q181790 |
| concepts[3].display_name | Composite material |
| keywords[0].id | https://openalex.org/keywords/compression |
| keywords[0].score | 0.5576062798500061 |
| keywords[0].display_name | Compression (physics) |
| keywords[1].id | https://openalex.org/keywords/computer-science |
| keywords[1].score | 0.5538371205329895 |
| keywords[1].display_name | Computer science |
| keywords[2].id | https://openalex.org/keywords/materials-science |
| keywords[2].score | 0.08233970403671265 |
| keywords[2].display_name | Materials science |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2412.19989 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2412.19989 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2412.19989 |
| locations[1].id | doi:10.48550/arxiv.2412.19989 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2412.19989 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5103396551 |
| authorships[0].author.orcid | |
| authorships[0].author.display_name | Jiaming Yan |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Yan, Jiaming |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5071164333 |
| authorships[1].author.orcid | https://orcid.org/0000-0002-1764-9303 |
| authorships[1].author.display_name | Jianchun Liu |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Liu, Jianchun |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5063184427 |
| authorships[2].author.orcid | https://orcid.org/0000-0003-3831-4577 |
| authorships[2].author.display_name | Hongli Xu |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Xu, Hongli |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5101771946 |
| authorships[3].author.orcid | https://orcid.org/0000-0003-1753-2372 |
| authorships[3].author.display_name | L. Q. Huang |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Huang, Liusheng |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5084571872 |
| authorships[4].author.orcid | https://orcid.org/0000-0001-9744-7201 |
| authorships[4].author.display_name | Jun Gong |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Gong, Jiantao |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5100354521 |
| authorships[5].author.orcid | https://orcid.org/0000-0003-2911-4899 |
| authorships[5].author.display_name | Xudong Liu |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Liu, Xudong |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5101892445 |
| authorships[6].author.orcid | https://orcid.org/0000-0002-4865-1780 |
| authorships[6].author.display_name | Kun Hou |
| authorships[6].author_position | last |
| authorships[6].raw_author_name | Hou, Kun |
| authorships[6].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2412.19989 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-01-01T00:00:00 |
| display_name | Caesar: A Low-deviation Compression Approach for Efficient Federated Learning |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10237 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9990000128746033 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Cryptography and Data Security |
| related_works | https://openalex.org/W4391375266, https://openalex.org/W2899084033, https://openalex.org/W2748952813, https://openalex.org/W2390279801, https://openalex.org/W4391913857, https://openalex.org/W2358668433, https://openalex.org/W4396701345, https://openalex.org/W2376932109, https://openalex.org/W2001405890, https://openalex.org/W4396696052 |
| cited_by_count | 1 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 1 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2412.19989 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2412.19989 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2412.19989 |
| primary_location.id | pmh:oai:arXiv.org:2412.19989 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2412.19989 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2412.19989 |
| publication_date | 2024-12-28 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 53, 65, 70, 81, 101, 204 |
| abstract_inverted_index.40 | 171 |
| abstract_inverted_index.80 | 174 |
| abstract_inverted_index.FL | 34, 67 |
| abstract_inverted_index.To | 51 |
| abstract_inverted_index.We | 162 |
| abstract_inverted_index.an | 2 |
| abstract_inverted_index.by | 188 |
| abstract_inverted_index.in | 207 |
| abstract_inverted_index.is | 1 |
| abstract_inverted_index.of | 11, 45, 96, 137 |
| abstract_inverted_index.on | 93, 166 |
| abstract_inverted_index.to | 5, 28, 84, 126, 192, 212 |
| abstract_inverted_index.we | 62, 79, 113 |
| abstract_inverted_index.For | 74 |
| abstract_inverted_index.and | 48, 59, 123, 173 |
| abstract_inverted_index.can | 150, 183 |
| abstract_inverted_index.for | 17, 32, 89, 105 |
| abstract_inverted_index.its | 128 |
| abstract_inverted_index.the | 7, 18, 21, 33, 38, 43, 75, 86, 94, 97, 109, 115, 135, 138, 144, 153, 159, 185, 193, 197, 213 |
| abstract_inverted_index.two | 167 |
| abstract_inverted_index.way | 4 |
| abstract_inverted_index.(FL) | 14 |
| abstract_inverted_index.data | 46, 118 |
| abstract_inverted_index.each | 90 |
| abstract_inverted_index.have | 163 |
| abstract_inverted_index.idle | 155 |
| abstract_inverted_index.lead | 27 |
| abstract_inverted_index.loss | 23 |
| abstract_inverted_index.only | 203 |
| abstract_inverted_index.same | 198 |
| abstract_inverted_index.show | 180 |
| abstract_inverted_index.size | 147 |
| abstract_inverted_index.test | 209 |
| abstract_inverted_index.that | 181 |
| abstract_inverted_index.then | 133 |
| abstract_inverted_index.time | 157 |
| abstract_inverted_index.will | 26 |
| abstract_inverted_index.with | 69, 143, 170, 196 |
| abstract_inverted_index.(\ie, | 120 |
| abstract_inverted_index.0.68% | 205 |
| abstract_inverted_index.about | 189 |
| abstract_inverted_index.based | 92 |
| abstract_inverted_index.batch | 146 |
| abstract_inverted_index.cost, | 61 |
| abstract_inverted_index.costs | 187 |
| abstract_inverted_index.final | 208 |
| abstract_inverted_index.label | 124 |
| abstract_inverted_index.local | 98, 106, 110, 117, 129 |
| abstract_inverted_index.model | 49, 57, 77, 104 |
| abstract_inverted_index.novel | 66 |
| abstract_inverted_index.ratio | 88 |
| abstract_inverted_index.under | 24, 42, 158 |
| abstract_inverted_index.which | 132 |
| abstract_inverted_index.while | 201 |
| abstract_inverted_index.Caesar | 149, 165, 182 |
| abstract_inverted_index.Jetson | 176 |
| abstract_inverted_index.NVIDIA | 175 |
| abstract_inverted_index.design | 80 |
| abstract_inverted_index.device | 91 |
| abstract_inverted_index.global | 76 |
| abstract_inverted_index.greedy | 82 |
| abstract_inverted_index.guides | 134 |
| abstract_inverted_index.method | 83 |
| abstract_inverted_index.model, | 99 |
| abstract_inverted_index.ratio. | 141 |
| abstract_inverted_index.reduce | 184 |
| abstract_inverted_index.sample | 121 |
| abstract_inverted_index.strike | 52 |
| abstract_inverted_index.target | 199 |
| abstract_inverted_index.volume | 122 |
| abstract_inverted_index.works, | 20 |
| abstract_inverted_index.Caesar, | 64 |
| abstract_inverted_index.between | 56 |
| abstract_inverted_index.initial | 103 |
| abstract_inverted_index.precise | 102 |
| abstract_inverted_index.propose | 63 |
| abstract_inverted_index.relieve | 6 |
| abstract_inverted_index.results | 179 |
| abstract_inverted_index.traffic | 60, 186 |
| abstract_inverted_index.upload, | 112 |
| abstract_inverted_index.utilize | 114 |
| abstract_inverted_index.waiting | 156 |
| abstract_inverted_index.Besides, | 142 |
| abstract_inverted_index.However, | 16 |
| abstract_inverted_index.accuracy | 58, 210 |
| abstract_inverted_index.barrier. | 161 |
| abstract_inverted_index.compared | 191 |
| abstract_inverted_index.delicate | 54 |
| abstract_inverted_index.device's | 116 |
| abstract_inverted_index.devices' | 154 |
| abstract_inverted_index.devices. | 177 |
| abstract_inverted_index.diminish | 152 |
| abstract_inverted_index.ensuring | 100 |
| abstract_inverted_index.existing | 19 |
| abstract_inverted_index.gradient | 111, 139 |
| abstract_inverted_index.learning | 13 |
| abstract_inverted_index.optimize | 85 |
| abstract_inverted_index.overhead | 10 |
| abstract_inverted_index.physical | 168 |
| abstract_inverted_index.quantify | 127 |
| abstract_inverted_index.relative | 211 |
| abstract_inverted_index.systems. | 15 |
| abstract_inverted_index.training | 39 |
| abstract_inverted_index.Extensive | 178 |
| abstract_inverted_index.Regarding | 108 |
| abstract_inverted_index.accuracy, | 200 |
| abstract_inverted_index.approach. | 73 |
| abstract_inverted_index.baselines | 195 |
| abstract_inverted_index.degrading | 37 |
| abstract_inverted_index.deviation | 31 |
| abstract_inverted_index.download, | 78 |
| abstract_inverted_index.efficient | 3 |
| abstract_inverted_index.federated | 12 |
| abstract_inverted_index.framework | 68 |
| abstract_inverted_index.incurring | 202 |
| abstract_inverted_index.platforms | 169 |
| abstract_inverted_index.staleness | 95 |
| abstract_inverted_index.trade-off | 55 |
| abstract_inverted_index.training, | 35 |
| abstract_inverted_index.training. | 107 |
| abstract_inverted_index.challenges | 44 |
| abstract_inverted_index.especially | 41 |
| abstract_inverted_index.gradient's | 130 |
| abstract_inverted_index.properties | 119 |
| abstract_inverted_index.tremendous | 8 |
| abstract_inverted_index.unexpected | 29 |
| abstract_inverted_index.Compression | 0 |
| abstract_inverted_index.compression | 25, 72, 87, 140 |
| abstract_inverted_index.degradation | 206 |
| abstract_inverted_index.implemented | 164 |
| abstract_inverted_index.importance, | 131 |
| abstract_inverted_index.information | 22 |
| abstract_inverted_index.smartphones | 172 |
| abstract_inverted_index.fine-grained | 145 |
| abstract_inverted_index.performance, | 40 |
| abstract_inverted_index.synchronized | 160 |
| abstract_inverted_index.communication | 9 |
| abstract_inverted_index.determination | 136 |
| abstract_inverted_index.distribution) | 125 |
| abstract_inverted_index.heterogeneity | 47 |
| abstract_inverted_index.low-deviation | 71 |
| abstract_inverted_index.obsolescence. | 50 |
| abstract_inverted_index.optimization, | 148 |
| abstract_inverted_index.significantly | 36, 151 |
| abstract_inverted_index.communication. | 215 |
| abstract_inverted_index.full-precision | 214 |
| abstract_inverted_index.model/gradient | 30 |
| abstract_inverted_index.compression-based | 194 |
| abstract_inverted_index.25.54%$\thicksim$37.88% | 190 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 7 |
| citation_normalized_percentile |