QuPeD: Quantized Personalization via Distillation with Applications to\n Federated Learning Article Swipe
Kaan Ozkara
,
Navjot Singh
,
Deepesh Data
,
Suhas Diggavi
·
YOU?
·
· 2021
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2107.13892
YOU?
·
· 2021
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2107.13892
Traditionally, federated learning (FL) aims to train a single global model\nwhile collaboratively using multiple clients and a server. Two natural\nchallenges that FL algorithms face are heterogeneity in data across clients and\ncollaboration of clients with {\\em diverse resources}. In this work, we\nintroduce a \\textit{quantized} and \\textit{personalized} FL algorithm QuPeD\nthat facilitates collective (personalized model compression) training via\n\\textit{knowledge distillation} (KD) among clients who have access to\nheterogeneous data and resources. For personalization, we allow clients to\nlearn \\textit{compressed personalized models} with different quantization\nparameters and model dimensions/structures. Towards this, first we propose an\nalgorithm for learning quantized models through a relaxed optimization problem,\nwhere quantization values are also optimized over. When each client\nparticipating in the (federated) learning process has different requirements\nfor the compressed model (both in model dimension and precision), we formulate\na compressed personalization framework by introducing knowledge distillation\nloss for local client objectives collaborating through a global model. We\ndevelop an alternating proximal gradient update for solving this compressed\npersonalization problem, and analyze its convergence properties. Numerically,\nwe validate that QuPeD outperforms competing personalized FL methods, FedAvg,\nand local training of clients in various heterogeneous settings.\n
Related Topics
Concepts
Metadata
- Type
- preprint
- Landing Page
- http://arxiv.org/abs/2107.13892
- https://arxiv.org/pdf/2107.13892
- OA Status
- green
- Cited By
- 17
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4287064345
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4287064345Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2107.13892Digital Object Identifier
- Title
-
QuPeD: Quantized Personalization via Distillation with Applications to\n Federated LearningWork title
- Type
-
preprintOpenAlex work type
- Publication year
-
2021Year of publication
- Publication date
-
2021-07-29Full publication date if available
- Authors
-
Kaan Ozkara, Navjot Singh, Deepesh Data, Suhas DiggaviList of authors in order
- Landing page
-
https://arxiv.org/abs/2107.13892Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2107.13892Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2107.13892Direct OA link when available
- Concepts
-
Personalization, Computer science, Quantization (signal processing), Federated learning, Convergence (economics), Distillation, Machine learning, Artificial intelligence, Algorithm, World Wide Web, Economics, Organic chemistry, Economic growth, ChemistryTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
17Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 1, 2024: 4, 2023: 7, 2022: 5Per-year citation counts (last 5 years)
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4287064345 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2107.13892 |
| ids.openalex | https://openalex.org/W4287064345 |
| fwci | 2.2576572 |
| type | preprint |
| title | QuPeD: Quantized Personalization via Distillation with Applications to\n Federated Learning |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10764 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9993000030517578 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Privacy-Preserving Technologies in Data |
| topics[1].id | https://openalex.org/T10203 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9879000186920166 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1710 |
| topics[1].subfield.display_name | Information Systems |
| topics[1].display_name | Recommender Systems and Techniques |
| topics[2].id | https://openalex.org/T11273 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9793999791145325 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1702 |
| topics[2].subfield.display_name | Artificial Intelligence |
| topics[2].display_name | Advanced Graph Neural Networks |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C183003079 |
| concepts[0].level | 2 |
| concepts[0].score | 0.8254715800285339 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q1000371 |
| concepts[0].display_name | Personalization |
| concepts[1].id | https://openalex.org/C41008148 |
| concepts[1].level | 0 |
| concepts[1].score | 0.772579550743103 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[1].display_name | Computer science |
| concepts[2].id | https://openalex.org/C28855332 |
| concepts[2].level | 2 |
| concepts[2].score | 0.6709606051445007 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q198099 |
| concepts[2].display_name | Quantization (signal processing) |
| concepts[3].id | https://openalex.org/C2992525071 |
| concepts[3].level | 2 |
| concepts[3].score | 0.5172196626663208 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q50818671 |
| concepts[3].display_name | Federated learning |
| concepts[4].id | https://openalex.org/C2777303404 |
| concepts[4].level | 2 |
| concepts[4].score | 0.44650745391845703 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q759757 |
| concepts[4].display_name | Convergence (economics) |
| concepts[5].id | https://openalex.org/C204030448 |
| concepts[5].level | 2 |
| concepts[5].score | 0.44101884961128235 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q101017 |
| concepts[5].display_name | Distillation |
| concepts[6].id | https://openalex.org/C119857082 |
| concepts[6].level | 1 |
| concepts[6].score | 0.3632722496986389 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q2539 |
| concepts[6].display_name | Machine learning |
| concepts[7].id | https://openalex.org/C154945302 |
| concepts[7].level | 1 |
| concepts[7].score | 0.31066906452178955 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[7].display_name | Artificial intelligence |
| concepts[8].id | https://openalex.org/C11413529 |
| concepts[8].level | 1 |
| concepts[8].score | 0.2783085107803345 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q8366 |
| concepts[8].display_name | Algorithm |
| concepts[9].id | https://openalex.org/C136764020 |
| concepts[9].level | 1 |
| concepts[9].score | 0.1376616656780243 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q466 |
| concepts[9].display_name | World Wide Web |
| concepts[10].id | https://openalex.org/C162324750 |
| concepts[10].level | 0 |
| concepts[10].score | 0.0 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q8134 |
| concepts[10].display_name | Economics |
| concepts[11].id | https://openalex.org/C178790620 |
| concepts[11].level | 1 |
| concepts[11].score | 0.0 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q11351 |
| concepts[11].display_name | Organic chemistry |
| concepts[12].id | https://openalex.org/C50522688 |
| concepts[12].level | 1 |
| concepts[12].score | 0.0 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q189833 |
| concepts[12].display_name | Economic growth |
| concepts[13].id | https://openalex.org/C185592680 |
| concepts[13].level | 0 |
| concepts[13].score | 0.0 |
| concepts[13].wikidata | https://www.wikidata.org/wiki/Q2329 |
| concepts[13].display_name | Chemistry |
| keywords[0].id | https://openalex.org/keywords/personalization |
| keywords[0].score | 0.8254715800285339 |
| keywords[0].display_name | Personalization |
| keywords[1].id | https://openalex.org/keywords/computer-science |
| keywords[1].score | 0.772579550743103 |
| keywords[1].display_name | Computer science |
| keywords[2].id | https://openalex.org/keywords/quantization |
| keywords[2].score | 0.6709606051445007 |
| keywords[2].display_name | Quantization (signal processing) |
| keywords[3].id | https://openalex.org/keywords/federated-learning |
| keywords[3].score | 0.5172196626663208 |
| keywords[3].display_name | Federated learning |
| keywords[4].id | https://openalex.org/keywords/convergence |
| keywords[4].score | 0.44650745391845703 |
| keywords[4].display_name | Convergence (economics) |
| keywords[5].id | https://openalex.org/keywords/distillation |
| keywords[5].score | 0.44101884961128235 |
| keywords[5].display_name | Distillation |
| keywords[6].id | https://openalex.org/keywords/machine-learning |
| keywords[6].score | 0.3632722496986389 |
| keywords[6].display_name | Machine learning |
| keywords[7].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[7].score | 0.31066906452178955 |
| keywords[7].display_name | Artificial intelligence |
| keywords[8].id | https://openalex.org/keywords/algorithm |
| keywords[8].score | 0.2783085107803345 |
| keywords[8].display_name | Algorithm |
| keywords[9].id | https://openalex.org/keywords/world-wide-web |
| keywords[9].score | 0.1376616656780243 |
| keywords[9].display_name | World Wide Web |
| language | |
| locations[0].id | pmh:oai:arXiv.org:2107.13892 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2107.13892 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2107.13892 |
| indexed_in | arxiv |
| authorships[0].author.id | https://openalex.org/A5061122810 |
| authorships[0].author.orcid | |
| authorships[0].author.display_name | Kaan Ozkara |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Ozkara, Kaan |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5082851516 |
| authorships[1].author.orcid | https://orcid.org/0000-0003-0409-8482 |
| authorships[1].author.display_name | Navjot Singh |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Singh, Navjot |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5023777978 |
| authorships[2].author.orcid | https://orcid.org/0000-0003-3544-8414 |
| authorships[2].author.display_name | Deepesh Data |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Data, Deepesh |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5083980887 |
| authorships[3].author.orcid | https://orcid.org/0000-0001-7313-9861 |
| authorships[3].author.display_name | Suhas Diggavi |
| authorships[3].author_position | last |
| authorships[3].raw_author_name | Diggavi, Suhas |
| authorships[3].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2107.13892 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2022-07-25T00:00:00 |
| display_name | QuPeD: Quantized Personalization via Distillation with Applications to\n Federated Learning |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T03:46:38.306776 |
| primary_topic.id | https://openalex.org/T10764 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9993000030517578 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Privacy-Preserving Technologies in Data |
| related_works | https://openalex.org/W4298221930, https://openalex.org/W2109940557, https://openalex.org/W2466832359, https://openalex.org/W1582019636, https://openalex.org/W1499005795, https://openalex.org/W2777914285, https://openalex.org/W3172493050, https://openalex.org/W4385420271, https://openalex.org/W4312192618, https://openalex.org/W4303448918 |
| cited_by_count | 17 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 1 |
| counts_by_year[1].year | 2024 |
| counts_by_year[1].cited_by_count | 4 |
| counts_by_year[2].year | 2023 |
| counts_by_year[2].cited_by_count | 7 |
| counts_by_year[3].year | 2022 |
| counts_by_year[3].cited_by_count | 5 |
| locations_count | 1 |
| best_oa_location.id | pmh:oai:arXiv.org:2107.13892 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2107.13892 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2107.13892 |
| primary_location.id | pmh:oai:arXiv.org:2107.13892 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2107.13892 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2107.13892 |
| publication_date | 2021-07-29 |
| publication_year | 2021 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 7, 16, 41, 92, 137 |
| abstract_inverted_index.FL | 21, 45, 163 |
| abstract_inverted_index.In | 37 |
| abstract_inverted_index.an | 141 |
| abstract_inverted_index.by | 127 |
| abstract_inverted_index.in | 26, 105, 117, 170 |
| abstract_inverted_index.of | 31, 168 |
| abstract_inverted_index.to | 5 |
| abstract_inverted_index.we | 68, 84, 122 |
| abstract_inverted_index.For | 66 |
| abstract_inverted_index.Two | 18 |
| abstract_inverted_index.and | 15, 43, 64, 78, 120, 151 |
| abstract_inverted_index.are | 24, 98 |
| abstract_inverted_index.for | 87, 131, 146 |
| abstract_inverted_index.has | 110 |
| abstract_inverted_index.its | 153 |
| abstract_inverted_index.the | 106, 113 |
| abstract_inverted_index.who | 59 |
| abstract_inverted_index.(FL) | 3 |
| abstract_inverted_index.(KD) | 56 |
| abstract_inverted_index.When | 102 |
| abstract_inverted_index.aims | 4 |
| abstract_inverted_index.also | 99 |
| abstract_inverted_index.data | 27, 63 |
| abstract_inverted_index.each | 103 |
| abstract_inverted_index.face | 23 |
| abstract_inverted_index.have | 60 |
| abstract_inverted_index.that | 20, 158 |
| abstract_inverted_index.this | 38, 148 |
| abstract_inverted_index.with | 33, 75 |
| abstract_inverted_index.(both | 116 |
| abstract_inverted_index.QuPeD | 159 |
| abstract_inverted_index.allow | 69 |
| abstract_inverted_index.among | 57 |
| abstract_inverted_index.first | 83 |
| abstract_inverted_index.local | 132, 166 |
| abstract_inverted_index.model | 51, 79, 115, 118 |
| abstract_inverted_index.over. | 101 |
| abstract_inverted_index.this, | 82 |
| abstract_inverted_index.train | 6 |
| abstract_inverted_index.using | 12 |
| abstract_inverted_index.work, | 39 |
| abstract_inverted_index.{\\em | 34 |
| abstract_inverted_index.access | 61 |
| abstract_inverted_index.across | 28 |
| abstract_inverted_index.client | 133 |
| abstract_inverted_index.global | 9, 138 |
| abstract_inverted_index.model. | 139 |
| abstract_inverted_index.models | 90 |
| abstract_inverted_index.single | 8 |
| abstract_inverted_index.update | 145 |
| abstract_inverted_index.values | 97 |
| abstract_inverted_index.Towards | 81 |
| abstract_inverted_index.analyze | 152 |
| abstract_inverted_index.clients | 14, 29, 32, 58, 70, 169 |
| abstract_inverted_index.diverse | 35 |
| abstract_inverted_index.models} | 74 |
| abstract_inverted_index.process | 109 |
| abstract_inverted_index.propose | 85 |
| abstract_inverted_index.relaxed | 93 |
| abstract_inverted_index.server. | 17 |
| abstract_inverted_index.solving | 147 |
| abstract_inverted_index.through | 91, 136 |
| abstract_inverted_index.various | 171 |
| abstract_inverted_index.gradient | 144 |
| abstract_inverted_index.learning | 2, 88, 108 |
| abstract_inverted_index.methods, | 164 |
| abstract_inverted_index.multiple | 13 |
| abstract_inverted_index.problem, | 150 |
| abstract_inverted_index.proximal | 143 |
| abstract_inverted_index.training | 53, 167 |
| abstract_inverted_index.validate | 157 |
| abstract_inverted_index.algorithm | 46 |
| abstract_inverted_index.competing | 161 |
| abstract_inverted_index.different | 76, 111 |
| abstract_inverted_index.dimension | 119 |
| abstract_inverted_index.federated | 1 |
| abstract_inverted_index.framework | 126 |
| abstract_inverted_index.knowledge | 129 |
| abstract_inverted_index.optimized | 100 |
| abstract_inverted_index.quantized | 89 |
| abstract_inverted_index.to\nlearn | 71 |
| abstract_inverted_index.algorithms | 22 |
| abstract_inverted_index.collective | 49 |
| abstract_inverted_index.compressed | 114, 124 |
| abstract_inverted_index.objectives | 134 |
| abstract_inverted_index.resources. | 65 |
| abstract_inverted_index.(federated) | 107 |
| abstract_inverted_index.QuPeD\nthat | 47 |
| abstract_inverted_index.We\ndevelop | 140 |
| abstract_inverted_index.alternating | 142 |
| abstract_inverted_index.convergence | 154 |
| abstract_inverted_index.facilitates | 48 |
| abstract_inverted_index.introducing | 128 |
| abstract_inverted_index.outperforms | 160 |
| abstract_inverted_index.precision), | 121 |
| abstract_inverted_index.properties. | 155 |
| abstract_inverted_index.resources}. | 36 |
| abstract_inverted_index.settings.\n | 173 |
| abstract_inverted_index.FedAvg,\nand | 165 |
| abstract_inverted_index.compression) | 52 |
| abstract_inverted_index.formulate\na | 123 |
| abstract_inverted_index.model\nwhile | 10 |
| abstract_inverted_index.optimization | 94 |
| abstract_inverted_index.personalized | 73, 162 |
| abstract_inverted_index.quantization | 96 |
| abstract_inverted_index.(personalized | 50 |
| abstract_inverted_index.an\nalgorithm | 86 |
| abstract_inverted_index.collaborating | 135 |
| abstract_inverted_index.distillation} | 55 |
| abstract_inverted_index.heterogeneity | 25 |
| abstract_inverted_index.heterogeneous | 172 |
| abstract_inverted_index.we\nintroduce | 40 |
| abstract_inverted_index.Traditionally, | 0 |
| abstract_inverted_index.collaboratively | 11 |
| abstract_inverted_index.personalization | 125 |
| abstract_inverted_index.problem,\nwhere | 95 |
| abstract_inverted_index.Numerically,\nwe | 156 |
| abstract_inverted_index.personalization, | 67 |
| abstract_inverted_index.requirements\nfor | 112 |
| abstract_inverted_index.to\nheterogeneous | 62 |
| abstract_inverted_index.and\ncollaboration | 30 |
| abstract_inverted_index.distillation\nloss | 130 |
| abstract_inverted_index.\\textit{compressed | 72 |
| abstract_inverted_index.\\textit{quantized} | 42 |
| abstract_inverted_index.natural\nchallenges | 19 |
| abstract_inverted_index.client\nparticipating | 104 |
| abstract_inverted_index.\\textit{personalized} | 44 |
| abstract_inverted_index.dimensions/structures. | 80 |
| abstract_inverted_index.via\n\\textit{knowledge | 54 |
| abstract_inverted_index.quantization\nparameters | 77 |
| abstract_inverted_index.compressed\npersonalization | 149 |
| cited_by_percentile_year.max | 99 |
| cited_by_percentile_year.min | 91 |
| countries_distinct_count | 0 |
| institutions_distinct_count | 4 |
| sustainable_development_goals[0].id | https://metadata.un.org/sdg/17 |
| sustainable_development_goals[0].score | 0.47999998927116394 |
| sustainable_development_goals[0].display_name | Partnerships for the goals |
| citation_normalized_percentile.value | 0.90381444 |
| citation_normalized_percentile.is_in_top_1_percent | False |
| citation_normalized_percentile.is_in_top_10_percent | False |