QuIC: Quantum-Inspired Compound Adapters for Parameter Efficient Fine-Tuning Article Swipe
Snehal Raj
,
Brian Coyle
·
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2502.06916
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2502.06916
Scaling full finetuning of large foundation models strains GPU memory and training time. Parameter Efficient Fine-Tuning (PEFT) methods address this issue via adapter modules which update only a small subset of model parameters. In this work, we introduce Quantum-Inspired Compound Adapters (QuIC Adapters), a PEFT approach inspired from Hamming-weight preserving quantum circuits that can effectively finetune a model using less than 0.02\% memory footprint of the base model. QuIC adapters preserve pretrained representations by enforcing orthogonality in weight parameters, and have native deployment mechanisms on quantum computers. We test QuIC adapters by finetuning large language models like LLaMA and vision transformers on language, math, reasoning and vision benchmarks. In its first-order configuration, QuIC recovers the performance of existing orthogonal methods, while higher-order configurations enable substantial parameter compression (over 40x smaller than LoRA) for a modest performance trade-off, unlocking applications in highly resource-constrained environments. Through ablation studies, we determine that combining multiple Hamming-weight orders with orthogonality and matrix compounding are essential for performant finetuning. Our findings suggest that QuIC adapters offers a promising direction for efficient finetuning of foundation models in resource-constrained environments.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2502.06916
- https://arxiv.org/pdf/2502.06916
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4407423568
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4407423568Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2502.06916Digital Object Identifier
- Title
-
QuIC: Quantum-Inspired Compound Adapters for Parameter Efficient Fine-TuningWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2025Year of publication
- Publication date
-
2025-02-10Full publication date if available
- Authors
-
Snehal Raj, Brian CoyleList of authors in order
- Landing page
-
https://arxiv.org/abs/2502.06916Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2502.06916Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2502.06916Direct OA link when available
- Concepts
-
Foundation (evidence), Quantum, Computer science, Compressed sensing, Physics, Artificial intelligence, Quantum mechanics, Geography, ArchaeologyTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4407423568 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2502.06916 |
| ids.doi | https://doi.org/10.48550/arxiv.2502.06916 |
| ids.openalex | https://openalex.org/W4407423568 |
| fwci | |
| type | preprint |
| title | QuIC: Quantum-Inspired Compound Adapters for Parameter Efficient Fine-Tuning |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10271 |
| topics[0].field.id | https://openalex.org/fields/19 |
| topics[0].field.display_name | Earth and Planetary Sciences |
| topics[0].score | 0.9855999946594238 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1908 |
| topics[0].subfield.display_name | Geophysics |
| topics[0].display_name | Seismic Imaging and Inversion Techniques |
| topics[1].id | https://openalex.org/T12510 |
| topics[1].field.id | https://openalex.org/fields/22 |
| topics[1].field.display_name | Engineering |
| topics[1].score | 0.9668999910354614 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/2208 |
| topics[1].subfield.display_name | Electrical and Electronic Engineering |
| topics[1].display_name | Magneto-Optical Properties and Applications |
| topics[2].id | https://openalex.org/T10299 |
| topics[2].field.id | https://openalex.org/fields/22 |
| topics[2].field.display_name | Engineering |
| topics[2].score | 0.9621000289916992 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/2208 |
| topics[2].subfield.display_name | Electrical and Electronic Engineering |
| topics[2].display_name | Photonic and Optical Devices |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C2780966255 |
| concepts[0].level | 2 |
| concepts[0].score | 0.8300486207008362 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q5474306 |
| concepts[0].display_name | Foundation (evidence) |
| concepts[1].id | https://openalex.org/C84114770 |
| concepts[1].level | 2 |
| concepts[1].score | 0.5444018840789795 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q46344 |
| concepts[1].display_name | Quantum |
| concepts[2].id | https://openalex.org/C41008148 |
| concepts[2].level | 0 |
| concepts[2].score | 0.4672788381576538 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[2].display_name | Computer science |
| concepts[3].id | https://openalex.org/C124851039 |
| concepts[3].level | 2 |
| concepts[3].score | 0.46053412556648254 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q2665459 |
| concepts[3].display_name | Compressed sensing |
| concepts[4].id | https://openalex.org/C121332964 |
| concepts[4].level | 0 |
| concepts[4].score | 0.23862987756729126 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q413 |
| concepts[4].display_name | Physics |
| concepts[5].id | https://openalex.org/C154945302 |
| concepts[5].level | 1 |
| concepts[5].score | 0.20372921228408813 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[5].display_name | Artificial intelligence |
| concepts[6].id | https://openalex.org/C62520636 |
| concepts[6].level | 1 |
| concepts[6].score | 0.1211094856262207 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q944 |
| concepts[6].display_name | Quantum mechanics |
| concepts[7].id | https://openalex.org/C205649164 |
| concepts[7].level | 0 |
| concepts[7].score | 0.08479297161102295 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q1071 |
| concepts[7].display_name | Geography |
| concepts[8].id | https://openalex.org/C166957645 |
| concepts[8].level | 1 |
| concepts[8].score | 0.0 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q23498 |
| concepts[8].display_name | Archaeology |
| keywords[0].id | https://openalex.org/keywords/foundation |
| keywords[0].score | 0.8300486207008362 |
| keywords[0].display_name | Foundation (evidence) |
| keywords[1].id | https://openalex.org/keywords/quantum |
| keywords[1].score | 0.5444018840789795 |
| keywords[1].display_name | Quantum |
| keywords[2].id | https://openalex.org/keywords/computer-science |
| keywords[2].score | 0.4672788381576538 |
| keywords[2].display_name | Computer science |
| keywords[3].id | https://openalex.org/keywords/compressed-sensing |
| keywords[3].score | 0.46053412556648254 |
| keywords[3].display_name | Compressed sensing |
| keywords[4].id | https://openalex.org/keywords/physics |
| keywords[4].score | 0.23862987756729126 |
| keywords[4].display_name | Physics |
| keywords[5].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[5].score | 0.20372921228408813 |
| keywords[5].display_name | Artificial intelligence |
| keywords[6].id | https://openalex.org/keywords/quantum-mechanics |
| keywords[6].score | 0.1211094856262207 |
| keywords[6].display_name | Quantum mechanics |
| keywords[7].id | https://openalex.org/keywords/geography |
| keywords[7].score | 0.08479297161102295 |
| keywords[7].display_name | Geography |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2502.06916 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2502.06916 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2502.06916 |
| locations[1].id | doi:10.48550/arxiv.2502.06916 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2502.06916 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5020604266 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-4063-3190 |
| authorships[0].author.display_name | Snehal Raj |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Raj, Snehal |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5009617378 |
| authorships[1].author.orcid | https://orcid.org/0000-0002-3436-8458 |
| authorships[1].author.display_name | Brian Coyle |
| authorships[1].author_position | last |
| authorships[1].raw_author_name | Coyle, Brian |
| authorships[1].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2502.06916 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-02-13T00:00:00 |
| display_name | QuIC: Quantum-Inspired Compound Adapters for Parameter Efficient Fine-Tuning |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10271 |
| primary_topic.field.id | https://openalex.org/fields/19 |
| primary_topic.field.display_name | Earth and Planetary Sciences |
| primary_topic.score | 0.9855999946594238 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1908 |
| primary_topic.subfield.display_name | Geophysics |
| primary_topic.display_name | Seismic Imaging and Inversion Techniques |
| related_works | https://openalex.org/W4391375266, https://openalex.org/W2899084033, https://openalex.org/W2748952813, https://openalex.org/W2158224665, https://openalex.org/W2379589510, https://openalex.org/W2381393187, https://openalex.org/W2332779545, https://openalex.org/W2810730439, https://openalex.org/W4300044672, https://openalex.org/W1881631164 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2502.06916 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2502.06916 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2502.06916 |
| primary_location.id | pmh:oai:arXiv.org:2502.06916 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2502.06916 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2502.06916 |
| publication_date | 2025-02-10 |
| publication_year | 2025 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 27, 43, 56, 133, 170 |
| abstract_inverted_index.In | 33, 108 |
| abstract_inverted_index.We | 87 |
| abstract_inverted_index.by | 73, 91 |
| abstract_inverted_index.in | 76, 139, 179 |
| abstract_inverted_index.of | 3, 30, 64, 116, 176 |
| abstract_inverted_index.on | 84, 101 |
| abstract_inverted_index.we | 36, 146 |
| abstract_inverted_index.40x | 128 |
| abstract_inverted_index.GPU | 8 |
| abstract_inverted_index.Our | 163 |
| abstract_inverted_index.and | 10, 79, 98, 105, 155 |
| abstract_inverted_index.are | 158 |
| abstract_inverted_index.can | 53 |
| abstract_inverted_index.for | 132, 160, 173 |
| abstract_inverted_index.its | 109 |
| abstract_inverted_index.the | 65, 114 |
| abstract_inverted_index.via | 21 |
| abstract_inverted_index.PEFT | 44 |
| abstract_inverted_index.QuIC | 68, 89, 112, 167 |
| abstract_inverted_index.base | 66 |
| abstract_inverted_index.from | 47 |
| abstract_inverted_index.full | 1 |
| abstract_inverted_index.have | 80 |
| abstract_inverted_index.less | 59 |
| abstract_inverted_index.like | 96 |
| abstract_inverted_index.only | 26 |
| abstract_inverted_index.test | 88 |
| abstract_inverted_index.than | 60, 130 |
| abstract_inverted_index.that | 52, 148, 166 |
| abstract_inverted_index.this | 19, 34 |
| abstract_inverted_index.with | 153 |
| abstract_inverted_index.(QuIC | 41 |
| abstract_inverted_index.(over | 127 |
| abstract_inverted_index.LLaMA | 97 |
| abstract_inverted_index.LoRA) | 131 |
| abstract_inverted_index.issue | 20 |
| abstract_inverted_index.large | 4, 93 |
| abstract_inverted_index.math, | 103 |
| abstract_inverted_index.model | 31, 57 |
| abstract_inverted_index.small | 28 |
| abstract_inverted_index.time. | 12 |
| abstract_inverted_index.using | 58 |
| abstract_inverted_index.which | 24 |
| abstract_inverted_index.while | 120 |
| abstract_inverted_index.work, | 35 |
| abstract_inverted_index.(PEFT) | 16 |
| abstract_inverted_index.0.02\% | 61 |
| abstract_inverted_index.enable | 123 |
| abstract_inverted_index.highly | 140 |
| abstract_inverted_index.matrix | 156 |
| abstract_inverted_index.memory | 9, 62 |
| abstract_inverted_index.model. | 67 |
| abstract_inverted_index.models | 6, 95, 178 |
| abstract_inverted_index.modest | 134 |
| abstract_inverted_index.native | 81 |
| abstract_inverted_index.offers | 169 |
| abstract_inverted_index.orders | 152 |
| abstract_inverted_index.subset | 29 |
| abstract_inverted_index.update | 25 |
| abstract_inverted_index.vision | 99, 106 |
| abstract_inverted_index.weight | 77 |
| abstract_inverted_index.Scaling | 0 |
| abstract_inverted_index.Through | 143 |
| abstract_inverted_index.adapter | 22 |
| abstract_inverted_index.address | 18 |
| abstract_inverted_index.methods | 17 |
| abstract_inverted_index.modules | 23 |
| abstract_inverted_index.quantum | 50, 85 |
| abstract_inverted_index.smaller | 129 |
| abstract_inverted_index.strains | 7 |
| abstract_inverted_index.suggest | 165 |
| abstract_inverted_index.Adapters | 40 |
| abstract_inverted_index.Compound | 39 |
| abstract_inverted_index.ablation | 144 |
| abstract_inverted_index.adapters | 69, 90, 168 |
| abstract_inverted_index.approach | 45 |
| abstract_inverted_index.circuits | 51 |
| abstract_inverted_index.existing | 117 |
| abstract_inverted_index.findings | 164 |
| abstract_inverted_index.finetune | 55 |
| abstract_inverted_index.inspired | 46 |
| abstract_inverted_index.language | 94 |
| abstract_inverted_index.methods, | 119 |
| abstract_inverted_index.multiple | 150 |
| abstract_inverted_index.preserve | 70 |
| abstract_inverted_index.recovers | 113 |
| abstract_inverted_index.studies, | 145 |
| abstract_inverted_index.training | 11 |
| abstract_inverted_index.Efficient | 14 |
| abstract_inverted_index.Parameter | 13 |
| abstract_inverted_index.combining | 149 |
| abstract_inverted_index.determine | 147 |
| abstract_inverted_index.direction | 172 |
| abstract_inverted_index.efficient | 174 |
| abstract_inverted_index.enforcing | 74 |
| abstract_inverted_index.essential | 159 |
| abstract_inverted_index.footprint | 63 |
| abstract_inverted_index.introduce | 37 |
| abstract_inverted_index.language, | 102 |
| abstract_inverted_index.parameter | 125 |
| abstract_inverted_index.promising | 171 |
| abstract_inverted_index.reasoning | 104 |
| abstract_inverted_index.unlocking | 137 |
| abstract_inverted_index.Adapters), | 42 |
| abstract_inverted_index.computers. | 86 |
| abstract_inverted_index.deployment | 82 |
| abstract_inverted_index.finetuning | 2, 92, 175 |
| abstract_inverted_index.foundation | 5, 177 |
| abstract_inverted_index.mechanisms | 83 |
| abstract_inverted_index.orthogonal | 118 |
| abstract_inverted_index.performant | 161 |
| abstract_inverted_index.preserving | 49 |
| abstract_inverted_index.pretrained | 71 |
| abstract_inverted_index.trade-off, | 136 |
| abstract_inverted_index.Fine-Tuning | 15 |
| abstract_inverted_index.benchmarks. | 107 |
| abstract_inverted_index.compounding | 157 |
| abstract_inverted_index.compression | 126 |
| abstract_inverted_index.effectively | 54 |
| abstract_inverted_index.finetuning. | 162 |
| abstract_inverted_index.first-order | 110 |
| abstract_inverted_index.parameters, | 78 |
| abstract_inverted_index.parameters. | 32 |
| abstract_inverted_index.performance | 115, 135 |
| abstract_inverted_index.substantial | 124 |
| abstract_inverted_index.applications | 138 |
| abstract_inverted_index.higher-order | 121 |
| abstract_inverted_index.transformers | 100 |
| abstract_inverted_index.environments. | 142, 181 |
| abstract_inverted_index.orthogonality | 75, 154 |
| abstract_inverted_index.Hamming-weight | 48, 151 |
| abstract_inverted_index.configuration, | 111 |
| abstract_inverted_index.configurations | 122 |
| abstract_inverted_index.representations | 72 |
| abstract_inverted_index.Quantum-Inspired | 38 |
| abstract_inverted_index.resource-constrained | 141, 180 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 2 |
| citation_normalized_percentile |