Constraint Decomposition for Multi-Objective Instruction-Following in Large Language Models Article Swipe
Large language models (LLMs) trained with reinforcement learning from human feed- back (RLHF) struggle with complex instructions that bundle multiple, potentially con- icting requirements. We introduce constraint decomposition, a framework that separates multi-objective instructions into orthogonal componentssemantic correctness, structural organization, format specications, and meta-level requirementsand optimizes each in- dependently before hierarchical combination. Our approach addresses the fundamental limitation of monolithic reward models: their inability to distinguish which specic con- straint failed when multiple requirements conict. We train decomposed reward models with aspect-level human preferences and demonstrate that explicit constraint separation, combined with conict-aware weight adaptation, enables more eective multi-objective op- timization. On the IFEval benchmark, our method achieves 73.8% prompt-level accuracy (±1.6%), a 32.6 percentage point improvement over standard RLHF (41.2%). Ablation studies show that constraint decomposition contributes 54% of the total improvement, with hierarchical combination adding 17%, weight prediction 15%, and conict detection 14%. Our method generalizes to GSM8K (+15.6 points), HumanEval (+11.1 points), and MT- Bench (+1.4 points). Code and data are available at https://github.com/epaunova/ constraint-decomposition-llm.
Related Topics
Concepts
Constraint (computer-aided design)
Computer science
Decomposition
Decomposition method (queueing theory)
Language model
Code (set theory)
Constraint satisfaction problem
Point (geometry)
Artificial intelligence
Algorithm
Constraint satisfaction
Reinforcement learning
Computation
Theoretical computer science
Mathematical optimization
Bundle
Data modeling
Constraint programming
Domain (mathematical analysis)
Local consistency
Constrained optimization
Constraint logic programming
Set (abstract data type)
Metadata
- Type
- preprint
- Landing Page
- https://doi.org/10.5281/zenodo.17850186
- OA Status
- green
- OpenAlex ID
- https://openalex.org/W7110325958
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W7110325958Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.5281/zenodo.17850186Digital Object Identifier
- Title
-
Constraint Decomposition for Multi-Objective Instruction-Following in Large Language ModelsWork title
- Type
-
preprintOpenAlex work type
- Publication year
-
2025Year of publication
- Publication date
-
2025-12-08Full publication date if available
- Authors
-
Paunova, EvaList of authors in order
- Landing page
-
https://doi.org/10.5281/zenodo.17850186Publisher landing page
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://doi.org/10.5281/zenodo.17850186Direct OA link when available
- Concepts
-
Constraint (computer-aided design), Computer science, Decomposition, Decomposition method (queueing theory), Language model, Code (set theory), Constraint satisfaction problem, Point (geometry), Artificial intelligence, Algorithm, Constraint satisfaction, Reinforcement learning, Computation, Theoretical computer science, Mathematical optimization, Bundle, Data modeling, Constraint programming, Domain (mathematical analysis), Local consistency, Constrained optimization, Constraint logic programming, Set (abstract data type)Top concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
Full payload
| id | https://openalex.org/W7110325958 |
|---|---|
| doi | https://doi.org/10.5281/zenodo.17850186 |
| ids.doi | https://doi.org/10.5281/zenodo.17850186 |
| ids.openalex | https://openalex.org/W7110325958 |
| fwci | |
| type | preprint |
| title | Constraint Decomposition for Multi-Objective Instruction-Following in Large Language Models |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C2776036281 |
| concepts[0].level | 2 |
| concepts[0].score | 0.747273325920105 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q48769818 |
| concepts[0].display_name | Constraint (computer-aided design) |
| concepts[1].id | https://openalex.org/C41008148 |
| concepts[1].level | 0 |
| concepts[1].score | 0.6373134851455688 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[1].display_name | Computer science |
| concepts[2].id | https://openalex.org/C124681953 |
| concepts[2].level | 2 |
| concepts[2].score | 0.5486883521080017 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q339062 |
| concepts[2].display_name | Decomposition |
| concepts[3].id | https://openalex.org/C2778258933 |
| concepts[3].level | 2 |
| concepts[3].score | 0.5420629978179932 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q16918986 |
| concepts[3].display_name | Decomposition method (queueing theory) |
| concepts[4].id | https://openalex.org/C137293760 |
| concepts[4].level | 2 |
| concepts[4].score | 0.5398157835006714 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q3621696 |
| concepts[4].display_name | Language model |
| concepts[5].id | https://openalex.org/C2776760102 |
| concepts[5].level | 3 |
| concepts[5].score | 0.4605359733104706 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q5139990 |
| concepts[5].display_name | Code (set theory) |
| concepts[6].id | https://openalex.org/C199622910 |
| concepts[6].level | 3 |
| concepts[6].score | 0.4525841176509857 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q1128326 |
| concepts[6].display_name | Constraint satisfaction problem |
| concepts[7].id | https://openalex.org/C28719098 |
| concepts[7].level | 2 |
| concepts[7].score | 0.4501391053199768 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q44946 |
| concepts[7].display_name | Point (geometry) |
| concepts[8].id | https://openalex.org/C154945302 |
| concepts[8].level | 1 |
| concepts[8].score | 0.4411029815673828 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[8].display_name | Artificial intelligence |
| concepts[9].id | https://openalex.org/C11413529 |
| concepts[9].level | 1 |
| concepts[9].score | 0.39750710129737854 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q8366 |
| concepts[9].display_name | Algorithm |
| concepts[10].id | https://openalex.org/C44616089 |
| concepts[10].level | 3 |
| concepts[10].score | 0.3887014091014862 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q30158686 |
| concepts[10].display_name | Constraint satisfaction |
| concepts[11].id | https://openalex.org/C97541855 |
| concepts[11].level | 2 |
| concepts[11].score | 0.3657139539718628 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q830687 |
| concepts[11].display_name | Reinforcement learning |
| concepts[12].id | https://openalex.org/C45374587 |
| concepts[12].level | 2 |
| concepts[12].score | 0.3628217279911041 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q12525525 |
| concepts[12].display_name | Computation |
| concepts[13].id | https://openalex.org/C80444323 |
| concepts[13].level | 1 |
| concepts[13].score | 0.33999067544937134 |
| concepts[13].wikidata | https://www.wikidata.org/wiki/Q2878974 |
| concepts[13].display_name | Theoretical computer science |
| concepts[14].id | https://openalex.org/C126255220 |
| concepts[14].level | 1 |
| concepts[14].score | 0.3315902650356293 |
| concepts[14].wikidata | https://www.wikidata.org/wiki/Q141495 |
| concepts[14].display_name | Mathematical optimization |
| concepts[15].id | https://openalex.org/C2778134712 |
| concepts[15].level | 2 |
| concepts[15].score | 0.28807130455970764 |
| concepts[15].wikidata | https://www.wikidata.org/wiki/Q1047307 |
| concepts[15].display_name | Bundle |
| concepts[16].id | https://openalex.org/C67186912 |
| concepts[16].level | 2 |
| concepts[16].score | 0.28482285141944885 |
| concepts[16].wikidata | https://www.wikidata.org/wiki/Q367664 |
| concepts[16].display_name | Data modeling |
| concepts[17].id | https://openalex.org/C173404611 |
| concepts[17].level | 3 |
| concepts[17].score | 0.27747219800949097 |
| concepts[17].wikidata | https://www.wikidata.org/wiki/Q528588 |
| concepts[17].display_name | Constraint programming |
| concepts[18].id | https://openalex.org/C36503486 |
| concepts[18].level | 2 |
| concepts[18].score | 0.27618712186813354 |
| concepts[18].wikidata | https://www.wikidata.org/wiki/Q11235244 |
| concepts[18].display_name | Domain (mathematical analysis) |
| concepts[19].id | https://openalex.org/C137105694 |
| concepts[19].level | 4 |
| concepts[19].score | 0.26995721459388733 |
| concepts[19].wikidata | https://www.wikidata.org/wiki/Q3407510 |
| concepts[19].display_name | Local consistency |
| concepts[20].id | https://openalex.org/C55660270 |
| concepts[20].level | 2 |
| concepts[20].score | 0.26134195923805237 |
| concepts[20].wikidata | https://www.wikidata.org/wiki/Q5164377 |
| concepts[20].display_name | Constrained optimization |
| concepts[21].id | https://openalex.org/C176783269 |
| concepts[21].level | 4 |
| concepts[21].score | 0.2560943365097046 |
| concepts[21].wikidata | https://www.wikidata.org/wiki/Q5164378 |
| concepts[21].display_name | Constraint logic programming |
| concepts[22].id | https://openalex.org/C177264268 |
| concepts[22].level | 2 |
| concepts[22].score | 0.25334632396698 |
| concepts[22].wikidata | https://www.wikidata.org/wiki/Q1514741 |
| concepts[22].display_name | Set (abstract data type) |
| keywords[0].id | https://openalex.org/keywords/constraint |
| keywords[0].score | 0.747273325920105 |
| keywords[0].display_name | Constraint (computer-aided design) |
| keywords[1].id | https://openalex.org/keywords/decomposition |
| keywords[1].score | 0.5486883521080017 |
| keywords[1].display_name | Decomposition |
| keywords[2].id | https://openalex.org/keywords/decomposition-method |
| keywords[2].score | 0.5420629978179932 |
| keywords[2].display_name | Decomposition method (queueing theory) |
| keywords[3].id | https://openalex.org/keywords/language-model |
| keywords[3].score | 0.5398157835006714 |
| keywords[3].display_name | Language model |
| keywords[4].id | https://openalex.org/keywords/code |
| keywords[4].score | 0.4605359733104706 |
| keywords[4].display_name | Code (set theory) |
| keywords[5].id | https://openalex.org/keywords/constraint-satisfaction-problem |
| keywords[5].score | 0.4525841176509857 |
| keywords[5].display_name | Constraint satisfaction problem |
| keywords[6].id | https://openalex.org/keywords/point |
| keywords[6].score | 0.4501391053199768 |
| keywords[6].display_name | Point (geometry) |
| keywords[7].id | https://openalex.org/keywords/constraint-satisfaction |
| keywords[7].score | 0.3887014091014862 |
| keywords[7].display_name | Constraint satisfaction |
| language | |
| locations[0].id | doi:10.5281/zenodo.17850186 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400562 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | Zenodo (CERN European Organization for Nuclear Research) |
| locations[0].source.host_organization | https://openalex.org/I67311998 |
| locations[0].source.host_organization_name | European Organization for Nuclear Research |
| locations[0].source.host_organization_lineage | https://openalex.org/I67311998 |
| locations[0].license | cc-by |
| locations[0].pdf_url | |
| locations[0].version | |
| locations[0].raw_type | article |
| locations[0].license_id | https://openalex.org/licenses/cc-by |
| locations[0].is_accepted | False |
| locations[0].is_published | |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | https://doi.org/10.5281/zenodo.17850186 |
| indexed_in | datacite |
| authorships[0].author.id | |
| authorships[0].author.orcid | |
| authorships[0].author.display_name | Paunova, Eva |
| authorships[0].affiliations[0].raw_affiliation_string | NVIDIA |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Paunova, Eva |
| authorships[0].is_corresponding | True |
| authorships[0].raw_affiliation_strings | NVIDIA |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://doi.org/10.5281/zenodo.17850186 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-12-08T00:00:00 |
| display_name | Constraint Decomposition for Multi-Objective Instruction-Following in Large Language Models |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-12-08T23:26:33.048892 |
| primary_topic | |
| cited_by_count | 0 |
| locations_count | 1 |
| best_oa_location.id | doi:10.5281/zenodo.17850186 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400562 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | Zenodo (CERN European Organization for Nuclear Research) |
| best_oa_location.source.host_organization | https://openalex.org/I67311998 |
| best_oa_location.source.host_organization_name | European Organization for Nuclear Research |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I67311998 |
| best_oa_location.license | cc-by |
| best_oa_location.pdf_url | |
| best_oa_location.version | |
| best_oa_location.raw_type | article |
| best_oa_location.license_id | https://openalex.org/licenses/cc-by |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | https://doi.org/10.5281/zenodo.17850186 |
| primary_location.id | doi:10.5281/zenodo.17850186 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400562 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | Zenodo (CERN European Organization for Nuclear Research) |
| primary_location.source.host_organization | https://openalex.org/I67311998 |
| primary_location.source.host_organization_name | European Organization for Nuclear Research |
| primary_location.source.host_organization_lineage | https://openalex.org/I67311998 |
| primary_location.license | cc-by |
| primary_location.pdf_url | |
| primary_location.version | |
| primary_location.raw_type | article |
| primary_location.license_id | https://openalex.org/licenses/cc-by |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | https://doi.org/10.5281/zenodo.17850186 |
| publication_date | 2025-12-08 |
| publication_year | 2025 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 28, 112 |
| abstract_inverted_index.On | 101 |
| abstract_inverted_index.We | 24, 75 |
| abstract_inverted_index.at | 165 |
| abstract_inverted_index.of | 58, 129 |
| abstract_inverted_index.to | 64, 148 |
| abstract_inverted_index.54% | 128 |
| abstract_inverted_index.MT- | 156 |
| abstract_inverted_index.Our | 52, 145 |
| abstract_inverted_index.and | 42, 84, 141, 155, 161 |
| abstract_inverted_index.are | 163 |
| abstract_inverted_index.in- | 47 |
| abstract_inverted_index.op- | 99 |
| abstract_inverted_index.our | 105 |
| abstract_inverted_index.the | 55, 102, 130 |
| abstract_inverted_index.14%. | 144 |
| abstract_inverted_index.15%, | 140 |
| abstract_inverted_index.17%, | 137 |
| abstract_inverted_index.32.6 | 113 |
| abstract_inverted_index.Code | 160 |
| abstract_inverted_index.RLHF | 119 |
| abstract_inverted_index.back | 11 |
| abstract_inverted_index.con- | 21, 68 |
| abstract_inverted_index.data | 162 |
| abstract_inverted_index.each | 46 |
| abstract_inverted_index.from | 8 |
| abstract_inverted_index.into | 34 |
| abstract_inverted_index.more | 96 |
| abstract_inverted_index.over | 117 |
| abstract_inverted_index.show | 123 |
| abstract_inverted_index.that | 17, 30, 86, 124 |
| abstract_inverted_index.when | 71 |
| abstract_inverted_index.with | 5, 14, 80, 91, 133 |
| abstract_inverted_index.(+1.4 | 158 |
| abstract_inverted_index.73.8% | 108 |
| abstract_inverted_index.Bench | 157 |
| abstract_inverted_index.GSM8K | 149 |
| abstract_inverted_index.Large | 0 |
| abstract_inverted_index.feed- | 10 |
| abstract_inverted_index.human | 9, 82 |
| abstract_inverted_index.point | 115 |
| abstract_inverted_index.their | 62 |
| abstract_inverted_index.total | 131 |
| abstract_inverted_index.train | 76 |
| abstract_inverted_index.which | 66 |
| abstract_inverted_index.(+11.1 | 153 |
| abstract_inverted_index.(+15.6 | 150 |
| abstract_inverted_index.(LLMs) | 3 |
| abstract_inverted_index.(RLHF) | 12 |
| abstract_inverted_index.IFEval | 103 |
| abstract_inverted_index.adding | 136 |
| abstract_inverted_index.before | 49 |
| abstract_inverted_index.bundle | 18 |
| abstract_inverted_index.conict | 142 |
| abstract_inverted_index.failed | 70 |
| abstract_inverted_index.format | 40 |
| abstract_inverted_index.icting | 22 |
| abstract_inverted_index.method | 106, 146 |
| abstract_inverted_index.models | 2, 79 |
| abstract_inverted_index.reward | 60, 78 |
| abstract_inverted_index.specic | 67 |
| abstract_inverted_index.weight | 93, 138 |
| abstract_inverted_index.complex | 15 |
| abstract_inverted_index.conict. | 74 |
| abstract_inverted_index.eective | 97 |
| abstract_inverted_index.enables | 95 |
| abstract_inverted_index.models: | 61 |
| abstract_inverted_index.straint | 69 |
| abstract_inverted_index.studies | 122 |
| abstract_inverted_index.trained | 4 |
| abstract_inverted_index.(41.2%). | 120 |
| abstract_inverted_index.Ablation | 121 |
| abstract_inverted_index.accuracy | 110 |
| abstract_inverted_index.achieves | 107 |
| abstract_inverted_index.approach | 53 |
| abstract_inverted_index.combined | 90 |
| abstract_inverted_index.explicit | 87 |
| abstract_inverted_index.language | 1 |
| abstract_inverted_index.learning | 7 |
| abstract_inverted_index.multiple | 72 |
| abstract_inverted_index.points), | 151, 154 |
| abstract_inverted_index.points). | 159 |
| abstract_inverted_index.standard | 118 |
| abstract_inverted_index.struggle | 13 |
| abstract_inverted_index.(±1.6%), | 111 |
| abstract_inverted_index.HumanEval | 152 |
| abstract_inverted_index.addresses | 54 |
| abstract_inverted_index.available | 164 |
| abstract_inverted_index.detection | 143 |
| abstract_inverted_index.framework | 29 |
| abstract_inverted_index.inability | 63 |
| abstract_inverted_index.introduce | 25 |
| abstract_inverted_index.multiple, | 19 |
| abstract_inverted_index.optimizes | 45 |
| abstract_inverted_index.separates | 31 |
| abstract_inverted_index.benchmark, | 104 |
| abstract_inverted_index.constraint | 26, 88, 125 |
| abstract_inverted_index.decomposed | 77 |
| abstract_inverted_index.limitation | 57 |
| abstract_inverted_index.meta-level | 43 |
| abstract_inverted_index.monolithic | 59 |
| abstract_inverted_index.orthogonal | 35 |
| abstract_inverted_index.percentage | 114 |
| abstract_inverted_index.prediction | 139 |
| abstract_inverted_index.structural | 38 |
| abstract_inverted_index.adaptation, | 94 |
| abstract_inverted_index.combination | 135 |
| abstract_inverted_index.contributes | 127 |
| abstract_inverted_index.demonstrate | 85 |
| abstract_inverted_index.dependently | 48 |
| abstract_inverted_index.distinguish | 65 |
| abstract_inverted_index.fundamental | 56 |
| abstract_inverted_index.generalizes | 147 |
| abstract_inverted_index.improvement | 116 |
| abstract_inverted_index.potentially | 20 |
| abstract_inverted_index.preferences | 83 |
| abstract_inverted_index.separation, | 89 |
| abstract_inverted_index.timization. | 100 |
| abstract_inverted_index.aspect-level | 81 |
| abstract_inverted_index.combination. | 51 |
| abstract_inverted_index.conict-aware | 92 |
| abstract_inverted_index.correctness, | 37 |
| abstract_inverted_index.hierarchical | 50, 134 |
| abstract_inverted_index.improvement, | 132 |
| abstract_inverted_index.instructions | 16, 33 |
| abstract_inverted_index.prompt-level | 109 |
| abstract_inverted_index.requirements | 73 |
| abstract_inverted_index.decomposition | 126 |
| abstract_inverted_index.organization, | 39 |
| abstract_inverted_index.reinforcement | 6 |
| abstract_inverted_index.requirements. | 23 |
| abstract_inverted_index.specications, | 41 |
| abstract_inverted_index.decomposition, | 27 |
| abstract_inverted_index.multi-objective | 32, 98 |
| abstract_inverted_index.requirementsand | 44 |
| abstract_inverted_index.componentssemantic | 36 |
| abstract_inverted_index.https://github.com/epaunova/ | 166 |
| abstract_inverted_index.constraint-decomposition-llm. | 167 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 1 |
| citation_normalized_percentile |