Diffusion Models as Constrained Samplers for Optimization with Unknown Constraints Article Swipe
Lingkai Kong
,
Yuanqi Du
,
Wenhao Mu
,
Kirill Neklyudov
,
Valentin De Bortoli
,
Haorui Wang
,
Dongxia Wu
,
Aaron Ferber
,
Yi-An Ma
,
Carla P. Gomes
,
Chao Zhang
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2402.18012
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2402.18012
Addressing real-world optimization problems becomes particularly challenging when analytic objective functions or constraints are unavailable. While numerous studies have addressed the issue of unknown objectives, limited research has focused on scenarios where feasibility constraints are not given explicitly. Overlooking these constraints can lead to spurious solutions that are unrealistic in practice. To deal with such unknown constraints, we propose to perform optimization within the data manifold using diffusion models. To constrain the optimization process to the data manifold, we reformulate the original optimization problem as a sampling problem from the product of the Boltzmann distribution defined by the objective function and the data distribution learned by the diffusion model. Depending on the differentiability of the objective function, we propose two different sampling methods. For differentiable objectives, we propose a two-stage framework that begins with a guided diffusion process for warm-up, followed by a Langevin dynamics stage for further correction. For non-differentiable objectives, we propose an iterative importance sampling strategy using the diffusion model as the proposal distribution. Comprehensive experiments on a synthetic dataset, six real-world black-box optimization datasets, and a multi-objective molecule optimization dataset show that our method achieves better or comparable performance with previous state-of-the-art baselines.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2402.18012
- https://arxiv.org/pdf/2402.18012
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4396570182
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4396570182Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2402.18012Digital Object Identifier
- Title
-
Diffusion Models as Constrained Samplers for Optimization with Unknown ConstraintsWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-02-28Full publication date if available
- Authors
-
Lingkai Kong, Yuanqi Du, Wenhao Mu, Kirill Neklyudov, Valentin De Bortoli, Haorui Wang, Dongxia Wu, Aaron Ferber, Yi-An Ma, Carla P. Gomes, Chao ZhangList of authors in order
- Landing page
-
https://arxiv.org/abs/2402.18012Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2402.18012Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2402.18012Direct OA link when available
- Concepts
-
Diffusion, Mathematical optimization, Computer science, Statistical physics, Mathematics, Physics, ThermodynamicsTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4396570182 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2402.18012 |
| ids.doi | https://doi.org/10.48550/arxiv.2402.18012 |
| ids.openalex | https://openalex.org/W4396570182 |
| fwci | |
| type | preprint |
| title | Diffusion Models as Constrained Samplers for Optimization with Unknown Constraints |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10791 |
| topics[0].field.id | https://openalex.org/fields/22 |
| topics[0].field.display_name | Engineering |
| topics[0].score | 0.08739999681711197 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/2207 |
| topics[0].subfield.display_name | Control and Systems Engineering |
| topics[0].display_name | Advanced Control Systems Optimization |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C69357855 |
| concepts[0].level | 2 |
| concepts[0].score | 0.6671106219291687 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q163214 |
| concepts[0].display_name | Diffusion |
| concepts[1].id | https://openalex.org/C126255220 |
| concepts[1].level | 1 |
| concepts[1].score | 0.4328896701335907 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q141495 |
| concepts[1].display_name | Mathematical optimization |
| concepts[2].id | https://openalex.org/C41008148 |
| concepts[2].level | 0 |
| concepts[2].score | 0.4271339476108551 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[2].display_name | Computer science |
| concepts[3].id | https://openalex.org/C121864883 |
| concepts[3].level | 1 |
| concepts[3].score | 0.33290761709213257 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q677916 |
| concepts[3].display_name | Statistical physics |
| concepts[4].id | https://openalex.org/C33923547 |
| concepts[4].level | 0 |
| concepts[4].score | 0.29127705097198486 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q395 |
| concepts[4].display_name | Mathematics |
| concepts[5].id | https://openalex.org/C121332964 |
| concepts[5].level | 0 |
| concepts[5].score | 0.23747387528419495 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q413 |
| concepts[5].display_name | Physics |
| concepts[6].id | https://openalex.org/C97355855 |
| concepts[6].level | 1 |
| concepts[6].score | 0.104102224111557 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q11473 |
| concepts[6].display_name | Thermodynamics |
| keywords[0].id | https://openalex.org/keywords/diffusion |
| keywords[0].score | 0.6671106219291687 |
| keywords[0].display_name | Diffusion |
| keywords[1].id | https://openalex.org/keywords/mathematical-optimization |
| keywords[1].score | 0.4328896701335907 |
| keywords[1].display_name | Mathematical optimization |
| keywords[2].id | https://openalex.org/keywords/computer-science |
| keywords[2].score | 0.4271339476108551 |
| keywords[2].display_name | Computer science |
| keywords[3].id | https://openalex.org/keywords/statistical-physics |
| keywords[3].score | 0.33290761709213257 |
| keywords[3].display_name | Statistical physics |
| keywords[4].id | https://openalex.org/keywords/mathematics |
| keywords[4].score | 0.29127705097198486 |
| keywords[4].display_name | Mathematics |
| keywords[5].id | https://openalex.org/keywords/physics |
| keywords[5].score | 0.23747387528419495 |
| keywords[5].display_name | Physics |
| keywords[6].id | https://openalex.org/keywords/thermodynamics |
| keywords[6].score | 0.104102224111557 |
| keywords[6].display_name | Thermodynamics |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2402.18012 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2402.18012 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2402.18012 |
| locations[1].id | doi:10.48550/arxiv.2402.18012 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | cc-by |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | https://openalex.org/licenses/cc-by |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2402.18012 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5102018639 |
| authorships[0].author.orcid | https://orcid.org/0000-0001-6480-513X |
| authorships[0].author.display_name | Lingkai Kong |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Kong, Lingkai |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5006709853 |
| authorships[1].author.orcid | https://orcid.org/0000-0003-2988-0374 |
| authorships[1].author.display_name | Yuanqi Du |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Du, Yuanqi |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5101346983 |
| authorships[2].author.orcid | |
| authorships[2].author.display_name | Wenhao Mu |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Mu, Wenhao |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5051657374 |
| authorships[3].author.orcid | |
| authorships[3].author.display_name | Kirill Neklyudov |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Neklyudov, Kirill |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5050614717 |
| authorships[4].author.orcid | https://orcid.org/0000-0002-7163-5391 |
| authorships[4].author.display_name | Valentin De Bortoli |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | De Bortoli, Valentin |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5101600525 |
| authorships[5].author.orcid | https://orcid.org/0009-0004-1480-5683 |
| authorships[5].author.display_name | Haorui Wang |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Wang, Haorui |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5101346984 |
| authorships[6].author.orcid | |
| authorships[6].author.display_name | Dongxia Wu |
| authorships[6].author_position | middle |
| authorships[6].raw_author_name | Wu, Dongxia |
| authorships[6].is_corresponding | False |
| authorships[7].author.id | https://openalex.org/A5027286402 |
| authorships[7].author.orcid | https://orcid.org/0000-0002-7422-0044 |
| authorships[7].author.display_name | Aaron Ferber |
| authorships[7].author_position | middle |
| authorships[7].raw_author_name | Ferber, Aaron |
| authorships[7].is_corresponding | False |
| authorships[8].author.id | https://openalex.org/A5041806431 |
| authorships[8].author.orcid | https://orcid.org/0000-0001-6074-6638 |
| authorships[8].author.display_name | Yi-An Ma |
| authorships[8].author_position | middle |
| authorships[8].raw_author_name | Ma, Yi-An |
| authorships[8].is_corresponding | False |
| authorships[9].author.id | https://openalex.org/A5069030030 |
| authorships[9].author.orcid | https://orcid.org/0000-0002-4441-7225 |
| authorships[9].author.display_name | Carla P. Gomes |
| authorships[9].author_position | middle |
| authorships[9].raw_author_name | Gomes, Carla P. |
| authorships[9].is_corresponding | False |
| authorships[10].author.id | https://openalex.org/A5100460151 |
| authorships[10].author.orcid | https://orcid.org/0000-0002-2019-6878 |
| authorships[10].author.display_name | Chao Zhang |
| authorships[10].author_position | last |
| authorships[10].raw_author_name | Zhang, Chao |
| authorships[10].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2402.18012 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Diffusion Models as Constrained Samplers for Optimization with Unknown Constraints |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10791 |
| primary_topic.field.id | https://openalex.org/fields/22 |
| primary_topic.field.display_name | Engineering |
| primary_topic.score | 0.08739999681711197 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/2207 |
| primary_topic.subfield.display_name | Control and Systems Engineering |
| primary_topic.display_name | Advanced Control Systems Optimization |
| related_works | https://openalex.org/W4391375266, https://openalex.org/W2748952813, https://openalex.org/W2390279801, https://openalex.org/W2358668433, https://openalex.org/W2376932109, https://openalex.org/W2001405890, https://openalex.org/W2382290278, https://openalex.org/W4395014643, https://openalex.org/W4391913857, https://openalex.org/W2350741829 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2402.18012 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2402.18012 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2402.18012 |
| primary_location.id | pmh:oai:arXiv.org:2402.18012 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2402.18012 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2402.18012 |
| publication_date | 2024-02-28 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 85, 128, 134, 142, 170, 179 |
| abstract_inverted_index.To | 51, 69 |
| abstract_inverted_index.an | 154 |
| abstract_inverted_index.as | 84, 163 |
| abstract_inverted_index.by | 96, 105, 141 |
| abstract_inverted_index.in | 49 |
| abstract_inverted_index.of | 22, 91, 113 |
| abstract_inverted_index.on | 29, 110, 169 |
| abstract_inverted_index.or | 11, 190 |
| abstract_inverted_index.to | 43, 59, 74 |
| abstract_inverted_index.we | 57, 78, 117, 126, 152 |
| abstract_inverted_index.For | 123, 149 |
| abstract_inverted_index.and | 100, 178 |
| abstract_inverted_index.are | 13, 34, 47 |
| abstract_inverted_index.can | 41 |
| abstract_inverted_index.for | 138, 146 |
| abstract_inverted_index.has | 27 |
| abstract_inverted_index.not | 35 |
| abstract_inverted_index.our | 186 |
| abstract_inverted_index.six | 173 |
| abstract_inverted_index.the | 20, 63, 71, 75, 80, 89, 92, 97, 101, 106, 111, 114, 160, 164 |
| abstract_inverted_index.two | 119 |
| abstract_inverted_index.data | 64, 76, 102 |
| abstract_inverted_index.deal | 52 |
| abstract_inverted_index.from | 88 |
| abstract_inverted_index.have | 18 |
| abstract_inverted_index.lead | 42 |
| abstract_inverted_index.show | 184 |
| abstract_inverted_index.such | 54 |
| abstract_inverted_index.that | 46, 131, 185 |
| abstract_inverted_index.when | 7 |
| abstract_inverted_index.with | 53, 133, 193 |
| abstract_inverted_index.While | 15 |
| abstract_inverted_index.given | 36 |
| abstract_inverted_index.issue | 21 |
| abstract_inverted_index.model | 162 |
| abstract_inverted_index.stage | 145 |
| abstract_inverted_index.these | 39 |
| abstract_inverted_index.using | 66, 159 |
| abstract_inverted_index.where | 31 |
| abstract_inverted_index.begins | 132 |
| abstract_inverted_index.better | 189 |
| abstract_inverted_index.guided | 135 |
| abstract_inverted_index.method | 187 |
| abstract_inverted_index.model. | 108 |
| abstract_inverted_index.within | 62 |
| abstract_inverted_index.becomes | 4 |
| abstract_inverted_index.dataset | 183 |
| abstract_inverted_index.defined | 95 |
| abstract_inverted_index.focused | 28 |
| abstract_inverted_index.further | 147 |
| abstract_inverted_index.learned | 104 |
| abstract_inverted_index.limited | 25 |
| abstract_inverted_index.models. | 68 |
| abstract_inverted_index.perform | 60 |
| abstract_inverted_index.problem | 83, 87 |
| abstract_inverted_index.process | 73, 137 |
| abstract_inverted_index.product | 90 |
| abstract_inverted_index.propose | 58, 118, 127, 153 |
| abstract_inverted_index.studies | 17 |
| abstract_inverted_index.unknown | 23, 55 |
| abstract_inverted_index.Langevin | 143 |
| abstract_inverted_index.achieves | 188 |
| abstract_inverted_index.analytic | 8 |
| abstract_inverted_index.dataset, | 172 |
| abstract_inverted_index.dynamics | 144 |
| abstract_inverted_index.followed | 140 |
| abstract_inverted_index.function | 99 |
| abstract_inverted_index.manifold | 65 |
| abstract_inverted_index.methods. | 122 |
| abstract_inverted_index.molecule | 181 |
| abstract_inverted_index.numerous | 16 |
| abstract_inverted_index.original | 81 |
| abstract_inverted_index.previous | 194 |
| abstract_inverted_index.problems | 3 |
| abstract_inverted_index.proposal | 165 |
| abstract_inverted_index.research | 26 |
| abstract_inverted_index.sampling | 86, 121, 157 |
| abstract_inverted_index.spurious | 44 |
| abstract_inverted_index.strategy | 158 |
| abstract_inverted_index.warm-up, | 139 |
| abstract_inverted_index.Boltzmann | 93 |
| abstract_inverted_index.Depending | 109 |
| abstract_inverted_index.addressed | 19 |
| abstract_inverted_index.black-box | 175 |
| abstract_inverted_index.constrain | 70 |
| abstract_inverted_index.datasets, | 177 |
| abstract_inverted_index.different | 120 |
| abstract_inverted_index.diffusion | 67, 107, 136, 161 |
| abstract_inverted_index.framework | 130 |
| abstract_inverted_index.function, | 116 |
| abstract_inverted_index.functions | 10 |
| abstract_inverted_index.iterative | 155 |
| abstract_inverted_index.manifold, | 77 |
| abstract_inverted_index.objective | 9, 98, 115 |
| abstract_inverted_index.practice. | 50 |
| abstract_inverted_index.scenarios | 30 |
| abstract_inverted_index.solutions | 45 |
| abstract_inverted_index.synthetic | 171 |
| abstract_inverted_index.two-stage | 129 |
| abstract_inverted_index.Addressing | 0 |
| abstract_inverted_index.baselines. | 196 |
| abstract_inverted_index.comparable | 191 |
| abstract_inverted_index.importance | 156 |
| abstract_inverted_index.real-world | 1, 174 |
| abstract_inverted_index.Overlooking | 38 |
| abstract_inverted_index.challenging | 6 |
| abstract_inverted_index.constraints | 12, 33, 40 |
| abstract_inverted_index.correction. | 148 |
| abstract_inverted_index.experiments | 168 |
| abstract_inverted_index.explicitly. | 37 |
| abstract_inverted_index.feasibility | 32 |
| abstract_inverted_index.objectives, | 24, 125, 151 |
| abstract_inverted_index.performance | 192 |
| abstract_inverted_index.reformulate | 79 |
| abstract_inverted_index.unrealistic | 48 |
| abstract_inverted_index.constraints, | 56 |
| abstract_inverted_index.distribution | 94, 103 |
| abstract_inverted_index.optimization | 2, 61, 72, 82, 176, 182 |
| abstract_inverted_index.particularly | 5 |
| abstract_inverted_index.unavailable. | 14 |
| abstract_inverted_index.Comprehensive | 167 |
| abstract_inverted_index.distribution. | 166 |
| abstract_inverted_index.differentiable | 124 |
| abstract_inverted_index.multi-objective | 180 |
| abstract_inverted_index.state-of-the-art | 195 |
| abstract_inverted_index.differentiability | 112 |
| abstract_inverted_index.non-differentiable | 150 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 11 |
| citation_normalized_percentile |