Faster Sampling via Stochastic Gradient Proximal Sampler Article Swipe
Xunpeng Huang
,
Difan Zou
,
Yi-An Ma
,
Hanze Dong
,
Tong Zhang
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2405.16734
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2405.16734
Stochastic gradients have been widely integrated into Langevin-based methods to improve their scalability and efficiency in solving large-scale sampling problems. However, the proximal sampler, which exhibits much faster convergence than Langevin-based algorithms in the deterministic setting Lee et al. (2021), has yet to be explored in its stochastic variants. In this paper, we study the Stochastic Proximal Samplers (SPS) for sampling from non-log-concave distributions. We first establish a general framework for implementing stochastic proximal samplers and establish the convergence theory accordingly. We show that the convergence to the target distribution can be guaranteed as long as the second moment of the algorithm trajectory is bounded and restricted Gaussian oracles can be well approximated. We then provide two implementable variants based on Stochastic gradient Langevin dynamics (SGLD) and Metropolis-adjusted Langevin algorithm (MALA), giving rise to SPS-SGLD and SPS-MALA. We further show that SPS-SGLD and SPS-MALA can achieve $ε$-sampling error in total variation (TV) distance within $\tilde{\mathcal{O}}(dε^{-2})$ and $\tilde{\mathcal{O}}(d^{1/2}ε^{-2})$ gradient complexities, which outperform the best-known result by at least an $\tilde{\mathcal{O}}(d^{1/3})$ factor. This enhancement in performance is corroborated by our empirical studies on synthetic data with various dimensions, demonstrating the efficiency of our proposed algorithm.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2405.16734
- https://arxiv.org/pdf/2405.16734
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4399115949
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4399115949Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2405.16734Digital Object Identifier
- Title
-
Faster Sampling via Stochastic Gradient Proximal SamplerWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-05-27Full publication date if available
- Authors
-
Xunpeng Huang, Difan Zou, Yi-An Ma, Hanze Dong, Tong ZhangList of authors in order
- Landing page
-
https://arxiv.org/abs/2405.16734Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2405.16734Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2405.16734Direct OA link when available
- Concepts
-
Sampling (signal processing), Computer science, Mathematics, Statistics, Telecommunications, DetectorTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4399115949 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2405.16734 |
| ids.doi | https://doi.org/10.48550/arxiv.2405.16734 |
| ids.openalex | https://openalex.org/W4399115949 |
| fwci | |
| type | preprint |
| title | Faster Sampling via Stochastic Gradient Proximal Sampler |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10500 |
| topics[0].field.id | https://openalex.org/fields/22 |
| topics[0].field.display_name | Engineering |
| topics[0].score | 0.9983999729156494 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/2206 |
| topics[0].subfield.display_name | Computational Mechanics |
| topics[0].display_name | Sparse and Compressive Sensing Techniques |
| topics[1].id | https://openalex.org/T12814 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9799000024795532 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Gaussian Processes and Bayesian Inference |
| topics[2].id | https://openalex.org/T11612 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9742000102996826 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1702 |
| topics[2].subfield.display_name | Artificial Intelligence |
| topics[2].display_name | Stochastic Gradient Optimization Techniques |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C140779682 |
| concepts[0].level | 3 |
| concepts[0].score | 0.532184362411499 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q210868 |
| concepts[0].display_name | Sampling (signal processing) |
| concepts[1].id | https://openalex.org/C41008148 |
| concepts[1].level | 0 |
| concepts[1].score | 0.3740665316581726 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[1].display_name | Computer science |
| concepts[2].id | https://openalex.org/C33923547 |
| concepts[2].level | 0 |
| concepts[2].score | 0.3725208044052124 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q395 |
| concepts[2].display_name | Mathematics |
| concepts[3].id | https://openalex.org/C105795698 |
| concepts[3].level | 1 |
| concepts[3].score | 0.35451820492744446 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q12483 |
| concepts[3].display_name | Statistics |
| concepts[4].id | https://openalex.org/C76155785 |
| concepts[4].level | 1 |
| concepts[4].score | 0.11330363154411316 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q418 |
| concepts[4].display_name | Telecommunications |
| concepts[5].id | https://openalex.org/C94915269 |
| concepts[5].level | 2 |
| concepts[5].score | 0.0 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q1834857 |
| concepts[5].display_name | Detector |
| keywords[0].id | https://openalex.org/keywords/sampling |
| keywords[0].score | 0.532184362411499 |
| keywords[0].display_name | Sampling (signal processing) |
| keywords[1].id | https://openalex.org/keywords/computer-science |
| keywords[1].score | 0.3740665316581726 |
| keywords[1].display_name | Computer science |
| keywords[2].id | https://openalex.org/keywords/mathematics |
| keywords[2].score | 0.3725208044052124 |
| keywords[2].display_name | Mathematics |
| keywords[3].id | https://openalex.org/keywords/statistics |
| keywords[3].score | 0.35451820492744446 |
| keywords[3].display_name | Statistics |
| keywords[4].id | https://openalex.org/keywords/telecommunications |
| keywords[4].score | 0.11330363154411316 |
| keywords[4].display_name | Telecommunications |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2405.16734 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2405.16734 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2405.16734 |
| locations[1].id | doi:10.48550/arxiv.2405.16734 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | cc-by |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | https://openalex.org/licenses/cc-by |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2405.16734 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5083207127 |
| authorships[0].author.orcid | |
| authorships[0].author.display_name | Xunpeng Huang |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Huang, Xunpeng |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5085848346 |
| authorships[1].author.orcid | https://orcid.org/0000-0002-6544-2593 |
| authorships[1].author.display_name | Difan Zou |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Zou, Difan |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5041806431 |
| authorships[2].author.orcid | https://orcid.org/0000-0001-6074-6638 |
| authorships[2].author.display_name | Yi-An Ma |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Ma, Yi-An |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5061778641 |
| authorships[3].author.orcid | https://orcid.org/0000-0002-8846-1260 |
| authorships[3].author.display_name | Hanze Dong |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Dong, Hanze |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5008509353 |
| authorships[4].author.orcid | https://orcid.org/0000-0002-9657-5129 |
| authorships[4].author.display_name | Tong Zhang |
| authorships[4].author_position | last |
| authorships[4].raw_author_name | Zhang, Tong |
| authorships[4].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2405.16734 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Faster Sampling via Stochastic Gradient Proximal Sampler |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10500 |
| primary_topic.field.id | https://openalex.org/fields/22 |
| primary_topic.field.display_name | Engineering |
| primary_topic.score | 0.9983999729156494 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/2206 |
| primary_topic.subfield.display_name | Computational Mechanics |
| primary_topic.display_name | Sparse and Compressive Sensing Techniques |
| related_works | https://openalex.org/W2748952813, https://openalex.org/W4391375266, https://openalex.org/W1979597421, https://openalex.org/W2007980826, https://openalex.org/W2061531152, https://openalex.org/W3002753104, https://openalex.org/W2077600819, https://openalex.org/W2142036596, https://openalex.org/W2072657027, https://openalex.org/W3107697994 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2405.16734 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2405.16734 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2405.16734 |
| primary_location.id | pmh:oai:arXiv.org:2405.16734 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2405.16734 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2405.16734 |
| publication_date | 2024-05-27 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 67 |
| abstract_inverted_index.In | 49 |
| abstract_inverted_index.We | 64, 81, 113, 137 |
| abstract_inverted_index.an | 167 |
| abstract_inverted_index.as | 93, 95 |
| abstract_inverted_index.at | 165 |
| abstract_inverted_index.be | 43, 91, 110 |
| abstract_inverted_index.by | 164, 176 |
| abstract_inverted_index.et | 37 |
| abstract_inverted_index.in | 15, 32, 45, 148, 172 |
| abstract_inverted_index.is | 103, 174 |
| abstract_inverted_index.of | 99, 189 |
| abstract_inverted_index.on | 120, 180 |
| abstract_inverted_index.to | 9, 42, 86, 133 |
| abstract_inverted_index.we | 52 |
| abstract_inverted_index.Lee | 36 |
| abstract_inverted_index.al. | 38 |
| abstract_inverted_index.and | 13, 75, 105, 126, 135, 142, 155 |
| abstract_inverted_index.can | 90, 109, 144 |
| abstract_inverted_index.for | 59, 70 |
| abstract_inverted_index.has | 40 |
| abstract_inverted_index.its | 46 |
| abstract_inverted_index.our | 177, 190 |
| abstract_inverted_index.the | 21, 33, 54, 77, 84, 87, 96, 100, 161, 187 |
| abstract_inverted_index.two | 116 |
| abstract_inverted_index.yet | 41 |
| abstract_inverted_index.(TV) | 151 |
| abstract_inverted_index.This | 170 |
| abstract_inverted_index.been | 3 |
| abstract_inverted_index.data | 182 |
| abstract_inverted_index.from | 61 |
| abstract_inverted_index.have | 2 |
| abstract_inverted_index.into | 6 |
| abstract_inverted_index.long | 94 |
| abstract_inverted_index.much | 26 |
| abstract_inverted_index.rise | 132 |
| abstract_inverted_index.show | 82, 139 |
| abstract_inverted_index.than | 29 |
| abstract_inverted_index.that | 83, 140 |
| abstract_inverted_index.then | 114 |
| abstract_inverted_index.this | 50 |
| abstract_inverted_index.well | 111 |
| abstract_inverted_index.with | 183 |
| abstract_inverted_index.(SPS) | 58 |
| abstract_inverted_index.based | 119 |
| abstract_inverted_index.error | 147 |
| abstract_inverted_index.first | 65 |
| abstract_inverted_index.least | 166 |
| abstract_inverted_index.study | 53 |
| abstract_inverted_index.their | 11 |
| abstract_inverted_index.total | 149 |
| abstract_inverted_index.which | 24, 159 |
| abstract_inverted_index.(SGLD) | 125 |
| abstract_inverted_index.faster | 27 |
| abstract_inverted_index.giving | 131 |
| abstract_inverted_index.moment | 98 |
| abstract_inverted_index.paper, | 51 |
| abstract_inverted_index.result | 163 |
| abstract_inverted_index.second | 97 |
| abstract_inverted_index.target | 88 |
| abstract_inverted_index.theory | 79 |
| abstract_inverted_index.widely | 4 |
| abstract_inverted_index.within | 153 |
| abstract_inverted_index.(2021), | 39 |
| abstract_inverted_index.(MALA), | 130 |
| abstract_inverted_index.achieve | 145 |
| abstract_inverted_index.bounded | 104 |
| abstract_inverted_index.factor. | 169 |
| abstract_inverted_index.further | 138 |
| abstract_inverted_index.general | 68 |
| abstract_inverted_index.improve | 10 |
| abstract_inverted_index.methods | 8 |
| abstract_inverted_index.oracles | 108 |
| abstract_inverted_index.provide | 115 |
| abstract_inverted_index.setting | 35 |
| abstract_inverted_index.solving | 16 |
| abstract_inverted_index.studies | 179 |
| abstract_inverted_index.various | 184 |
| abstract_inverted_index.Gaussian | 107 |
| abstract_inverted_index.However, | 20 |
| abstract_inverted_index.Langevin | 123, 128 |
| abstract_inverted_index.Proximal | 56 |
| abstract_inverted_index.SPS-MALA | 143 |
| abstract_inverted_index.SPS-SGLD | 134, 141 |
| abstract_inverted_index.Samplers | 57 |
| abstract_inverted_index.distance | 152 |
| abstract_inverted_index.dynamics | 124 |
| abstract_inverted_index.exhibits | 25 |
| abstract_inverted_index.explored | 44 |
| abstract_inverted_index.gradient | 122, 157 |
| abstract_inverted_index.proposed | 191 |
| abstract_inverted_index.proximal | 22, 73 |
| abstract_inverted_index.sampler, | 23 |
| abstract_inverted_index.samplers | 74 |
| abstract_inverted_index.sampling | 18, 60 |
| abstract_inverted_index.variants | 118 |
| abstract_inverted_index.SPS-MALA. | 136 |
| abstract_inverted_index.algorithm | 101, 129 |
| abstract_inverted_index.empirical | 178 |
| abstract_inverted_index.establish | 66, 76 |
| abstract_inverted_index.framework | 69 |
| abstract_inverted_index.gradients | 1 |
| abstract_inverted_index.problems. | 19 |
| abstract_inverted_index.synthetic | 181 |
| abstract_inverted_index.variants. | 48 |
| abstract_inverted_index.variation | 150 |
| abstract_inverted_index.Stochastic | 0, 55, 121 |
| abstract_inverted_index.algorithm. | 192 |
| abstract_inverted_index.algorithms | 31 |
| abstract_inverted_index.best-known | 162 |
| abstract_inverted_index.efficiency | 14, 188 |
| abstract_inverted_index.guaranteed | 92 |
| abstract_inverted_index.integrated | 5 |
| abstract_inverted_index.outperform | 160 |
| abstract_inverted_index.restricted | 106 |
| abstract_inverted_index.stochastic | 47, 72 |
| abstract_inverted_index.trajectory | 102 |
| abstract_inverted_index.convergence | 28, 78, 85 |
| abstract_inverted_index.dimensions, | 185 |
| abstract_inverted_index.enhancement | 171 |
| abstract_inverted_index.large-scale | 17 |
| abstract_inverted_index.performance | 173 |
| abstract_inverted_index.scalability | 12 |
| abstract_inverted_index.accordingly. | 80 |
| abstract_inverted_index.corroborated | 175 |
| abstract_inverted_index.distribution | 89 |
| abstract_inverted_index.implementing | 71 |
| abstract_inverted_index.$ε$-sampling | 146 |
| abstract_inverted_index.approximated. | 112 |
| abstract_inverted_index.complexities, | 158 |
| abstract_inverted_index.demonstrating | 186 |
| abstract_inverted_index.deterministic | 34 |
| abstract_inverted_index.implementable | 117 |
| abstract_inverted_index.Langevin-based | 7, 30 |
| abstract_inverted_index.distributions. | 63 |
| abstract_inverted_index.non-log-concave | 62 |
| abstract_inverted_index.Metropolis-adjusted | 127 |
| abstract_inverted_index.$\tilde{\mathcal{O}}(d^{1/3})$ | 168 |
| abstract_inverted_index.$\tilde{\mathcal{O}}(dε^{-2})$ | 154 |
| abstract_inverted_index.$\tilde{\mathcal{O}}(d^{1/2}ε^{-2})$ | 156 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 5 |
| citation_normalized_percentile |