Surrogate Likelihoods for Variational Annealed Importance Sampling Article Swipe
Martin Jankowiak
,
Du Phan
·
YOU?
·
· 2021
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2112.12194
YOU?
·
· 2021
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2112.12194
Variational inference is a powerful paradigm for approximate Bayesian inference with a number of appealing properties, including support for model learning and data subsampling. By contrast MCMC methods like Hamiltonian Monte Carlo do not share these properties but remain attractive since, contrary to parametric methods, MCMC is asymptotically unbiased. For these reasons researchers have sought to combine the strengths of both classes of algorithms, with recent approaches coming closer to realizing this vision in practice. However, supporting data subsampling in these hybrid methods can be a challenge, a shortcoming that we address by introducing a surrogate likelihood that can be learned jointly with other variational parameters. We argue theoretically that the resulting algorithm permits the user to make an intuitive trade-off between inference fidelity and computational cost. In an extensive empirical comparison we show that our method performs well in practice and that it is well-suited for black-box inference in probabilistic programming frameworks.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2112.12194
- https://arxiv.org/pdf/2112.12194
- OA Status
- green
- Cited By
- 2
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4226099274
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4226099274Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2112.12194Digital Object Identifier
- Title
-
Surrogate Likelihoods for Variational Annealed Importance SamplingWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2021Year of publication
- Publication date
-
2021-12-22Full publication date if available
- Authors
-
Martin Jankowiak, Du PhanList of authors in order
- Landing page
-
https://arxiv.org/abs/2112.12194Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2112.12194Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2112.12194Direct OA link when available
- Concepts
-
Computer science, Inference, Markov chain Monte Carlo, Machine learning, Bayesian inference, Artificial intelligence, Parametric statistics, Bayesian probability, Monte Carlo method, Fidelity, Algorithm, Mathematical optimization, Mathematics, Statistics, TelecommunicationsTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
2Total citation count in OpenAlex
- Citations by year (recent)
-
2024: 1, 2023: 1Per-year citation counts (last 5 years)
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4226099274 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2112.12194 |
| ids.doi | https://doi.org/10.48550/arxiv.2112.12194 |
| ids.openalex | https://openalex.org/W4226099274 |
| fwci | |
| type | preprint |
| title | Surrogate Likelihoods for Variational Annealed Importance Sampling |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T12814 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9991999864578247 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Gaussian Processes and Bayesian Inference |
| topics[1].id | https://openalex.org/T12072 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9968000054359436 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Machine Learning and Algorithms |
| topics[2].id | https://openalex.org/T11901 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9939000010490417 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1702 |
| topics[2].subfield.display_name | Artificial Intelligence |
| topics[2].display_name | Bayesian Methods and Mixture Models |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C41008148 |
| concepts[0].level | 0 |
| concepts[0].score | 0.6915929317474365 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[0].display_name | Computer science |
| concepts[1].id | https://openalex.org/C2776214188 |
| concepts[1].level | 2 |
| concepts[1].score | 0.6653604507446289 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q408386 |
| concepts[1].display_name | Inference |
| concepts[2].id | https://openalex.org/C111350023 |
| concepts[2].level | 3 |
| concepts[2].score | 0.6405816078186035 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q1191869 |
| concepts[2].display_name | Markov chain Monte Carlo |
| concepts[3].id | https://openalex.org/C119857082 |
| concepts[3].level | 1 |
| concepts[3].score | 0.5497182607650757 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q2539 |
| concepts[3].display_name | Machine learning |
| concepts[4].id | https://openalex.org/C160234255 |
| concepts[4].level | 3 |
| concepts[4].score | 0.5468173027038574 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q812535 |
| concepts[4].display_name | Bayesian inference |
| concepts[5].id | https://openalex.org/C154945302 |
| concepts[5].level | 1 |
| concepts[5].score | 0.4644119441509247 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[5].display_name | Artificial intelligence |
| concepts[6].id | https://openalex.org/C117251300 |
| concepts[6].level | 2 |
| concepts[6].score | 0.4406314492225647 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q1849855 |
| concepts[6].display_name | Parametric statistics |
| concepts[7].id | https://openalex.org/C107673813 |
| concepts[7].level | 2 |
| concepts[7].score | 0.432478129863739 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q812534 |
| concepts[7].display_name | Bayesian probability |
| concepts[8].id | https://openalex.org/C19499675 |
| concepts[8].level | 2 |
| concepts[8].score | 0.4227710962295532 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q232207 |
| concepts[8].display_name | Monte Carlo method |
| concepts[9].id | https://openalex.org/C2776459999 |
| concepts[9].level | 2 |
| concepts[9].score | 0.4155750274658203 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q2119376 |
| concepts[9].display_name | Fidelity |
| concepts[10].id | https://openalex.org/C11413529 |
| concepts[10].level | 1 |
| concepts[10].score | 0.393943190574646 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q8366 |
| concepts[10].display_name | Algorithm |
| concepts[11].id | https://openalex.org/C126255220 |
| concepts[11].level | 1 |
| concepts[11].score | 0.32047563791275024 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q141495 |
| concepts[11].display_name | Mathematical optimization |
| concepts[12].id | https://openalex.org/C33923547 |
| concepts[12].level | 0 |
| concepts[12].score | 0.20909202098846436 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q395 |
| concepts[12].display_name | Mathematics |
| concepts[13].id | https://openalex.org/C105795698 |
| concepts[13].level | 1 |
| concepts[13].score | 0.07541283965110779 |
| concepts[13].wikidata | https://www.wikidata.org/wiki/Q12483 |
| concepts[13].display_name | Statistics |
| concepts[14].id | https://openalex.org/C76155785 |
| concepts[14].level | 1 |
| concepts[14].score | 0.0 |
| concepts[14].wikidata | https://www.wikidata.org/wiki/Q418 |
| concepts[14].display_name | Telecommunications |
| keywords[0].id | https://openalex.org/keywords/computer-science |
| keywords[0].score | 0.6915929317474365 |
| keywords[0].display_name | Computer science |
| keywords[1].id | https://openalex.org/keywords/inference |
| keywords[1].score | 0.6653604507446289 |
| keywords[1].display_name | Inference |
| keywords[2].id | https://openalex.org/keywords/markov-chain-monte-carlo |
| keywords[2].score | 0.6405816078186035 |
| keywords[2].display_name | Markov chain Monte Carlo |
| keywords[3].id | https://openalex.org/keywords/machine-learning |
| keywords[3].score | 0.5497182607650757 |
| keywords[3].display_name | Machine learning |
| keywords[4].id | https://openalex.org/keywords/bayesian-inference |
| keywords[4].score | 0.5468173027038574 |
| keywords[4].display_name | Bayesian inference |
| keywords[5].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[5].score | 0.4644119441509247 |
| keywords[5].display_name | Artificial intelligence |
| keywords[6].id | https://openalex.org/keywords/parametric-statistics |
| keywords[6].score | 0.4406314492225647 |
| keywords[6].display_name | Parametric statistics |
| keywords[7].id | https://openalex.org/keywords/bayesian-probability |
| keywords[7].score | 0.432478129863739 |
| keywords[7].display_name | Bayesian probability |
| keywords[8].id | https://openalex.org/keywords/monte-carlo-method |
| keywords[8].score | 0.4227710962295532 |
| keywords[8].display_name | Monte Carlo method |
| keywords[9].id | https://openalex.org/keywords/fidelity |
| keywords[9].score | 0.4155750274658203 |
| keywords[9].display_name | Fidelity |
| keywords[10].id | https://openalex.org/keywords/algorithm |
| keywords[10].score | 0.393943190574646 |
| keywords[10].display_name | Algorithm |
| keywords[11].id | https://openalex.org/keywords/mathematical-optimization |
| keywords[11].score | 0.32047563791275024 |
| keywords[11].display_name | Mathematical optimization |
| keywords[12].id | https://openalex.org/keywords/mathematics |
| keywords[12].score | 0.20909202098846436 |
| keywords[12].display_name | Mathematics |
| keywords[13].id | https://openalex.org/keywords/statistics |
| keywords[13].score | 0.07541283965110779 |
| keywords[13].display_name | Statistics |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2112.12194 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2112.12194 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2112.12194 |
| locations[1].id | doi:10.48550/arxiv.2112.12194 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2112.12194 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5054350362 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-1724-0436 |
| authorships[0].author.display_name | Martin Jankowiak |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Jankowiak, Martin |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5102448182 |
| authorships[1].author.orcid | |
| authorships[1].author.display_name | Du Phan |
| authorships[1].author_position | last |
| authorships[1].raw_author_name | Phan, Du |
| authorships[1].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2112.12194 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Surrogate Likelihoods for Variational Annealed Importance Sampling |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T12814 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9991999864578247 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Gaussian Processes and Bayesian Inference |
| related_works | https://openalex.org/W3087071515, https://openalex.org/W4283077537, https://openalex.org/W2999603699, https://openalex.org/W2902858271, https://openalex.org/W2464065341, https://openalex.org/W2947536360, https://openalex.org/W3086697448, https://openalex.org/W1987558550, https://openalex.org/W2968689489, https://openalex.org/W4302573481 |
| cited_by_count | 2 |
| counts_by_year[0].year | 2024 |
| counts_by_year[0].cited_by_count | 1 |
| counts_by_year[1].year | 2023 |
| counts_by_year[1].cited_by_count | 1 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2112.12194 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2112.12194 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2112.12194 |
| primary_location.id | pmh:oai:arXiv.org:2112.12194 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2112.12194 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2112.12194 |
| publication_date | 2021-12-22 |
| publication_year | 2021 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 3, 11, 85, 87, 94 |
| abstract_inverted_index.By | 24 |
| abstract_inverted_index.In | 127 |
| abstract_inverted_index.We | 106 |
| abstract_inverted_index.an | 118, 128 |
| abstract_inverted_index.be | 84, 99 |
| abstract_inverted_index.by | 92 |
| abstract_inverted_index.do | 32 |
| abstract_inverted_index.in | 73, 79, 139, 149 |
| abstract_inverted_index.is | 2, 46, 144 |
| abstract_inverted_index.it | 143 |
| abstract_inverted_index.of | 13, 59, 62 |
| abstract_inverted_index.to | 42, 55, 69, 116 |
| abstract_inverted_index.we | 90, 132 |
| abstract_inverted_index.For | 49 |
| abstract_inverted_index.and | 21, 124, 141 |
| abstract_inverted_index.but | 37 |
| abstract_inverted_index.can | 83, 98 |
| abstract_inverted_index.for | 6, 18, 146 |
| abstract_inverted_index.not | 33 |
| abstract_inverted_index.our | 135 |
| abstract_inverted_index.the | 57, 110, 114 |
| abstract_inverted_index.MCMC | 26, 45 |
| abstract_inverted_index.both | 60 |
| abstract_inverted_index.data | 22, 77 |
| abstract_inverted_index.have | 53 |
| abstract_inverted_index.like | 28 |
| abstract_inverted_index.make | 117 |
| abstract_inverted_index.show | 133 |
| abstract_inverted_index.that | 89, 97, 109, 134, 142 |
| abstract_inverted_index.this | 71 |
| abstract_inverted_index.user | 115 |
| abstract_inverted_index.well | 138 |
| abstract_inverted_index.with | 10, 64, 102 |
| abstract_inverted_index.Carlo | 31 |
| abstract_inverted_index.Monte | 30 |
| abstract_inverted_index.argue | 107 |
| abstract_inverted_index.cost. | 126 |
| abstract_inverted_index.model | 19 |
| abstract_inverted_index.other | 103 |
| abstract_inverted_index.share | 34 |
| abstract_inverted_index.these | 35, 50, 80 |
| abstract_inverted_index.closer | 68 |
| abstract_inverted_index.coming | 67 |
| abstract_inverted_index.hybrid | 81 |
| abstract_inverted_index.method | 136 |
| abstract_inverted_index.number | 12 |
| abstract_inverted_index.recent | 65 |
| abstract_inverted_index.remain | 38 |
| abstract_inverted_index.since, | 40 |
| abstract_inverted_index.sought | 54 |
| abstract_inverted_index.vision | 72 |
| abstract_inverted_index.address | 91 |
| abstract_inverted_index.between | 121 |
| abstract_inverted_index.classes | 61 |
| abstract_inverted_index.combine | 56 |
| abstract_inverted_index.jointly | 101 |
| abstract_inverted_index.learned | 100 |
| abstract_inverted_index.methods | 27, 82 |
| abstract_inverted_index.permits | 113 |
| abstract_inverted_index.reasons | 51 |
| abstract_inverted_index.support | 17 |
| abstract_inverted_index.Bayesian | 8 |
| abstract_inverted_index.However, | 75 |
| abstract_inverted_index.contrary | 41 |
| abstract_inverted_index.contrast | 25 |
| abstract_inverted_index.fidelity | 123 |
| abstract_inverted_index.learning | 20 |
| abstract_inverted_index.methods, | 44 |
| abstract_inverted_index.paradigm | 5 |
| abstract_inverted_index.performs | 137 |
| abstract_inverted_index.powerful | 4 |
| abstract_inverted_index.practice | 140 |
| abstract_inverted_index.algorithm | 112 |
| abstract_inverted_index.appealing | 14 |
| abstract_inverted_index.black-box | 147 |
| abstract_inverted_index.empirical | 130 |
| abstract_inverted_index.extensive | 129 |
| abstract_inverted_index.including | 16 |
| abstract_inverted_index.inference | 1, 9, 122, 148 |
| abstract_inverted_index.intuitive | 119 |
| abstract_inverted_index.practice. | 74 |
| abstract_inverted_index.realizing | 70 |
| abstract_inverted_index.resulting | 111 |
| abstract_inverted_index.strengths | 58 |
| abstract_inverted_index.surrogate | 95 |
| abstract_inverted_index.trade-off | 120 |
| abstract_inverted_index.unbiased. | 48 |
| abstract_inverted_index.approaches | 66 |
| abstract_inverted_index.attractive | 39 |
| abstract_inverted_index.challenge, | 86 |
| abstract_inverted_index.comparison | 131 |
| abstract_inverted_index.likelihood | 96 |
| abstract_inverted_index.parametric | 43 |
| abstract_inverted_index.properties | 36 |
| abstract_inverted_index.supporting | 76 |
| abstract_inverted_index.Hamiltonian | 29 |
| abstract_inverted_index.Variational | 0 |
| abstract_inverted_index.algorithms, | 63 |
| abstract_inverted_index.approximate | 7 |
| abstract_inverted_index.frameworks. | 152 |
| abstract_inverted_index.introducing | 93 |
| abstract_inverted_index.parameters. | 105 |
| abstract_inverted_index.programming | 151 |
| abstract_inverted_index.properties, | 15 |
| abstract_inverted_index.researchers | 52 |
| abstract_inverted_index.shortcoming | 88 |
| abstract_inverted_index.subsampling | 78 |
| abstract_inverted_index.variational | 104 |
| abstract_inverted_index.well-suited | 145 |
| abstract_inverted_index.subsampling. | 23 |
| abstract_inverted_index.computational | 125 |
| abstract_inverted_index.probabilistic | 150 |
| abstract_inverted_index.theoretically | 108 |
| abstract_inverted_index.asymptotically | 47 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 2 |
| citation_normalized_percentile |