Causal Contextual Bandits with Adaptive Context Article Swipe
Rahul Madhavan
,
Aurghya Maiti
,
Gaurav Sinha
,
Siddharth Barman
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2405.18626
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2405.18626
We study a variant of causal contextual bandits where the context is chosen based on an initial intervention chosen by the learner. At the beginning of each round, the learner selects an initial action, depending on which a stochastic context is revealed by the environment. Following this, the learner then selects a final action and receives a reward. Given $T$ rounds of interactions with the environment, the objective of the learner is to learn a policy (of selecting the initial and the final action) with maximum expected reward. In this paper we study the specific situation where every action corresponds to intervening on a node in some known causal graph. We extend prior work from the deterministic context setting to obtain simple regret minimization guarantees. This is achieved through an instance-dependent causal parameter, $λ$, which characterizes our upper bound. Furthermore, we prove that our simple regret is essentially tight for a large class of instances. A key feature of our work is that we use convex optimization to address the bandit exploration problem. We also conduct experiments to validate our theoretical results, and release our code at our project GitHub repository: https://github.com/adaptiveContextualCausalBandits/aCCB.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2405.18626
- https://arxiv.org/pdf/2405.18626
- OA Status
- green
- Cited By
- 1
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4399197950
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4399197950Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2405.18626Digital Object Identifier
- Title
-
Causal Contextual Bandits with Adaptive ContextWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-05-28Full publication date if available
- Authors
-
Rahul Madhavan, Aurghya Maiti, Gaurav Sinha, Siddharth BarmanList of authors in order
- Landing page
-
https://arxiv.org/abs/2405.18626Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2405.18626Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2405.18626Direct OA link when available
- Concepts
-
Context (archaeology), Computer science, Cognitive psychology, Psychology, Artificial intelligence, History, ArchaeologyTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
1Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 1Per-year citation counts (last 5 years)
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4399197950 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2405.18626 |
| ids.doi | https://doi.org/10.48550/arxiv.2405.18626 |
| ids.openalex | https://openalex.org/W4399197950 |
| fwci | |
| type | preprint |
| title | Causal Contextual Bandits with Adaptive Context |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T12101 |
| topics[0].field.id | https://openalex.org/fields/18 |
| topics[0].field.display_name | Decision Sciences |
| topics[0].score | 0.9934999942779541 |
| topics[0].domain.id | https://openalex.org/domains/2 |
| topics[0].domain.display_name | Social Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1803 |
| topics[0].subfield.display_name | Management Science and Operations Research |
| topics[0].display_name | Advanced Bandit Algorithms Research |
| topics[1].id | https://openalex.org/T11147 |
| topics[1].field.id | https://openalex.org/fields/33 |
| topics[1].field.display_name | Social Sciences |
| topics[1].score | 0.9753000140190125 |
| topics[1].domain.id | https://openalex.org/domains/2 |
| topics[1].domain.display_name | Social Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/3312 |
| topics[1].subfield.display_name | Sociology and Political Science |
| topics[1].display_name | Misinformation and Its Impacts |
| topics[2].id | https://openalex.org/T10315 |
| topics[2].field.id | https://openalex.org/fields/18 |
| topics[2].field.display_name | Decision Sciences |
| topics[2].score | 0.9675999879837036 |
| topics[2].domain.id | https://openalex.org/domains/2 |
| topics[2].domain.display_name | Social Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1800 |
| topics[2].subfield.display_name | General Decision Sciences |
| topics[2].display_name | Decision-Making and Behavioral Economics |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C2779343474 |
| concepts[0].level | 2 |
| concepts[0].score | 0.6424574255943298 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q3109175 |
| concepts[0].display_name | Context (archaeology) |
| concepts[1].id | https://openalex.org/C41008148 |
| concepts[1].level | 0 |
| concepts[1].score | 0.5070593357086182 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[1].display_name | Computer science |
| concepts[2].id | https://openalex.org/C180747234 |
| concepts[2].level | 1 |
| concepts[2].score | 0.3913443982601166 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q23373 |
| concepts[2].display_name | Cognitive psychology |
| concepts[3].id | https://openalex.org/C15744967 |
| concepts[3].level | 0 |
| concepts[3].score | 0.3895125389099121 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q9418 |
| concepts[3].display_name | Psychology |
| concepts[4].id | https://openalex.org/C154945302 |
| concepts[4].level | 1 |
| concepts[4].score | 0.3620189428329468 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[4].display_name | Artificial intelligence |
| concepts[5].id | https://openalex.org/C95457728 |
| concepts[5].level | 0 |
| concepts[5].score | 0.18568691611289978 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q309 |
| concepts[5].display_name | History |
| concepts[6].id | https://openalex.org/C166957645 |
| concepts[6].level | 1 |
| concepts[6].score | 0.0 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q23498 |
| concepts[6].display_name | Archaeology |
| keywords[0].id | https://openalex.org/keywords/context |
| keywords[0].score | 0.6424574255943298 |
| keywords[0].display_name | Context (archaeology) |
| keywords[1].id | https://openalex.org/keywords/computer-science |
| keywords[1].score | 0.5070593357086182 |
| keywords[1].display_name | Computer science |
| keywords[2].id | https://openalex.org/keywords/cognitive-psychology |
| keywords[2].score | 0.3913443982601166 |
| keywords[2].display_name | Cognitive psychology |
| keywords[3].id | https://openalex.org/keywords/psychology |
| keywords[3].score | 0.3895125389099121 |
| keywords[3].display_name | Psychology |
| keywords[4].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[4].score | 0.3620189428329468 |
| keywords[4].display_name | Artificial intelligence |
| keywords[5].id | https://openalex.org/keywords/history |
| keywords[5].score | 0.18568691611289978 |
| keywords[5].display_name | History |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2405.18626 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2405.18626 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2405.18626 |
| locations[1].id | doi:10.48550/arxiv.2405.18626 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2405.18626 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5081959578 |
| authorships[0].author.orcid | |
| authorships[0].author.display_name | Rahul Madhavan |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Madhavan, Rahul |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5005957014 |
| authorships[1].author.orcid | https://orcid.org/0000-0001-8231-0165 |
| authorships[1].author.display_name | Aurghya Maiti |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Maiti, Aurghya |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5101951616 |
| authorships[2].author.orcid | https://orcid.org/0000-0002-3590-9543 |
| authorships[2].author.display_name | Gaurav Sinha |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Sinha, Gaurav |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5039018238 |
| authorships[3].author.orcid | https://orcid.org/0000-0001-9276-2181 |
| authorships[3].author.display_name | Siddharth Barman |
| authorships[3].author_position | last |
| authorships[3].raw_author_name | Barman, Siddharth |
| authorships[3].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2405.18626 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Causal Contextual Bandits with Adaptive Context |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T12101 |
| primary_topic.field.id | https://openalex.org/fields/18 |
| primary_topic.field.display_name | Decision Sciences |
| primary_topic.score | 0.9934999942779541 |
| primary_topic.domain.id | https://openalex.org/domains/2 |
| primary_topic.domain.display_name | Social Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1803 |
| primary_topic.subfield.display_name | Management Science and Operations Research |
| primary_topic.display_name | Advanced Bandit Algorithms Research |
| related_works | https://openalex.org/W4391375266, https://openalex.org/W2748952813, https://openalex.org/W2390279801, https://openalex.org/W2358668433, https://openalex.org/W4396701345, https://openalex.org/W2376932109, https://openalex.org/W2001405890, https://openalex.org/W4396696052, https://openalex.org/W2382290278, https://openalex.org/W4395014643 |
| cited_by_count | 1 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 1 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2405.18626 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2405.18626 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2405.18626 |
| primary_location.id | pmh:oai:arXiv.org:2405.18626 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2405.18626 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2405.18626 |
| publication_date | 2024-05-28 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.A | 155 |
| abstract_inverted_index.a | 2, 37, 51, 56, 74, 103, 150 |
| abstract_inverted_index.At | 22 |
| abstract_inverted_index.In | 88 |
| abstract_inverted_index.We | 0, 110, 173 |
| abstract_inverted_index.an | 15, 31, 129 |
| abstract_inverted_index.at | 186 |
| abstract_inverted_index.by | 19, 42 |
| abstract_inverted_index.in | 105 |
| abstract_inverted_index.is | 11, 40, 71, 126, 146, 161 |
| abstract_inverted_index.of | 4, 25, 61, 68, 153, 158 |
| abstract_inverted_index.on | 14, 35, 102 |
| abstract_inverted_index.to | 72, 100, 119, 167, 177 |
| abstract_inverted_index.we | 91, 140, 163 |
| abstract_inverted_index.$T$ | 59 |
| abstract_inverted_index.(of | 76 |
| abstract_inverted_index.and | 54, 80, 182 |
| abstract_inverted_index.for | 149 |
| abstract_inverted_index.key | 156 |
| abstract_inverted_index.our | 136, 143, 159, 179, 184, 187 |
| abstract_inverted_index.the | 9, 20, 23, 28, 43, 47, 64, 66, 69, 78, 81, 93, 115, 169 |
| abstract_inverted_index.use | 164 |
| abstract_inverted_index.This | 125 |
| abstract_inverted_index.also | 174 |
| abstract_inverted_index.code | 185 |
| abstract_inverted_index.each | 26 |
| abstract_inverted_index.from | 114 |
| abstract_inverted_index.node | 104 |
| abstract_inverted_index.some | 106 |
| abstract_inverted_index.that | 142, 162 |
| abstract_inverted_index.then | 49 |
| abstract_inverted_index.this | 89 |
| abstract_inverted_index.with | 63, 84 |
| abstract_inverted_index.work | 113, 160 |
| abstract_inverted_index.$λ$, | 133 |
| abstract_inverted_index.Given | 58 |
| abstract_inverted_index.based | 13 |
| abstract_inverted_index.class | 152 |
| abstract_inverted_index.every | 97 |
| abstract_inverted_index.final | 52, 82 |
| abstract_inverted_index.known | 107 |
| abstract_inverted_index.large | 151 |
| abstract_inverted_index.learn | 73 |
| abstract_inverted_index.paper | 90 |
| abstract_inverted_index.prior | 112 |
| abstract_inverted_index.prove | 141 |
| abstract_inverted_index.study | 1, 92 |
| abstract_inverted_index.this, | 46 |
| abstract_inverted_index.tight | 148 |
| abstract_inverted_index.upper | 137 |
| abstract_inverted_index.where | 8, 96 |
| abstract_inverted_index.which | 36, 134 |
| abstract_inverted_index.GitHub | 189 |
| abstract_inverted_index.action | 53, 98 |
| abstract_inverted_index.bandit | 170 |
| abstract_inverted_index.bound. | 138 |
| abstract_inverted_index.causal | 5, 108, 131 |
| abstract_inverted_index.chosen | 12, 18 |
| abstract_inverted_index.convex | 165 |
| abstract_inverted_index.extend | 111 |
| abstract_inverted_index.graph. | 109 |
| abstract_inverted_index.obtain | 120 |
| abstract_inverted_index.policy | 75 |
| abstract_inverted_index.regret | 122, 145 |
| abstract_inverted_index.round, | 27 |
| abstract_inverted_index.rounds | 60 |
| abstract_inverted_index.simple | 121, 144 |
| abstract_inverted_index.action) | 83 |
| abstract_inverted_index.action, | 33 |
| abstract_inverted_index.address | 168 |
| abstract_inverted_index.bandits | 7 |
| abstract_inverted_index.conduct | 175 |
| abstract_inverted_index.context | 10, 39, 117 |
| abstract_inverted_index.feature | 157 |
| abstract_inverted_index.initial | 16, 32, 79 |
| abstract_inverted_index.learner | 29, 48, 70 |
| abstract_inverted_index.maximum | 85 |
| abstract_inverted_index.project | 188 |
| abstract_inverted_index.release | 183 |
| abstract_inverted_index.reward. | 57, 87 |
| abstract_inverted_index.selects | 30, 50 |
| abstract_inverted_index.setting | 118 |
| abstract_inverted_index.through | 128 |
| abstract_inverted_index.variant | 3 |
| abstract_inverted_index.achieved | 127 |
| abstract_inverted_index.expected | 86 |
| abstract_inverted_index.learner. | 21 |
| abstract_inverted_index.problem. | 172 |
| abstract_inverted_index.receives | 55 |
| abstract_inverted_index.results, | 181 |
| abstract_inverted_index.revealed | 41 |
| abstract_inverted_index.specific | 94 |
| abstract_inverted_index.validate | 178 |
| abstract_inverted_index.Following | 45 |
| abstract_inverted_index.beginning | 24 |
| abstract_inverted_index.depending | 34 |
| abstract_inverted_index.objective | 67 |
| abstract_inverted_index.selecting | 77 |
| abstract_inverted_index.situation | 95 |
| abstract_inverted_index.contextual | 6 |
| abstract_inverted_index.instances. | 154 |
| abstract_inverted_index.parameter, | 132 |
| abstract_inverted_index.stochastic | 38 |
| abstract_inverted_index.corresponds | 99 |
| abstract_inverted_index.essentially | 147 |
| abstract_inverted_index.experiments | 176 |
| abstract_inverted_index.exploration | 171 |
| abstract_inverted_index.guarantees. | 124 |
| abstract_inverted_index.intervening | 101 |
| abstract_inverted_index.repository: | 190 |
| abstract_inverted_index.theoretical | 180 |
| abstract_inverted_index.Furthermore, | 139 |
| abstract_inverted_index.environment, | 65 |
| abstract_inverted_index.environment. | 44 |
| abstract_inverted_index.interactions | 62 |
| abstract_inverted_index.intervention | 17 |
| abstract_inverted_index.minimization | 123 |
| abstract_inverted_index.optimization | 166 |
| abstract_inverted_index.characterizes | 135 |
| abstract_inverted_index.deterministic | 116 |
| abstract_inverted_index.instance-dependent | 130 |
| abstract_inverted_index.https://github.com/adaptiveContextualCausalBandits/aCCB. | 191 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 4 |
| citation_normalized_percentile |