Cost-Benefit Arbitration Between Multiple Reinforcement-Learning Systems Article Swipe
Wouter Kool
,
Samuel J. Gershman
,
Fiery Cushman
·
YOU?
·
· 2017
· Open Access
·
· DOI: https://doi.org/10.1177/0956797617708288
YOU?
·
· 2017
· Open Access
·
· DOI: https://doi.org/10.1177/0956797617708288
Human behavior is sometimes determined by habit and other times by goal-directed planning. Modern reinforcement-learning theories formalize this distinction as a competition between a computationally cheap but inaccurate model-free system that gives rise to habits and a computationally expensive but accurate model-based system that implements planning. It is unclear, however, how people choose to allocate control between these systems. Here, we propose that arbitration occurs by comparing each system’s task-specific costs and benefits. To investigate this proposal, we conducted two experiments showing that people increase model-based control when it achieves greater accuracy than model-free control, and especially when the rewards of accurate performance are amplified. In contrast, they are insensitive to reward amplification when model-based and model-free control yield equivalent accuracy. This suggests that humans adaptively balance habitual and planned action through on-line cost-benefit analysis.
Related Topics
Concepts
Reinforcement learning
Task (project management)
Control (management)
Contrast (vision)
Arbitration
Psychology
Computer science
Reinforcement
Habit
Action (physics)
Artificial intelligence
Cognitive psychology
Machine learning
Social psychology
Economics
Physics
Quantum mechanics
Political science
Law
Management
Metadata
- Type
- article
- Language
- en
- Landing Page
- https://doi.org/10.1177/0956797617708288
- OA Status
- green
- Cited By
- 261
- References
- 36
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W2736983092
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W2736983092Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.1177/0956797617708288Digital Object Identifier
- Title
-
Cost-Benefit Arbitration Between Multiple Reinforcement-Learning SystemsWork title
- Type
-
articleOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2017Year of publication
- Publication date
-
2017-07-21Full publication date if available
- Authors
-
Wouter Kool, Samuel J. Gershman, Fiery CushmanList of authors in order
- Landing page
-
https://doi.org/10.1177/0956797617708288Publisher landing page
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://nrs.harvard.edu/urn-3:HUL.InstRepos:41467544Direct OA link when available
- Concepts
-
Reinforcement learning, Task (project management), Control (management), Contrast (vision), Arbitration, Psychology, Computer science, Reinforcement, Habit, Action (physics), Artificial intelligence, Cognitive psychology, Machine learning, Social psychology, Economics, Physics, Quantum mechanics, Political science, Law, ManagementTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
261Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 43, 2024: 22, 2023: 28, 2022: 34, 2021: 34Per-year citation counts (last 5 years)
- References (count)
-
36Number of works referenced by this work
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W2736983092 |
|---|---|
| doi | https://doi.org/10.1177/0956797617708288 |
| ids.doi | https://doi.org/10.1177/0956797617708288 |
| ids.mag | 2736983092 |
| ids.pmid | https://pubmed.ncbi.nlm.nih.gov/28731839 |
| ids.openalex | https://openalex.org/W2736983092 |
| fwci | 15.3515539 |
| mesh[0].qualifier_ui | |
| mesh[0].descriptor_ui | D000328 |
| mesh[0].is_major_topic | False |
| mesh[0].qualifier_name | |
| mesh[0].descriptor_name | Adult |
| mesh[1].qualifier_ui | Q000502 |
| mesh[1].descriptor_ui | D003657 |
| mesh[1].is_major_topic | False |
| mesh[1].qualifier_name | physiology |
| mesh[1].descriptor_name | Decision Making |
| mesh[2].qualifier_ui | Q000502 |
| mesh[2].descriptor_ui | D056344 |
| mesh[2].is_major_topic | False |
| mesh[2].qualifier_name | physiology |
| mesh[2].descriptor_name | Executive Function |
| mesh[3].qualifier_ui | |
| mesh[3].descriptor_ui | D005260 |
| mesh[3].is_major_topic | False |
| mesh[3].qualifier_name | |
| mesh[3].descriptor_name | Female |
| mesh[4].qualifier_ui | |
| mesh[4].descriptor_ui | D006801 |
| mesh[4].is_major_topic | False |
| mesh[4].qualifier_name | |
| mesh[4].descriptor_name | Humans |
| mesh[5].qualifier_ui | |
| mesh[5].descriptor_ui | D008297 |
| mesh[5].is_major_topic | False |
| mesh[5].qualifier_name | |
| mesh[5].descriptor_name | Male |
| mesh[6].qualifier_ui | |
| mesh[6].descriptor_ui | D008875 |
| mesh[6].is_major_topic | False |
| mesh[6].qualifier_name | |
| mesh[6].descriptor_name | Middle Aged |
| mesh[7].qualifier_ui | Q000502 |
| mesh[7].descriptor_ui | D011597 |
| mesh[7].is_major_topic | False |
| mesh[7].qualifier_name | physiology |
| mesh[7].descriptor_name | Psychomotor Performance |
| mesh[8].qualifier_ui | |
| mesh[8].descriptor_ui | D012054 |
| mesh[8].is_major_topic | True |
| mesh[8].qualifier_name | |
| mesh[8].descriptor_name | Reinforcement, Psychology |
| mesh[9].qualifier_ui | |
| mesh[9].descriptor_ui | D055815 |
| mesh[9].is_major_topic | False |
| mesh[9].qualifier_name | |
| mesh[9].descriptor_name | Young Adult |
| mesh[10].qualifier_ui | |
| mesh[10].descriptor_ui | D000328 |
| mesh[10].is_major_topic | False |
| mesh[10].qualifier_name | |
| mesh[10].descriptor_name | Adult |
| mesh[11].qualifier_ui | Q000502 |
| mesh[11].descriptor_ui | D003657 |
| mesh[11].is_major_topic | False |
| mesh[11].qualifier_name | physiology |
| mesh[11].descriptor_name | Decision Making |
| mesh[12].qualifier_ui | Q000502 |
| mesh[12].descriptor_ui | D056344 |
| mesh[12].is_major_topic | False |
| mesh[12].qualifier_name | physiology |
| mesh[12].descriptor_name | Executive Function |
| mesh[13].qualifier_ui | |
| mesh[13].descriptor_ui | D005260 |
| mesh[13].is_major_topic | False |
| mesh[13].qualifier_name | |
| mesh[13].descriptor_name | Female |
| mesh[14].qualifier_ui | |
| mesh[14].descriptor_ui | D006801 |
| mesh[14].is_major_topic | False |
| mesh[14].qualifier_name | |
| mesh[14].descriptor_name | Humans |
| mesh[15].qualifier_ui | |
| mesh[15].descriptor_ui | D008297 |
| mesh[15].is_major_topic | False |
| mesh[15].qualifier_name | |
| mesh[15].descriptor_name | Male |
| mesh[16].qualifier_ui | |
| mesh[16].descriptor_ui | D008875 |
| mesh[16].is_major_topic | False |
| mesh[16].qualifier_name | |
| mesh[16].descriptor_name | Middle Aged |
| mesh[17].qualifier_ui | Q000502 |
| mesh[17].descriptor_ui | D011597 |
| mesh[17].is_major_topic | False |
| mesh[17].qualifier_name | physiology |
| mesh[17].descriptor_name | Psychomotor Performance |
| mesh[18].qualifier_ui | |
| mesh[18].descriptor_ui | D012054 |
| mesh[18].is_major_topic | True |
| mesh[18].qualifier_name | |
| mesh[18].descriptor_name | Reinforcement, Psychology |
| mesh[19].qualifier_ui | |
| mesh[19].descriptor_ui | D055815 |
| mesh[19].is_major_topic | False |
| mesh[19].qualifier_name | |
| mesh[19].descriptor_name | Young Adult |
| type | article |
| title | Cost-Benefit Arbitration Between Multiple Reinforcement-Learning Systems |
| biblio.issue | 9 |
| biblio.volume | 28 |
| biblio.last_page | 1333 |
| biblio.first_page | 1321 |
| topics[0].id | https://openalex.org/T10315 |
| topics[0].field.id | https://openalex.org/fields/18 |
| topics[0].field.display_name | Decision Sciences |
| topics[0].score | 0.9959999918937683 |
| topics[0].domain.id | https://openalex.org/domains/2 |
| topics[0].domain.display_name | Social Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1800 |
| topics[0].subfield.display_name | General Decision Sciences |
| topics[0].display_name | Decision-Making and Behavioral Economics |
| topics[1].id | https://openalex.org/T11542 |
| topics[1].field.id | https://openalex.org/fields/32 |
| topics[1].field.display_name | Psychology |
| topics[1].score | 0.9902999997138977 |
| topics[1].domain.id | https://openalex.org/domains/2 |
| topics[1].domain.display_name | Social Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/3202 |
| topics[1].subfield.display_name | Applied Psychology |
| topics[1].display_name | Behavioral Health and Interventions |
| topics[2].id | https://openalex.org/T10042 |
| topics[2].field.id | https://openalex.org/fields/28 |
| topics[2].field.display_name | Neuroscience |
| topics[2].score | 0.9884999990463257 |
| topics[2].domain.id | https://openalex.org/domains/1 |
| topics[2].domain.display_name | Life Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/2805 |
| topics[2].subfield.display_name | Cognitive Neuroscience |
| topics[2].display_name | Neural and Behavioral Psychology Studies |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C97541855 |
| concepts[0].level | 2 |
| concepts[0].score | 0.7994138598442078 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q830687 |
| concepts[0].display_name | Reinforcement learning |
| concepts[1].id | https://openalex.org/C2780451532 |
| concepts[1].level | 2 |
| concepts[1].score | 0.7455471158027649 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q759676 |
| concepts[1].display_name | Task (project management) |
| concepts[2].id | https://openalex.org/C2775924081 |
| concepts[2].level | 2 |
| concepts[2].score | 0.6201901435852051 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q55608371 |
| concepts[2].display_name | Control (management) |
| concepts[3].id | https://openalex.org/C2776502983 |
| concepts[3].level | 2 |
| concepts[3].score | 0.5913732051849365 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q690182 |
| concepts[3].display_name | Contrast (vision) |
| concepts[4].id | https://openalex.org/C160151201 |
| concepts[4].level | 2 |
| concepts[4].score | 0.5809386372566223 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q207946 |
| concepts[4].display_name | Arbitration |
| concepts[5].id | https://openalex.org/C15744967 |
| concepts[5].level | 0 |
| concepts[5].score | 0.5279912948608398 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q9418 |
| concepts[5].display_name | Psychology |
| concepts[6].id | https://openalex.org/C41008148 |
| concepts[6].level | 0 |
| concepts[6].score | 0.5089254379272461 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[6].display_name | Computer science |
| concepts[7].id | https://openalex.org/C67203356 |
| concepts[7].level | 2 |
| concepts[7].score | 0.5032498240470886 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q1321905 |
| concepts[7].display_name | Reinforcement |
| concepts[8].id | https://openalex.org/C44670240 |
| concepts[8].level | 2 |
| concepts[8].score | 0.4940081238746643 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q1299714 |
| concepts[8].display_name | Habit |
| concepts[9].id | https://openalex.org/C2780791683 |
| concepts[9].level | 2 |
| concepts[9].score | 0.49181845784187317 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q846785 |
| concepts[9].display_name | Action (physics) |
| concepts[10].id | https://openalex.org/C154945302 |
| concepts[10].level | 1 |
| concepts[10].score | 0.4602808356285095 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[10].display_name | Artificial intelligence |
| concepts[11].id | https://openalex.org/C180747234 |
| concepts[11].level | 1 |
| concepts[11].score | 0.41140228509902954 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q23373 |
| concepts[11].display_name | Cognitive psychology |
| concepts[12].id | https://openalex.org/C119857082 |
| concepts[12].level | 1 |
| concepts[12].score | 0.35965535044670105 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q2539 |
| concepts[12].display_name | Machine learning |
| concepts[13].id | https://openalex.org/C77805123 |
| concepts[13].level | 1 |
| concepts[13].score | 0.21272578835487366 |
| concepts[13].wikidata | https://www.wikidata.org/wiki/Q161272 |
| concepts[13].display_name | Social psychology |
| concepts[14].id | https://openalex.org/C162324750 |
| concepts[14].level | 0 |
| concepts[14].score | 0.09767496585845947 |
| concepts[14].wikidata | https://www.wikidata.org/wiki/Q8134 |
| concepts[14].display_name | Economics |
| concepts[15].id | https://openalex.org/C121332964 |
| concepts[15].level | 0 |
| concepts[15].score | 0.0 |
| concepts[15].wikidata | https://www.wikidata.org/wiki/Q413 |
| concepts[15].display_name | Physics |
| concepts[16].id | https://openalex.org/C62520636 |
| concepts[16].level | 1 |
| concepts[16].score | 0.0 |
| concepts[16].wikidata | https://www.wikidata.org/wiki/Q944 |
| concepts[16].display_name | Quantum mechanics |
| concepts[17].id | https://openalex.org/C17744445 |
| concepts[17].level | 0 |
| concepts[17].score | 0.0 |
| concepts[17].wikidata | https://www.wikidata.org/wiki/Q36442 |
| concepts[17].display_name | Political science |
| concepts[18].id | https://openalex.org/C199539241 |
| concepts[18].level | 1 |
| concepts[18].score | 0.0 |
| concepts[18].wikidata | https://www.wikidata.org/wiki/Q7748 |
| concepts[18].display_name | Law |
| concepts[19].id | https://openalex.org/C187736073 |
| concepts[19].level | 1 |
| concepts[19].score | 0.0 |
| concepts[19].wikidata | https://www.wikidata.org/wiki/Q2920921 |
| concepts[19].display_name | Management |
| keywords[0].id | https://openalex.org/keywords/reinforcement-learning |
| keywords[0].score | 0.7994138598442078 |
| keywords[0].display_name | Reinforcement learning |
| keywords[1].id | https://openalex.org/keywords/task |
| keywords[1].score | 0.7455471158027649 |
| keywords[1].display_name | Task (project management) |
| keywords[2].id | https://openalex.org/keywords/control |
| keywords[2].score | 0.6201901435852051 |
| keywords[2].display_name | Control (management) |
| keywords[3].id | https://openalex.org/keywords/contrast |
| keywords[3].score | 0.5913732051849365 |
| keywords[3].display_name | Contrast (vision) |
| keywords[4].id | https://openalex.org/keywords/arbitration |
| keywords[4].score | 0.5809386372566223 |
| keywords[4].display_name | Arbitration |
| keywords[5].id | https://openalex.org/keywords/psychology |
| keywords[5].score | 0.5279912948608398 |
| keywords[5].display_name | Psychology |
| keywords[6].id | https://openalex.org/keywords/computer-science |
| keywords[6].score | 0.5089254379272461 |
| keywords[6].display_name | Computer science |
| keywords[7].id | https://openalex.org/keywords/reinforcement |
| keywords[7].score | 0.5032498240470886 |
| keywords[7].display_name | Reinforcement |
| keywords[8].id | https://openalex.org/keywords/habit |
| keywords[8].score | 0.4940081238746643 |
| keywords[8].display_name | Habit |
| keywords[9].id | https://openalex.org/keywords/action |
| keywords[9].score | 0.49181845784187317 |
| keywords[9].display_name | Action (physics) |
| keywords[10].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[10].score | 0.4602808356285095 |
| keywords[10].display_name | Artificial intelligence |
| keywords[11].id | https://openalex.org/keywords/cognitive-psychology |
| keywords[11].score | 0.41140228509902954 |
| keywords[11].display_name | Cognitive psychology |
| keywords[12].id | https://openalex.org/keywords/machine-learning |
| keywords[12].score | 0.35965535044670105 |
| keywords[12].display_name | Machine learning |
| keywords[13].id | https://openalex.org/keywords/social-psychology |
| keywords[13].score | 0.21272578835487366 |
| keywords[13].display_name | Social psychology |
| keywords[14].id | https://openalex.org/keywords/economics |
| keywords[14].score | 0.09767496585845947 |
| keywords[14].display_name | Economics |
| language | en |
| locations[0].id | doi:10.1177/0956797617708288 |
| locations[0].is_oa | False |
| locations[0].source.id | https://openalex.org/S58854535 |
| locations[0].source.issn | 0956-7976, 1467-9280 |
| locations[0].source.type | journal |
| locations[0].source.is_oa | False |
| locations[0].source.issn_l | 0956-7976 |
| locations[0].source.is_core | True |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | Psychological Science |
| locations[0].source.host_organization | https://openalex.org/P4310320017 |
| locations[0].source.host_organization_name | SAGE Publishing |
| locations[0].source.host_organization_lineage | https://openalex.org/P4310320017 |
| locations[0].source.host_organization_lineage_names | SAGE Publishing |
| locations[0].license | |
| locations[0].pdf_url | |
| locations[0].version | publishedVersion |
| locations[0].raw_type | journal-article |
| locations[0].license_id | |
| locations[0].is_accepted | True |
| locations[0].is_published | True |
| locations[0].raw_source_name | Psychological Science |
| locations[0].landing_page_url | https://doi.org/10.1177/0956797617708288 |
| locations[1].id | pmid:28731839 |
| locations[1].is_oa | False |
| locations[1].source.id | https://openalex.org/S4306525036 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | False |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | PubMed |
| locations[1].source.host_organization | https://openalex.org/I1299303238 |
| locations[1].source.host_organization_name | National Institutes of Health |
| locations[1].source.host_organization_lineage | https://openalex.org/I1299303238 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | publishedVersion |
| locations[1].raw_type | |
| locations[1].license_id | |
| locations[1].is_accepted | True |
| locations[1].is_published | True |
| locations[1].raw_source_name | Psychological science |
| locations[1].landing_page_url | https://pubmed.ncbi.nlm.nih.gov/28731839 |
| locations[2].id | pmh:oai:dash.harvard.edu:1/41467544 |
| locations[2].is_oa | True |
| locations[2].source.id | https://openalex.org/S4306401540 |
| locations[2].source.issn | |
| locations[2].source.type | repository |
| locations[2].source.is_oa | False |
| locations[2].source.issn_l | |
| locations[2].source.is_core | False |
| locations[2].source.is_in_doaj | False |
| locations[2].source.display_name | Digital Access to Scholarship at Harvard (DASH) (Harvard University) |
| locations[2].source.host_organization | https://openalex.org/I136199984 |
| locations[2].source.host_organization_name | Harvard University |
| locations[2].source.host_organization_lineage | https://openalex.org/I136199984 |
| locations[2].license | cc-by |
| locations[2].pdf_url | |
| locations[2].version | submittedVersion |
| locations[2].raw_type | Journal Article |
| locations[2].license_id | https://openalex.org/licenses/cc-by |
| locations[2].is_accepted | False |
| locations[2].is_published | False |
| locations[2].raw_source_name | Psychol Sci |
| locations[2].landing_page_url | http://nrs.harvard.edu/urn-3:HUL.InstRepos:41467544 |
| indexed_in | crossref, pubmed |
| authorships[0].author.id | https://openalex.org/A5091709254 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-4792-7044 |
| authorships[0].author.display_name | Wouter Kool |
| authorships[0].countries | US |
| authorships[0].affiliations[0].institution_ids | https://openalex.org/I136199984 |
| authorships[0].affiliations[0].raw_affiliation_string | Department of Psychology, Harvard University |
| authorships[0].institutions[0].id | https://openalex.org/I136199984 |
| authorships[0].institutions[0].ror | https://ror.org/03vek6s52 |
| authorships[0].institutions[0].type | education |
| authorships[0].institutions[0].lineage | https://openalex.org/I136199984 |
| authorships[0].institutions[0].country_code | US |
| authorships[0].institutions[0].display_name | Harvard University |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Wouter Kool |
| authorships[0].is_corresponding | True |
| authorships[0].raw_affiliation_strings | Department of Psychology, Harvard University |
| authorships[1].author.id | https://openalex.org/A5031715686 |
| authorships[1].author.orcid | https://orcid.org/0000-0002-6546-3298 |
| authorships[1].author.display_name | Samuel J. Gershman |
| authorships[1].countries | US |
| authorships[1].affiliations[0].institution_ids | https://openalex.org/I136199984, https://openalex.org/I4210150141 |
| authorships[1].affiliations[0].raw_affiliation_string | Center for Brain Science, Harvard University |
| authorships[1].affiliations[1].institution_ids | https://openalex.org/I136199984 |
| authorships[1].affiliations[1].raw_affiliation_string | Department of Psychology, Harvard University |
| authorships[1].institutions[0].id | https://openalex.org/I4210150141 |
| authorships[1].institutions[0].ror | https://ror.org/04zxw0v05 |
| authorships[1].institutions[0].type | nonprofit |
| authorships[1].institutions[0].lineage | https://openalex.org/I1288882113, https://openalex.org/I2801148111, https://openalex.org/I4210087915, https://openalex.org/I4210150141, https://openalex.org/I48633490 |
| authorships[1].institutions[0].country_code | US |
| authorships[1].institutions[0].display_name | Center for Pain and the Brain |
| authorships[1].institutions[1].id | https://openalex.org/I136199984 |
| authorships[1].institutions[1].ror | https://ror.org/03vek6s52 |
| authorships[1].institutions[1].type | education |
| authorships[1].institutions[1].lineage | https://openalex.org/I136199984 |
| authorships[1].institutions[1].country_code | US |
| authorships[1].institutions[1].display_name | Harvard University |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Samuel J. Gershman |
| authorships[1].is_corresponding | False |
| authorships[1].raw_affiliation_strings | Center for Brain Science, Harvard University, Department of Psychology, Harvard University |
| authorships[2].author.id | https://openalex.org/A5066361095 |
| authorships[2].author.orcid | https://orcid.org/0000-0002-6929-9982 |
| authorships[2].author.display_name | Fiery Cushman |
| authorships[2].countries | US |
| authorships[2].affiliations[0].institution_ids | https://openalex.org/I136199984 |
| authorships[2].affiliations[0].raw_affiliation_string | Department of Psychology, Harvard University |
| authorships[2].institutions[0].id | https://openalex.org/I136199984 |
| authorships[2].institutions[0].ror | https://ror.org/03vek6s52 |
| authorships[2].institutions[0].type | education |
| authorships[2].institutions[0].lineage | https://openalex.org/I136199984 |
| authorships[2].institutions[0].country_code | US |
| authorships[2].institutions[0].display_name | Harvard University |
| authorships[2].author_position | last |
| authorships[2].raw_author_name | Fiery A. Cushman |
| authorships[2].is_corresponding | False |
| authorships[2].raw_affiliation_strings | Department of Psychology, Harvard University |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | http://nrs.harvard.edu/urn-3:HUL.InstRepos:41467544 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Cost-Benefit Arbitration Between Multiple Reinforcement-Learning Systems |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T03:46:38.306776 |
| primary_topic.id | https://openalex.org/T10315 |
| primary_topic.field.id | https://openalex.org/fields/18 |
| primary_topic.field.display_name | Decision Sciences |
| primary_topic.score | 0.9959999918937683 |
| primary_topic.domain.id | https://openalex.org/domains/2 |
| primary_topic.domain.display_name | Social Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1800 |
| primary_topic.subfield.display_name | General Decision Sciences |
| primary_topic.display_name | Decision-Making and Behavioral Economics |
| related_works | https://openalex.org/W3095478977, https://openalex.org/W2388352914, https://openalex.org/W2914422975, https://openalex.org/W2153247772, https://openalex.org/W1992917560, https://openalex.org/W2351937989, https://openalex.org/W2112428735, https://openalex.org/W2378516794, https://openalex.org/W2347397841, https://openalex.org/W2381910504 |
| cited_by_count | 261 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 43 |
| counts_by_year[1].year | 2024 |
| counts_by_year[1].cited_by_count | 22 |
| counts_by_year[2].year | 2023 |
| counts_by_year[2].cited_by_count | 28 |
| counts_by_year[3].year | 2022 |
| counts_by_year[3].cited_by_count | 34 |
| counts_by_year[4].year | 2021 |
| counts_by_year[4].cited_by_count | 34 |
| counts_by_year[5].year | 2020 |
| counts_by_year[5].cited_by_count | 32 |
| counts_by_year[6].year | 2019 |
| counts_by_year[6].cited_by_count | 25 |
| counts_by_year[7].year | 2018 |
| counts_by_year[7].cited_by_count | 30 |
| counts_by_year[8].year | 2017 |
| counts_by_year[8].cited_by_count | 13 |
| locations_count | 3 |
| best_oa_location.id | pmh:oai:dash.harvard.edu:1/41467544 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306401540 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | False |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | Digital Access to Scholarship at Harvard (DASH) (Harvard University) |
| best_oa_location.source.host_organization | https://openalex.org/I136199984 |
| best_oa_location.source.host_organization_name | Harvard University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I136199984 |
| best_oa_location.license | cc-by |
| best_oa_location.pdf_url | |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | Journal Article |
| best_oa_location.license_id | https://openalex.org/licenses/cc-by |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | Psychol Sci |
| best_oa_location.landing_page_url | http://nrs.harvard.edu/urn-3:HUL.InstRepos:41467544 |
| primary_location.id | doi:10.1177/0956797617708288 |
| primary_location.is_oa | False |
| primary_location.source.id | https://openalex.org/S58854535 |
| primary_location.source.issn | 0956-7976, 1467-9280 |
| primary_location.source.type | journal |
| primary_location.source.is_oa | False |
| primary_location.source.issn_l | 0956-7976 |
| primary_location.source.is_core | True |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | Psychological Science |
| primary_location.source.host_organization | https://openalex.org/P4310320017 |
| primary_location.source.host_organization_name | SAGE Publishing |
| primary_location.source.host_organization_lineage | https://openalex.org/P4310320017 |
| primary_location.source.host_organization_lineage_names | SAGE Publishing |
| primary_location.license | |
| primary_location.pdf_url | |
| primary_location.version | publishedVersion |
| primary_location.raw_type | journal-article |
| primary_location.license_id | |
| primary_location.is_accepted | True |
| primary_location.is_published | True |
| primary_location.raw_source_name | Psychological Science |
| primary_location.landing_page_url | https://doi.org/10.1177/0956797617708288 |
| publication_date | 2017-07-21 |
| publication_year | 2017 |
| referenced_works | https://openalex.org/W2951968427, https://openalex.org/W2110113299, https://openalex.org/W2166756161, https://openalex.org/W2025162942, https://openalex.org/W2102803424, https://openalex.org/W2167362547, https://openalex.org/W2336319629, https://openalex.org/W2115998792, https://openalex.org/W1988699227, https://openalex.org/W2028384357, https://openalex.org/W2281891645, https://openalex.org/W1892018222, https://openalex.org/W2149892666, https://openalex.org/W2149565728, https://openalex.org/W2141467654, https://openalex.org/W2046411892, https://openalex.org/W2080834410, https://openalex.org/W2031474948, https://openalex.org/W2510619540, https://openalex.org/W2092813663, https://openalex.org/W2583311906, https://openalex.org/W2164682760, https://openalex.org/W2129299900, https://openalex.org/W2111535985, https://openalex.org/W2090933055, https://openalex.org/W2128349062, https://openalex.org/W2132415562, https://openalex.org/W2143594200, https://openalex.org/W2121001250, https://openalex.org/W2103492232, https://openalex.org/W2001898186, https://openalex.org/W2146112275, https://openalex.org/W4214717370, https://openalex.org/W3127231893, https://openalex.org/W2121863487, https://openalex.org/W2288341381 |
| referenced_works_count | 36 |
| abstract_inverted_index.a | 20, 23, 36 |
| abstract_inverted_index.In | 105 |
| abstract_inverted_index.It | 46 |
| abstract_inverted_index.To | 73 |
| abstract_inverted_index.as | 19 |
| abstract_inverted_index.by | 5, 10, 65 |
| abstract_inverted_index.is | 2, 47 |
| abstract_inverted_index.it | 88 |
| abstract_inverted_index.of | 100 |
| abstract_inverted_index.to | 33, 53, 110 |
| abstract_inverted_index.we | 60, 77 |
| abstract_inverted_index.and | 7, 35, 71, 95, 115, 128 |
| abstract_inverted_index.are | 103, 108 |
| abstract_inverted_index.but | 26, 39 |
| abstract_inverted_index.how | 50 |
| abstract_inverted_index.the | 98 |
| abstract_inverted_index.two | 79 |
| abstract_inverted_index.This | 121 |
| abstract_inverted_index.each | 67 |
| abstract_inverted_index.rise | 32 |
| abstract_inverted_index.than | 92 |
| abstract_inverted_index.that | 30, 43, 62, 82, 123 |
| abstract_inverted_index.they | 107 |
| abstract_inverted_index.this | 17, 75 |
| abstract_inverted_index.when | 87, 97, 113 |
| abstract_inverted_index.Here, | 59 |
| abstract_inverted_index.Human | 0 |
| abstract_inverted_index.cheap | 25 |
| abstract_inverted_index.costs | 70 |
| abstract_inverted_index.gives | 31 |
| abstract_inverted_index.habit | 6 |
| abstract_inverted_index.other | 8 |
| abstract_inverted_index.these | 57 |
| abstract_inverted_index.times | 9 |
| abstract_inverted_index.yield | 118 |
| abstract_inverted_index.Modern | 13 |
| abstract_inverted_index.action | 130 |
| abstract_inverted_index.choose | 52 |
| abstract_inverted_index.habits | 34 |
| abstract_inverted_index.humans | 124 |
| abstract_inverted_index.occurs | 64 |
| abstract_inverted_index.people | 51, 83 |
| abstract_inverted_index.reward | 111 |
| abstract_inverted_index.system | 29, 42 |
| abstract_inverted_index.balance | 126 |
| abstract_inverted_index.between | 22, 56 |
| abstract_inverted_index.control | 55, 86, 117 |
| abstract_inverted_index.greater | 90 |
| abstract_inverted_index.on-line | 132 |
| abstract_inverted_index.planned | 129 |
| abstract_inverted_index.propose | 61 |
| abstract_inverted_index.rewards | 99 |
| abstract_inverted_index.showing | 81 |
| abstract_inverted_index.through | 131 |
| abstract_inverted_index.accuracy | 91 |
| abstract_inverted_index.accurate | 40, 101 |
| abstract_inverted_index.achieves | 89 |
| abstract_inverted_index.allocate | 54 |
| abstract_inverted_index.behavior | 1 |
| abstract_inverted_index.control, | 94 |
| abstract_inverted_index.habitual | 127 |
| abstract_inverted_index.however, | 49 |
| abstract_inverted_index.increase | 84 |
| abstract_inverted_index.suggests | 122 |
| abstract_inverted_index.systems. | 58 |
| abstract_inverted_index.theories | 15 |
| abstract_inverted_index.unclear, | 48 |
| abstract_inverted_index.accuracy. | 120 |
| abstract_inverted_index.analysis. | 134 |
| abstract_inverted_index.benefits. | 72 |
| abstract_inverted_index.comparing | 66 |
| abstract_inverted_index.conducted | 78 |
| abstract_inverted_index.contrast, | 106 |
| abstract_inverted_index.expensive | 38 |
| abstract_inverted_index.formalize | 16 |
| abstract_inverted_index.planning. | 12, 45 |
| abstract_inverted_index.proposal, | 76 |
| abstract_inverted_index.sometimes | 3 |
| abstract_inverted_index.adaptively | 125 |
| abstract_inverted_index.amplified. | 104 |
| abstract_inverted_index.determined | 4 |
| abstract_inverted_index.equivalent | 119 |
| abstract_inverted_index.especially | 96 |
| abstract_inverted_index.implements | 44 |
| abstract_inverted_index.inaccurate | 27 |
| abstract_inverted_index.model-free | 28, 93, 116 |
| abstract_inverted_index.system’s | 68 |
| abstract_inverted_index.arbitration | 63 |
| abstract_inverted_index.competition | 21 |
| abstract_inverted_index.distinction | 18 |
| abstract_inverted_index.experiments | 80 |
| abstract_inverted_index.insensitive | 109 |
| abstract_inverted_index.investigate | 74 |
| abstract_inverted_index.model-based | 41, 85, 114 |
| abstract_inverted_index.performance | 102 |
| abstract_inverted_index.cost-benefit | 133 |
| abstract_inverted_index.amplification | 112 |
| abstract_inverted_index.goal-directed | 11 |
| abstract_inverted_index.task-specific | 69 |
| abstract_inverted_index.computationally | 24, 37 |
| abstract_inverted_index.reinforcement-learning | 14 |
| cited_by_percentile_year.max | 100 |
| cited_by_percentile_year.min | 99 |
| corresponding_author_ids | https://openalex.org/A5091709254 |
| countries_distinct_count | 1 |
| institutions_distinct_count | 3 |
| corresponding_institution_ids | https://openalex.org/I136199984 |
| sustainable_development_goals[0].id | https://metadata.un.org/sdg/16 |
| sustainable_development_goals[0].score | 0.5400000214576721 |
| sustainable_development_goals[0].display_name | Peace, Justice and strong institutions |
| citation_normalized_percentile.value | 0.99465598 |
| citation_normalized_percentile.is_in_top_1_percent | True |
| citation_normalized_percentile.is_in_top_10_percent | True |