Deciding what to replicate Article Swipe
Peder Mortvedt Isager
,
Robbie C. M. van Aert
,
Štěpán Bahník
,
Mark J. Brandt
,
K. Andrew DeSoto
,
Roger Giner‐Sorolla
,
Joachim I. Krueger
,
Marco Perugini
,
Ivan Ropovik
,
Anna van 't Veer
·
YOU?
·
· 2020
· Open Access
·
· DOI: https://doi.org/10.17605/osf.io/asype
YOU?
·
· 2020
· Open Access
·
· DOI: https://doi.org/10.17605/osf.io/asype
Robust scientific knowledge is contingent upon replication of original findings. However, researchers who conduct replication studies face a difficult problem; there are many more studies in need of replication than there are funds available for replicating. To select studies for replication efficiently, we need to understand which studies are the most in need of replication. In other words, we need to understand which replication efforts have the highest expected utility. In this article we propose a general rule for study selection in replication research based on the replication value of the claims considered for replication. The replication value of a claim is defined as the maximum expected utility we could gain by replicating the claim, and is a function of (1) the value of being certain about the claim, and (2) uncertainty about the claim based on current evidence. We formalize this definition in terms of a causal decision model, utilizing concepts from decision theory and causal graph modeling. We discuss the validity of using replication value as a measure of expected utility gain, and we suggest approaches for deriving quantitative estimates of replication value.
Related Topics
Concepts
Metadata
- Type
- article
- Language
- en
- Landing Page
- https://doi.org/10.17605/OSF.IO/ASYPE
- OA Status
- green
- Cited By
- 3
- Related Works
- 20
- OpenAlex ID
- https://openalex.org/W3126499695
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W3126499695Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.17605/osf.io/asypeDigital Object Identifier
- Title
-
Deciding what to replicateWork title
- Type
-
articleOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2020Year of publication
- Publication date
-
2020-01-01Full publication date if available
- Authors
-
Peder Mortvedt Isager, Robbie C. M. van Aert, Štěpán Bahník, Mark J. Brandt, K. Andrew DeSoto, Roger Giner‐Sorolla, Joachim I. Krueger, Marco Perugini, Ivan Ropovik, Anna van 't VeerList of authors in order
- Landing page
-
https://doi.org/10.17605/OSF.IO/ASYPEPublisher landing page
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://doi.org/10.17605/OSF.IO/ASYPEDirect OA link when available
- Concepts
-
Replicate, Computer science, Mathematics, StatisticsTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
3Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 1, 2023: 1, 2021: 1Per-year citation counts (last 5 years)
- Related works (count)
-
20Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W3126499695 |
|---|---|
| doi | https://doi.org/10.17605/osf.io/asype |
| ids.doi | https://doi.org/10.17605/osf.io/asype |
| ids.mag | 3126499695 |
| ids.openalex | https://openalex.org/W3126499695 |
| fwci | 0.36758242 |
| type | article |
| title | Deciding what to replicate |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T11642 |
| topics[0].field.id | https://openalex.org/fields/13 |
| topics[0].field.display_name | Biochemistry, Genetics and Molecular Biology |
| topics[0].score | 0.20100000500679016 |
| topics[0].domain.id | https://openalex.org/domains/1 |
| topics[0].domain.display_name | Life Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1311 |
| topics[0].subfield.display_name | Genetics |
| topics[0].display_name | Genomics and Rare Diseases |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C2781162219 |
| concepts[0].level | 2 |
| concepts[0].score | 0.8181475400924683 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q26250693 |
| concepts[0].display_name | Replicate |
| concepts[1].id | https://openalex.org/C41008148 |
| concepts[1].level | 0 |
| concepts[1].score | 0.3797008991241455 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[1].display_name | Computer science |
| concepts[2].id | https://openalex.org/C33923547 |
| concepts[2].level | 0 |
| concepts[2].score | 0.1243581771850586 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q395 |
| concepts[2].display_name | Mathematics |
| concepts[3].id | https://openalex.org/C105795698 |
| concepts[3].level | 1 |
| concepts[3].score | 0.0 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q12483 |
| concepts[3].display_name | Statistics |
| keywords[0].id | https://openalex.org/keywords/replicate |
| keywords[0].score | 0.8181475400924683 |
| keywords[0].display_name | Replicate |
| keywords[1].id | https://openalex.org/keywords/computer-science |
| keywords[1].score | 0.3797008991241455 |
| keywords[1].display_name | Computer science |
| keywords[2].id | https://openalex.org/keywords/mathematics |
| keywords[2].score | 0.1243581771850586 |
| keywords[2].display_name | Mathematics |
| language | en |
| locations[0].id | pmh:oai:share.osf.io:08953612-8f17-4ab0-b2af-10874befa9d4 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400047 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | False |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | Arabixiv (OSF Preprints) |
| locations[0].source.host_organization | https://openalex.org/I2799848540 |
| locations[0].source.host_organization_name | Center for Open Science |
| locations[0].source.host_organization_lineage | https://openalex.org/I2799848540 |
| locations[0].license | cc-by |
| locations[0].pdf_url | |
| locations[0].version | submittedVersion |
| locations[0].raw_type | Project |
| locations[0].license_id | https://openalex.org/licenses/cc-by |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | https://doi.org/10.17605/OSF.IO/ASYPE |
| locations[1].id | pmh:oai:share.osf.io:E0169-161-39D |
| locations[1].is_oa | False |
| locations[1].source.id | https://openalex.org/S4306400047 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | False |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | Arabixiv (OSF Preprints) |
| locations[1].source.host_organization | https://openalex.org/I2799848540 |
| locations[1].source.host_organization_name | Center for Open Science |
| locations[1].source.host_organization_lineage | https://openalex.org/I2799848540 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | submittedVersion |
| locations[1].raw_type | project |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | False |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | http://doi.org/10.17605/OSF.IO/ASYPE |
| locations[2].id | doi:10.17605/osf.io/asype |
| locations[2].is_oa | True |
| locations[2].source.id | https://openalex.org/S7407050956 |
| locations[2].source.type | repository |
| locations[2].source.is_oa | False |
| locations[2].source.issn_l | |
| locations[2].source.is_core | False |
| locations[2].source.is_in_doaj | False |
| locations[2].source.display_name | Open Science Framework |
| locations[2].source.host_organization | |
| locations[2].source.host_organization_name | |
| locations[2].license | cc-by |
| locations[2].pdf_url | |
| locations[2].version | |
| locations[2].raw_type | article-journal |
| locations[2].license_id | https://openalex.org/licenses/cc-by |
| locations[2].is_accepted | False |
| locations[2].is_published | |
| locations[2].raw_source_name | |
| locations[2].landing_page_url | https://doi.org/10.17605/osf.io/asype |
| locations[3].id | mag:3126499695 |
| locations[3].is_oa | False |
| locations[3].source | |
| locations[3].license | |
| locations[3].pdf_url | |
| locations[3].version | |
| locations[3].raw_type | |
| locations[3].license_id | |
| locations[3].is_accepted | False |
| locations[3].is_published | |
| locations[3].raw_source_name | |
| locations[3].landing_page_url | https://osf.io/asype/#! |
| indexed_in | datacite |
| authorships[0].author.id | https://openalex.org/A5037893321 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-6922-3590 |
| authorships[0].author.display_name | Peder Mortvedt Isager |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Peder M. Isager |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5013441677 |
| authorships[1].author.orcid | https://orcid.org/0000-0001-6187-0665 |
| authorships[1].author.display_name | Robbie C. M. van Aert |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Robbie C. M. van Aert |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5040943392 |
| authorships[2].author.orcid | https://orcid.org/0000-0002-0579-6808 |
| authorships[2].author.display_name | Štěpán Bahník |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Štěpán Bahník |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5025573426 |
| authorships[3].author.orcid | https://orcid.org/0000-0002-7185-7031 |
| authorships[3].author.display_name | Mark J. Brandt |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Mark Brandt |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5009137113 |
| authorships[4].author.orcid | https://orcid.org/0000-0001-9061-0301 |
| authorships[4].author.display_name | K. Andrew DeSoto |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | K. Andrew DeSoto |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5003218340 |
| authorships[5].author.orcid | https://orcid.org/0000-0002-6690-8842 |
| authorships[5].author.display_name | Roger Giner‐Sorolla |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Roger Giner-Sorolla |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5062678392 |
| authorships[6].author.orcid | https://orcid.org/0000-0001-9607-1695 |
| authorships[6].author.display_name | Joachim I. Krueger |
| authorships[6].author_position | middle |
| authorships[6].raw_author_name | Joachim Krueger |
| authorships[6].is_corresponding | False |
| authorships[7].author.id | https://openalex.org/A5080384873 |
| authorships[7].author.orcid | https://orcid.org/0000-0002-4864-6623 |
| authorships[7].author.display_name | Marco Perugini |
| authorships[7].author_position | middle |
| authorships[7].raw_author_name | Marco Perugini |
| authorships[7].is_corresponding | False |
| authorships[8].author.id | https://openalex.org/A5017083279 |
| authorships[8].author.orcid | https://orcid.org/0000-0001-5222-1233 |
| authorships[8].author.display_name | Ivan Ropovik |
| authorships[8].author_position | middle |
| authorships[8].raw_author_name | Ivan Ropovik |
| authorships[8].is_corresponding | False |
| authorships[9].author.id | https://openalex.org/A5021362649 |
| authorships[9].author.orcid | https://orcid.org/0000-0002-2733-1841 |
| authorships[9].author.display_name | Anna van 't Veer |
| authorships[9].author_position | last |
| authorships[9].raw_author_name | Anna van 't Veer |
| authorships[9].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://doi.org/10.17605/OSF.IO/ASYPE |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Deciding what to replicate |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-17T23:14:22.571179 |
| primary_topic.id | https://openalex.org/T11642 |
| primary_topic.field.id | https://openalex.org/fields/13 |
| primary_topic.field.display_name | Biochemistry, Genetics and Molecular Biology |
| primary_topic.score | 0.20100000500679016 |
| primary_topic.domain.id | https://openalex.org/domains/1 |
| primary_topic.domain.display_name | Life Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1311 |
| primary_topic.subfield.display_name | Genetics |
| primary_topic.display_name | Genomics and Rare Diseases |
| related_works | https://openalex.org/W3129198752, https://openalex.org/W3181462022, https://openalex.org/W3175328064, https://openalex.org/W3169303212, https://openalex.org/W1793242419, https://openalex.org/W3086204866, https://openalex.org/W3040288200, https://openalex.org/W1537450671, https://openalex.org/W2929153518, https://openalex.org/W2996096539, https://openalex.org/W1510873898, https://openalex.org/W2287075396, https://openalex.org/W2183337259, https://openalex.org/W3209919106, https://openalex.org/W2930562709, https://openalex.org/W1600577375, https://openalex.org/W2184215017, https://openalex.org/W2852657622, https://openalex.org/W2142545010, https://openalex.org/W2390554833 |
| cited_by_count | 3 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 1 |
| counts_by_year[1].year | 2023 |
| counts_by_year[1].cited_by_count | 1 |
| counts_by_year[2].year | 2021 |
| counts_by_year[2].cited_by_count | 1 |
| locations_count | 4 |
| best_oa_location.id | pmh:oai:share.osf.io:08953612-8f17-4ab0-b2af-10874befa9d4 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400047 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | False |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | Arabixiv (OSF Preprints) |
| best_oa_location.source.host_organization | https://openalex.org/I2799848540 |
| best_oa_location.source.host_organization_name | Center for Open Science |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I2799848540 |
| best_oa_location.license | cc-by |
| best_oa_location.pdf_url | |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | Project |
| best_oa_location.license_id | https://openalex.org/licenses/cc-by |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | https://doi.org/10.17605/OSF.IO/ASYPE |
| primary_location.id | pmh:oai:share.osf.io:08953612-8f17-4ab0-b2af-10874befa9d4 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400047 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | False |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | Arabixiv (OSF Preprints) |
| primary_location.source.host_organization | https://openalex.org/I2799848540 |
| primary_location.source.host_organization_name | Center for Open Science |
| primary_location.source.host_organization_lineage | https://openalex.org/I2799848540 |
| primary_location.license | cc-by |
| primary_location.pdf_url | |
| primary_location.version | submittedVersion |
| primary_location.raw_type | Project |
| primary_location.license_id | https://openalex.org/licenses/cc-by |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | https://doi.org/10.17605/OSF.IO/ASYPE |
| publication_date | 2020-01-01 |
| publication_year | 2020 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 17, 75, 99, 117, 146, 168 |
| abstract_inverted_index.In | 55, 70 |
| abstract_inverted_index.To | 36 |
| abstract_inverted_index.We | 139, 159 |
| abstract_inverted_index.as | 103, 167 |
| abstract_inverted_index.by | 111 |
| abstract_inverted_index.in | 25, 51, 81, 143 |
| abstract_inverted_index.is | 3, 101, 116 |
| abstract_inverted_index.of | 7, 27, 53, 89, 98, 119, 123, 145, 163, 170, 182 |
| abstract_inverted_index.on | 85, 136 |
| abstract_inverted_index.to | 44, 60 |
| abstract_inverted_index.we | 42, 58, 73, 108, 175 |
| abstract_inverted_index.(1) | 120 |
| abstract_inverted_index.(2) | 130 |
| abstract_inverted_index.The | 95 |
| abstract_inverted_index.and | 115, 129, 155, 174 |
| abstract_inverted_index.are | 21, 31, 48 |
| abstract_inverted_index.for | 34, 39, 78, 93, 178 |
| abstract_inverted_index.the | 49, 66, 86, 90, 104, 113, 121, 127, 133, 161 |
| abstract_inverted_index.who | 12 |
| abstract_inverted_index.face | 16 |
| abstract_inverted_index.from | 152 |
| abstract_inverted_index.gain | 110 |
| abstract_inverted_index.have | 65 |
| abstract_inverted_index.many | 22 |
| abstract_inverted_index.more | 23 |
| abstract_inverted_index.most | 50 |
| abstract_inverted_index.need | 26, 43, 52, 59 |
| abstract_inverted_index.rule | 77 |
| abstract_inverted_index.than | 29 |
| abstract_inverted_index.this | 71, 141 |
| abstract_inverted_index.upon | 5 |
| abstract_inverted_index.about | 126, 132 |
| abstract_inverted_index.based | 84, 135 |
| abstract_inverted_index.being | 124 |
| abstract_inverted_index.claim | 100, 134 |
| abstract_inverted_index.could | 109 |
| abstract_inverted_index.funds | 32 |
| abstract_inverted_index.gain, | 173 |
| abstract_inverted_index.graph | 157 |
| abstract_inverted_index.other | 56 |
| abstract_inverted_index.study | 79 |
| abstract_inverted_index.terms | 144 |
| abstract_inverted_index.there | 20, 30 |
| abstract_inverted_index.using | 164 |
| abstract_inverted_index.value | 88, 97, 122, 166 |
| abstract_inverted_index.which | 46, 62 |
| abstract_inverted_index.Robust | 0 |
| abstract_inverted_index.causal | 147, 156 |
| abstract_inverted_index.claim, | 114, 128 |
| abstract_inverted_index.claims | 91 |
| abstract_inverted_index.model, | 149 |
| abstract_inverted_index.select | 37 |
| abstract_inverted_index.theory | 154 |
| abstract_inverted_index.value. | 184 |
| abstract_inverted_index.words, | 57 |
| abstract_inverted_index.article | 72 |
| abstract_inverted_index.certain | 125 |
| abstract_inverted_index.conduct | 13 |
| abstract_inverted_index.current | 137 |
| abstract_inverted_index.defined | 102 |
| abstract_inverted_index.discuss | 160 |
| abstract_inverted_index.efforts | 64 |
| abstract_inverted_index.general | 76 |
| abstract_inverted_index.highest | 67 |
| abstract_inverted_index.maximum | 105 |
| abstract_inverted_index.measure | 169 |
| abstract_inverted_index.propose | 74 |
| abstract_inverted_index.studies | 15, 24, 38, 47 |
| abstract_inverted_index.suggest | 176 |
| abstract_inverted_index.utility | 107, 172 |
| abstract_inverted_index.However, | 10 |
| abstract_inverted_index.concepts | 151 |
| abstract_inverted_index.decision | 148, 153 |
| abstract_inverted_index.deriving | 179 |
| abstract_inverted_index.expected | 68, 106, 171 |
| abstract_inverted_index.function | 118 |
| abstract_inverted_index.original | 8 |
| abstract_inverted_index.problem; | 19 |
| abstract_inverted_index.research | 83 |
| abstract_inverted_index.utility. | 69 |
| abstract_inverted_index.validity | 162 |
| abstract_inverted_index.available | 33 |
| abstract_inverted_index.difficult | 18 |
| abstract_inverted_index.estimates | 181 |
| abstract_inverted_index.evidence. | 138 |
| abstract_inverted_index.findings. | 9 |
| abstract_inverted_index.formalize | 140 |
| abstract_inverted_index.knowledge | 2 |
| abstract_inverted_index.modeling. | 158 |
| abstract_inverted_index.selection | 80 |
| abstract_inverted_index.utilizing | 150 |
| abstract_inverted_index.approaches | 177 |
| abstract_inverted_index.considered | 92 |
| abstract_inverted_index.contingent | 4 |
| abstract_inverted_index.definition | 142 |
| abstract_inverted_index.scientific | 1 |
| abstract_inverted_index.understand | 45, 61 |
| abstract_inverted_index.replicating | 112 |
| abstract_inverted_index.replication | 6, 14, 28, 40, 63, 82, 87, 96, 165, 183 |
| abstract_inverted_index.researchers | 11 |
| abstract_inverted_index.uncertainty | 131 |
| abstract_inverted_index.efficiently, | 41 |
| abstract_inverted_index.quantitative | 180 |
| abstract_inverted_index.replicating. | 35 |
| abstract_inverted_index.replication. | 54, 94 |
| cited_by_percentile_year.max | 95 |
| cited_by_percentile_year.min | 89 |
| countries_distinct_count | 0 |
| institutions_distinct_count | 10 |
| citation_normalized_percentile.value | 0.69375069 |
| citation_normalized_percentile.is_in_top_1_percent | False |
| citation_normalized_percentile.is_in_top_10_percent | False |