Improving open and rigorous science: ten key future research opportunities related to rigor, reproducibility, and transparency in scientific research Article Swipe

PDF

Danny Valdez , Colby J. Vorland , Andrew W. Brown , Evan Mayo‐Wilson , Justin Otten , Richard Ball , Sean Grant , Rachel Lévy , Dubravka Svetina , David B. Allison ·

YOU? · · 2020 · Open Access · · DOI: https://doi.org/10.12688/f1000research.26594.1

Background:As part of a coordinated effort to expand research activity around rigor, reproducibility, and transparency (RRT) across scientific disciplines, a team of investigators at the Indiana University School of Public Health-Bloomington hosted a workshop in October 2019 with international leaders to discuss key opportunities for RRT research.Objective:The workshop aimed to identify research priorities and opportunities related to RRT.Design:Over two-days, workshop attendees gave presentations and participated in three working groups: (1) Improving Education & Training in RRT, (2) Reducing Statistical Errors and Increasing Analytic Transparency, and (3) Looking Outward: Increasing Truthfulness and Accuracy of Research Communications. Following small-group discussions, the working groups presented their findings, and participants discussed the research opportunities identified. The investigators compiled a list of research priorities, which were circulated to all participants for feedback.Results:Participants identified the following priority research questions: (1) Can RRT-focused statistics and mathematical modeling courses improve statistics practice?; (2) Can specialized training in scientific writing improve transparency?; (3) Does modality (e.g. face to face, online) affect the efficacy RRT-related education?; (4) How can automated programs help identify errors more efficiently?; (5) What is the prevalence and impact of errors in scientific publications (e.g., analytic inconsistencies, statistical errors, and other objective errors)?; (6) Do error prevention workflows reduce errors?; (7) How do we encourage post-publication error correction?; (8) How does ‘spin’ in research communication affect stakeholder understanding and use of research evidence?; (9) Do tools to aid writing research reports increase comprehensiveness and clarity of research reports?; and (10) Is it possible to inculcate scientific values and norms related to truthful, rigorous, accurate, and comprehensive scientific reporting?Conclusion:Participants identified important and relatively unexplored questions related to improving RRT. This list may be useful to the scientific community and investigators seeking to advance meta-science (i.e. research on research).

Related Topics

Concepts

Open science Open peer review Transparency (behavior) Open data Plant biology Key (lock) Rigour Research integrity Engineering ethics Open research Data science Medicine Management science Computer science Political science Biology Engineering Mathematics World Wide Web Law Geometry Statistics Botany Computer security

Metadata

Type: preprint
Language: en
Landing Page: https://doi.org/10.12688/f1000research.26594.1
PDF: https://f1000research.com/articles/9-1235/v1/pdf
OA Status: gold
Cited By: 12
References: 96
Related Works: 10
OpenAlex ID: https://openalex.org/W3093196515

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W3093196515

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.12688/f1000research.26594.1

Digital Object Identifier
Title: Improving open and rigorous science: ten key future research opportunities related to rigor, reproducibility, and transparency in scientific research

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2020

Year of publication
Publication date: 2020-10-14

Full publication date if available
Authors: Danny Valdez, Colby J. Vorland, Andrew W. Brown, Evan Mayo‐Wilson, Justin Otten, Richard Ball, Sean Grant, Rachel Lévy, Dubravka Svetina, David B. Allison

List of authors in order
Landing page: https://doi.org/10.12688/f1000research.26594.1

Publisher landing page
PDF URL: https://f1000research.com/articles/9-1235/v1/pdf

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: gold

Open access status per OpenAlex
OA URL: https://f1000research.com/articles/9-1235/v1/pdf

Direct OA link when available
Concepts: Open science, Open peer review, Transparency (behavior), Open data, Plant biology, Key (lock), Rigour, Research integrity, Engineering ethics, Open research, Data science, Medicine, Management science, Computer science, Political science, Biology, Engineering, Mathematics, World Wide Web, Law, Geometry, Statistics, Botany, Computer security

Top concepts (fields/topics) attached by OpenAlex
Cited by: 12

Total citation count in OpenAlex
Citations by year (recent): 2025: 3, 2024: 6, 2023: 1, 2022: 1, 2021: 1

Per-year citation counts (last 5 years)
References (count): 96

Number of works referenced by this work
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W3093196515
doi	https://doi.org/10.12688/f1000research.26594.1
ids.doi	https://doi.org/10.12688/f1000research.26594.1
ids.mag	3093196515
ids.pmid	https://pubmed.ncbi.nlm.nih.gov/33628434
ids.openalex	https://openalex.org/W3093196515
fwci	0.50788825
mesh[0].qualifier_ui
mesh[0].descriptor_ui	D006801
mesh[0].is_major_topic	False
mesh[0].qualifier_name
mesh[0].descriptor_name	Humans
mesh[1].qualifier_ui
mesh[1].descriptor_ui	D015203
mesh[1].is_major_topic	False
mesh[1].qualifier_name
mesh[1].descriptor_name	Reproducibility of Results
mesh[2].qualifier_ui
mesh[2].descriptor_ui	D012108
mesh[2].is_major_topic	True
mesh[2].qualifier_name
mesh[2].descriptor_name	Research Personnel
mesh[3].qualifier_ui
mesh[3].descriptor_ui	D011795
mesh[3].is_major_topic	False
mesh[3].qualifier_name
mesh[3].descriptor_name	Surveys and Questionnaires
mesh[4].qualifier_ui
mesh[4].descriptor_ui	D014956
mesh[4].is_major_topic	True
mesh[4].qualifier_name
mesh[4].descriptor_name	Writing
mesh[5].qualifier_ui
mesh[5].descriptor_ui	D006801
mesh[5].is_major_topic	False
mesh[5].qualifier_name
mesh[5].descriptor_name	Humans
mesh[6].qualifier_ui
mesh[6].descriptor_ui	D015203
mesh[6].is_major_topic	False
mesh[6].qualifier_name
mesh[6].descriptor_name	Reproducibility of Results
mesh[7].qualifier_ui
mesh[7].descriptor_ui	D012108
mesh[7].is_major_topic	True
mesh[7].qualifier_name
mesh[7].descriptor_name	Research Personnel
mesh[8].qualifier_ui
mesh[8].descriptor_ui	D011795
mesh[8].is_major_topic	False
mesh[8].qualifier_name
mesh[8].descriptor_name	Surveys and Questionnaires
mesh[9].qualifier_ui
mesh[9].descriptor_ui	D014956
mesh[9].is_major_topic	True
mesh[9].qualifier_name
mesh[9].descriptor_name	Writing
type	preprint
title	Improving open and rigorous science: ten key future research opportunities related to rigor, reproducibility, and transparency in scientific research
awards[0].id	https://openalex.org/G1983528584
awards[0].funder_id	https://openalex.org/F4320306151
awards[0].display_name
awards[0].funder_award_id	G-2019-11438
awards[0].funder_display_name	Alfred P. Sloan Foundation
biblio.issue
biblio.volume	9
biblio.last_page	1235
biblio.first_page	1235
topics[0].id	https://openalex.org/T10206
topics[0].field.id	https://openalex.org/fields/18
topics[0].field.display_name	Decision Sciences
topics[0].score	0.9983999729156494
topics[0].domain.id	https://openalex.org/domains/2
topics[0].domain.display_name	Social Sciences
topics[0].subfield.id	https://openalex.org/subfields/1804
topics[0].subfield.display_name	Statistics, Probability and Uncertainty
topics[0].display_name	Meta-analysis and systematic reviews
topics[1].id	https://openalex.org/T11986
topics[1].field.id	https://openalex.org/fields/18
topics[1].field.display_name	Decision Sciences
topics[1].score	0.9937000274658203
topics[1].domain.id	https://openalex.org/domains/2
topics[1].domain.display_name	Social Sciences
topics[1].subfield.id	https://openalex.org/subfields/1802
topics[1].subfield.display_name	Information Systems and Management
topics[1].display_name	Scientific Computing and Data Management
topics[2].id	https://openalex.org/T10102
topics[2].field.id	https://openalex.org/fields/18
topics[2].field.display_name	Decision Sciences
topics[2].score	0.9742000102996826
topics[2].domain.id	https://openalex.org/domains/2
topics[2].domain.display_name	Social Sciences
topics[2].subfield.id	https://openalex.org/subfields/1804
topics[2].subfield.display_name	Statistics, Probability and Uncertainty
topics[2].display_name	scientometrics and bibliometrics research
funders[0].id	https://openalex.org/F4320306151
funders[0].ror	https://ror.org/052csg198
funders[0].display_name	Alfred P. Sloan Foundation
is_xpac	False
apc_list.value	1350
apc_list.currency	USD
apc_list.value_usd	1350
apc_paid.value	1350
apc_paid.currency	USD
apc_paid.value_usd	1350
concepts[0].id	https://openalex.org/C2778149293
concepts[0].level	2
concepts[0].score	0.9250595569610596
concepts[0].wikidata	https://www.wikidata.org/wiki/Q309823
concepts[0].display_name	Open science
concepts[1].id	https://openalex.org/C2983643791
concepts[1].level	3
concepts[1].score	0.7879546284675598
concepts[1].wikidata	https://www.wikidata.org/wiki/Q1999993
concepts[1].display_name	Open peer review
concepts[2].id	https://openalex.org/C2780233690
concepts[2].level	2
concepts[2].score	0.7843027114868164
concepts[2].wikidata	https://www.wikidata.org/wiki/Q535347
concepts[2].display_name	Transparency (behavior)
concepts[3].id	https://openalex.org/C2780535194
concepts[3].level	2
concepts[3].score	0.6952107548713684
concepts[3].wikidata	https://www.wikidata.org/wiki/Q309901
concepts[3].display_name	Open data
concepts[4].id	https://openalex.org/C2992936677
concepts[4].level	2
concepts[4].score	0.5809481143951416
concepts[4].wikidata	https://www.wikidata.org/wiki/Q441
concepts[4].display_name	Plant biology
concepts[5].id	https://openalex.org/C26517878
concepts[5].level	2
concepts[5].score	0.5690244436264038
concepts[5].wikidata	https://www.wikidata.org/wiki/Q228039
concepts[5].display_name	Key (lock)
concepts[6].id	https://openalex.org/C71008984
concepts[6].level	2
concepts[6].score	0.5539280772209167
concepts[6].wikidata	https://www.wikidata.org/wiki/Q2890076
concepts[6].display_name	Rigour
concepts[7].id	https://openalex.org/C2994190893
concepts[7].level	2
concepts[7].score	0.5443820357322693
concepts[7].wikidata	https://www.wikidata.org/wiki/Q91248609
concepts[7].display_name	Research integrity
concepts[8].id	https://openalex.org/C55587333
concepts[8].level	1
concepts[8].score	0.4754749834537506
concepts[8].wikidata	https://www.wikidata.org/wiki/Q1133029
concepts[8].display_name	Engineering ethics
concepts[9].id	https://openalex.org/C2778464652
concepts[9].level	2
concepts[9].score	0.4102928638458252
concepts[9].wikidata	https://www.wikidata.org/wiki/Q309849
concepts[9].display_name	Open research
concepts[10].id	https://openalex.org/C2522767166
concepts[10].level	1
concepts[10].score	0.3979434370994568
concepts[10].wikidata	https://www.wikidata.org/wiki/Q2374463
concepts[10].display_name	Data science
concepts[11].id	https://openalex.org/C71924100
concepts[11].level	0
concepts[11].score	0.3462485373020172
concepts[11].wikidata	https://www.wikidata.org/wiki/Q11190
concepts[11].display_name	Medicine
concepts[12].id	https://openalex.org/C539667460
concepts[12].level	1
concepts[12].score	0.34247639775276184
concepts[12].wikidata	https://www.wikidata.org/wiki/Q2414942
concepts[12].display_name	Management science
concepts[13].id	https://openalex.org/C41008148
concepts[13].level	0
concepts[13].score	0.33529558777809143
concepts[13].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[13].display_name	Computer science
concepts[14].id	https://openalex.org/C17744445
concepts[14].level	0
concepts[14].score	0.2963131368160248
concepts[14].wikidata	https://www.wikidata.org/wiki/Q36442
concepts[14].display_name	Political science
concepts[15].id	https://openalex.org/C86803240
concepts[15].level	0
concepts[15].score	0.19114360213279724
concepts[15].wikidata	https://www.wikidata.org/wiki/Q420
concepts[15].display_name	Biology
concepts[16].id	https://openalex.org/C127413603
concepts[16].level	0
concepts[16].score	0.18672335147857666
concepts[16].wikidata	https://www.wikidata.org/wiki/Q11023
concepts[16].display_name	Engineering
concepts[17].id	https://openalex.org/C33923547
concepts[17].level	0
concepts[17].score	0.14411941170692444
concepts[17].wikidata	https://www.wikidata.org/wiki/Q395
concepts[17].display_name	Mathematics
concepts[18].id	https://openalex.org/C136764020
concepts[18].level	1
concepts[18].score	0.1032605767250061
concepts[18].wikidata	https://www.wikidata.org/wiki/Q466
concepts[18].display_name	World Wide Web
concepts[19].id	https://openalex.org/C199539241
concepts[19].level	1
concepts[19].score	0.07113134860992432
concepts[19].wikidata	https://www.wikidata.org/wiki/Q7748
concepts[19].display_name	Law
concepts[20].id	https://openalex.org/C2524010
concepts[20].level	1
concepts[20].score	0.0
concepts[20].wikidata	https://www.wikidata.org/wiki/Q8087
concepts[20].display_name	Geometry
concepts[21].id	https://openalex.org/C105795698
concepts[21].level	1
concepts[21].score	0.0
concepts[21].wikidata	https://www.wikidata.org/wiki/Q12483
concepts[21].display_name	Statistics
concepts[22].id	https://openalex.org/C59822182
concepts[22].level	1
concepts[22].score	0.0
concepts[22].wikidata	https://www.wikidata.org/wiki/Q441
concepts[22].display_name	Botany
concepts[23].id	https://openalex.org/C38652104
concepts[23].level	1
concepts[23].score	0.0
concepts[23].wikidata	https://www.wikidata.org/wiki/Q3510521
concepts[23].display_name	Computer security
keywords[0].id	https://openalex.org/keywords/open-science
keywords[0].score	0.9250595569610596
keywords[0].display_name	Open science
keywords[1].id	https://openalex.org/keywords/open-peer-review
keywords[1].score	0.7879546284675598
keywords[1].display_name	Open peer review
keywords[2].id	https://openalex.org/keywords/transparency
keywords[2].score	0.7843027114868164
keywords[2].display_name	Transparency (behavior)
keywords[3].id	https://openalex.org/keywords/open-data
keywords[3].score	0.6952107548713684
keywords[3].display_name	Open data
keywords[4].id	https://openalex.org/keywords/plant-biology
keywords[4].score	0.5809481143951416
keywords[4].display_name	Plant biology
keywords[5].id	https://openalex.org/keywords/key
keywords[5].score	0.5690244436264038
keywords[5].display_name	Key (lock)
keywords[6].id	https://openalex.org/keywords/rigour
keywords[6].score	0.5539280772209167
keywords[6].display_name	Rigour
keywords[7].id	https://openalex.org/keywords/research-integrity
keywords[7].score	0.5443820357322693
keywords[7].display_name	Research integrity
keywords[8].id	https://openalex.org/keywords/engineering-ethics
keywords[8].score	0.4754749834537506
keywords[8].display_name	Engineering ethics
keywords[9].id	https://openalex.org/keywords/open-research
keywords[9].score	0.4102928638458252
keywords[9].display_name	Open research
keywords[10].id	https://openalex.org/keywords/data-science
keywords[10].score	0.3979434370994568
keywords[10].display_name	Data science
keywords[11].id	https://openalex.org/keywords/medicine
keywords[11].score	0.3462485373020172
keywords[11].display_name	Medicine
keywords[12].id	https://openalex.org/keywords/management-science
keywords[12].score	0.34247639775276184
keywords[12].display_name	Management science
keywords[13].id	https://openalex.org/keywords/computer-science
keywords[13].score	0.33529558777809143
keywords[13].display_name	Computer science
keywords[14].id	https://openalex.org/keywords/political-science
keywords[14].score	0.2963131368160248
keywords[14].display_name	Political science
keywords[15].id	https://openalex.org/keywords/biology
keywords[15].score	0.19114360213279724
keywords[15].display_name	Biology
keywords[16].id	https://openalex.org/keywords/engineering
keywords[16].score	0.18672335147857666
keywords[16].display_name	Engineering
keywords[17].id	https://openalex.org/keywords/mathematics
keywords[17].score	0.14411941170692444
keywords[17].display_name	Mathematics
keywords[18].id	https://openalex.org/keywords/world-wide-web
keywords[18].score	0.1032605767250061
keywords[18].display_name	World Wide Web
keywords[19].id	https://openalex.org/keywords/law
keywords[19].score	0.07113134860992432
keywords[19].display_name	Law
language	en
locations[0].id	doi:10.12688/f1000research.26594.1
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4210239046
locations[0].source.issn	2046-1402
locations[0].source.type	journal
locations[0].source.is_oa	True
locations[0].source.issn_l	2046-1402
locations[0].source.is_core	True
locations[0].source.is_in_doaj	True
locations[0].source.display_name	F1000Research
locations[0].source.host_organization	https://openalex.org/P4310320357
locations[0].source.host_organization_name	Faculty of 1000
locations[0].source.host_organization_lineage	https://openalex.org/P4310320357
locations[0].source.host_organization_lineage_names	Faculty of 1000
locations[0].license	cc-by
locations[0].pdf_url	https://f1000research.com/articles/9-1235/v1/pdf
locations[0].version	publishedVersion
locations[0].raw_type	journal-article
locations[0].license_id	https://openalex.org/licenses/cc-by
locations[0].is_accepted	True
locations[0].is_published	True
locations[0].raw_source_name	F1000Research
locations[0].landing_page_url	https://doi.org/10.12688/f1000research.26594.1
locations[1].id	pmid:33628434
locations[1].is_oa	False
locations[1].source.id	https://openalex.org/S4306525036
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	False
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	PubMed
locations[1].source.host_organization	https://openalex.org/I1299303238
locations[1].source.host_organization_name	National Institutes of Health
locations[1].source.host_organization_lineage	https://openalex.org/I1299303238
locations[1].license
locations[1].pdf_url
locations[1].version	publishedVersion
locations[1].raw_type
locations[1].license_id
locations[1].is_accepted	True
locations[1].is_published	True
locations[1].raw_source_name	F1000Research
locations[1].landing_page_url	https://pubmed.ncbi.nlm.nih.gov/33628434
locations[2].id	pmh:oai:doaj.org/article:f952230d206240f1ae2d6154976c610c
locations[2].is_oa	True
locations[2].source.id	https://openalex.org/S4306401280
locations[2].source.issn
locations[2].source.type	repository
locations[2].source.is_oa	False
locations[2].source.issn_l
locations[2].source.is_core	False
locations[2].source.is_in_doaj	False
locations[2].source.display_name	DOAJ (DOAJ: Directory of Open Access Journals)
locations[2].source.host_organization
locations[2].source.host_organization_name
locations[2].license	cc-by-sa
locations[2].pdf_url
locations[2].version	submittedVersion
locations[2].raw_type	article
locations[2].license_id	https://openalex.org/licenses/cc-by-sa
locations[2].is_accepted	False
locations[2].is_published	False
locations[2].raw_source_name	F1000Research, Vol 9 (2020)
locations[2].landing_page_url	https://doaj.org/article/f952230d206240f1ae2d6154976c610c
locations[3].id	pmh:oai:pubmedcentral.nih.gov:7898357
locations[3].is_oa	True
locations[3].source.id	https://openalex.org/S2764455111
locations[3].source.issn
locations[3].source.type	repository
locations[3].source.is_oa	False
locations[3].source.issn_l
locations[3].source.is_core	False
locations[3].source.is_in_doaj	False
locations[3].source.display_name	PubMed Central
locations[3].source.host_organization	https://openalex.org/I1299303238
locations[3].source.host_organization_name	National Institutes of Health
locations[3].source.host_organization_lineage	https://openalex.org/I1299303238
locations[3].license	cc-by
locations[3].pdf_url
locations[3].version	submittedVersion
locations[3].raw_type	Text
locations[3].license_id	https://openalex.org/licenses/cc-by
locations[3].is_accepted	False
locations[3].is_published	False
locations[3].raw_source_name	F1000Res
locations[3].landing_page_url	https://www.ncbi.nlm.nih.gov/pmc/articles/7898357
indexed_in	crossref, doaj, pubmed
authorships[0].author.id	https://openalex.org/A5002606014
authorships[0].author.orcid	https://orcid.org/0000-0002-2355-9881
authorships[0].author.display_name	Danny Valdez
authorships[0].countries	US
authorships[0].affiliations[0].institution_ids	https://openalex.org/I4210119109
authorships[0].affiliations[0].raw_affiliation_string	Indiana University School of Public Health, Bloomington, IN, 47403
authorships[0].institutions[0].id	https://openalex.org/I4210119109
authorships[0].institutions[0].ror	https://ror.org/02k40bc56
authorships[0].institutions[0].type	education
authorships[0].institutions[0].lineage	https://openalex.org/I4210119109, https://openalex.org/I592451
authorships[0].institutions[0].country_code	US
authorships[0].institutions[0].display_name	Indiana University Bloomington
authorships[0].author_position	first
authorships[0].raw_author_name	Danny Valdez
authorships[0].is_corresponding	False
authorships[0].raw_affiliation_strings	Indiana University School of Public Health, Bloomington, IN, 47403
authorships[1].author.id	https://openalex.org/A5074473858
authorships[1].author.orcid	https://orcid.org/0000-0003-4225-372X
authorships[1].author.display_name	Colby J. Vorland
authorships[1].countries	US
authorships[1].affiliations[0].institution_ids	https://openalex.org/I4210119109
authorships[1].affiliations[0].raw_affiliation_string	Indiana University School of Public Health, Bloomington, IN, 47403
authorships[1].institutions[0].id	https://openalex.org/I4210119109
authorships[1].institutions[0].ror	https://ror.org/02k40bc56
authorships[1].institutions[0].type	education
authorships[1].institutions[0].lineage	https://openalex.org/I4210119109, https://openalex.org/I592451
authorships[1].institutions[0].country_code	US
authorships[1].institutions[0].display_name	Indiana University Bloomington
authorships[1].author_position	middle
authorships[1].raw_author_name	Colby J. Vorland
authorships[1].is_corresponding	False
authorships[1].raw_affiliation_strings	Indiana University School of Public Health, Bloomington, IN, 47403
authorships[2].author.id	https://openalex.org/A5009250776
authorships[2].author.orcid	https://orcid.org/0000-0002-1758-8205
authorships[2].author.display_name	Andrew W. Brown
authorships[2].countries	US
authorships[2].affiliations[0].institution_ids	https://openalex.org/I4210119109
authorships[2].affiliations[0].raw_affiliation_string	Indiana University School of Public Health, Bloomington, IN, 47403
authorships[2].institutions[0].id	https://openalex.org/I4210119109
authorships[2].institutions[0].ror	https://ror.org/02k40bc56
authorships[2].institutions[0].type	education
authorships[2].institutions[0].lineage	https://openalex.org/I4210119109, https://openalex.org/I592451
authorships[2].institutions[0].country_code	US
authorships[2].institutions[0].display_name	Indiana University Bloomington
authorships[2].author_position	middle
authorships[2].raw_author_name	Andrew W. Brown
authorships[2].is_corresponding	False
authorships[2].raw_affiliation_strings	Indiana University School of Public Health, Bloomington, IN, 47403
authorships[3].author.id	https://openalex.org/A5055917772
authorships[3].author.orcid	https://orcid.org/0000-0001-6126-2459
authorships[3].author.display_name	Evan Mayo‐Wilson
authorships[3].countries	US
authorships[3].affiliations[0].institution_ids	https://openalex.org/I4210119109
authorships[3].affiliations[0].raw_affiliation_string	Indiana University School of Public Health, Bloomington, IN, 47403
authorships[3].institutions[0].id	https://openalex.org/I4210119109
authorships[3].institutions[0].ror	https://ror.org/02k40bc56
authorships[3].institutions[0].type	education
authorships[3].institutions[0].lineage	https://openalex.org/I4210119109, https://openalex.org/I592451
authorships[3].institutions[0].country_code	US
authorships[3].institutions[0].display_name	Indiana University Bloomington
authorships[3].author_position	middle
authorships[3].raw_author_name	Evan Mayo-Wilson
authorships[3].is_corresponding	False
authorships[3].raw_affiliation_strings	Indiana University School of Public Health, Bloomington, IN, 47403
authorships[4].author.id	https://openalex.org/A5036479518
authorships[4].author.orcid
authorships[4].author.display_name	Justin Otten
authorships[4].countries	US
authorships[4].affiliations[0].institution_ids	https://openalex.org/I4210119109
authorships[4].affiliations[0].raw_affiliation_string	Indiana University School of Public Health, Bloomington, IN, 47403
authorships[4].institutions[0].id	https://openalex.org/I4210119109
authorships[4].institutions[0].ror	https://ror.org/02k40bc56
authorships[4].institutions[0].type	education
authorships[4].institutions[0].lineage	https://openalex.org/I4210119109, https://openalex.org/I592451
authorships[4].institutions[0].country_code	US
authorships[4].institutions[0].display_name	Indiana University Bloomington
authorships[4].author_position	middle
authorships[4].raw_author_name	Justin Otten
authorships[4].is_corresponding	False
authorships[4].raw_affiliation_strings	Indiana University School of Public Health, Bloomington, IN, 47403
authorships[5].author.id	https://openalex.org/A5023400549
authorships[5].author.orcid	https://orcid.org/0000-0002-7413-3944
authorships[5].author.display_name	Richard Ball
authorships[5].countries	US
authorships[5].affiliations[0].institution_ids	https://openalex.org/I155707491
authorships[5].affiliations[0].raw_affiliation_string	Project TIER, Haverford College, Haverford, Pennsylvania, 19041
authorships[5].institutions[0].id	https://openalex.org/I155707491
authorships[5].institutions[0].ror	https://ror.org/04fnrxr62
authorships[5].institutions[0].type	education
authorships[5].institutions[0].lineage	https://openalex.org/I155707491
authorships[5].institutions[0].country_code	US
authorships[5].institutions[0].display_name	Haverford College
authorships[5].author_position	middle
authorships[5].raw_author_name	Richard Ball
authorships[5].is_corresponding	False
authorships[5].raw_affiliation_strings	Project TIER, Haverford College, Haverford, Pennsylvania, 19041
authorships[6].author.id	https://openalex.org/A5056179382
authorships[6].author.orcid	https://orcid.org/0000-0002-7775-3022
authorships[6].author.display_name	Sean Grant
authorships[6].countries	US
authorships[6].affiliations[0].institution_ids	https://openalex.org/I55769427
authorships[6].affiliations[0].raw_affiliation_string	Indiana University Purdue University Indianapolis Fairbanks School of Public Health, Indianapolis, IN, 46223
authorships[6].institutions[0].id	https://openalex.org/I55769427
authorships[6].institutions[0].ror	https://ror.org/05gxnyn08
authorships[6].institutions[0].type	education
authorships[6].institutions[0].lineage	https://openalex.org/I55769427, https://openalex.org/I592451
authorships[6].institutions[0].country_code	US
authorships[6].institutions[0].display_name	Indiana University – Purdue University Indianapolis
authorships[6].author_position	middle
authorships[6].raw_author_name	Sean Grant
authorships[6].is_corresponding	False
authorships[6].raw_affiliation_strings	Indiana University Purdue University Indianapolis Fairbanks School of Public Health, Indianapolis, IN, 46223
authorships[7].author.id	https://openalex.org/A5101471684
authorships[7].author.orcid	https://orcid.org/0000-0003-2544-5711
authorships[7].author.display_name	Rachel Lévy
authorships[7].countries	US
authorships[7].affiliations[0].institution_ids	https://openalex.org/I192560614
authorships[7].affiliations[0].raw_affiliation_string	Rachel Levy, Mathematical Association of America, 1529 18th St. NW, Washington, DC, 20036
authorships[7].institutions[0].id	https://openalex.org/I192560614
authorships[7].institutions[0].ror	https://ror.org/00701k896
authorships[7].institutions[0].type	other
authorships[7].institutions[0].lineage	https://openalex.org/I192560614
authorships[7].institutions[0].country_code	US
authorships[7].institutions[0].display_name	Mathematical Association of America
authorships[7].author_position	middle
authorships[7].raw_author_name	Rachel Levy
authorships[7].is_corresponding	False
authorships[7].raw_affiliation_strings	Rachel Levy, Mathematical Association of America, 1529 18th St. NW, Washington, DC, 20036
authorships[8].author.id	https://openalex.org/A5044120243
authorships[8].author.orcid	https://orcid.org/0000-0003-3286-3540
authorships[8].author.display_name	Dubravka Svetina
authorships[8].countries	US
authorships[8].affiliations[0].institution_ids	https://openalex.org/I4210119109
authorships[8].affiliations[0].raw_affiliation_string	Indiana University School of Education, Bloomington, IN, 47401
authorships[8].institutions[0].id	https://openalex.org/I4210119109
authorships[8].institutions[0].ror	https://ror.org/02k40bc56
authorships[8].institutions[0].type	education
authorships[8].institutions[0].lineage	https://openalex.org/I4210119109, https://openalex.org/I592451
authorships[8].institutions[0].country_code	US
authorships[8].institutions[0].display_name	Indiana University Bloomington
authorships[8].author_position	middle
authorships[8].raw_author_name	Dubravka Svetina Valdivia
authorships[8].is_corresponding	False
authorships[8].raw_affiliation_strings	Indiana University School of Education, Bloomington, IN, 47401
authorships[9].author.id	https://openalex.org/A5062622249
authorships[9].author.orcid	https://orcid.org/0000-0003-3566-9399
authorships[9].author.display_name	David B. Allison
authorships[9].countries	US
authorships[9].affiliations[0].institution_ids	https://openalex.org/I4210119109
authorships[9].affiliations[0].raw_affiliation_string	Indiana University School of Public Health, Bloomington, IN, 47403
authorships[9].institutions[0].id	https://openalex.org/I4210119109
authorships[9].institutions[0].ror	https://ror.org/02k40bc56
authorships[9].institutions[0].type	education
authorships[9].institutions[0].lineage	https://openalex.org/I4210119109, https://openalex.org/I592451
authorships[9].institutions[0].country_code	US
authorships[9].institutions[0].display_name	Indiana University Bloomington
authorships[9].author_position	last
authorships[9].raw_author_name	David B. Allison
authorships[9].is_corresponding	False
authorships[9].raw_affiliation_strings	Indiana University School of Public Health, Bloomington, IN, 47403
has_content.pdf	True
has_content.grobid_xml	True
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://f1000research.com/articles/9-1235/v1/pdf
open_access.oa_status	gold
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	Improving open and rigorous science: ten key future research opportunities related to rigor, reproducibility, and transparency in scientific research
has_fulltext	True
is_retracted	False
updated_date	2025-11-06T03:46:38.306776
primary_topic.id	https://openalex.org/T10206
primary_topic.field.id	https://openalex.org/fields/18
primary_topic.field.display_name	Decision Sciences
primary_topic.score	0.9983999729156494
primary_topic.domain.id	https://openalex.org/domains/2
primary_topic.domain.display_name	Social Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1804
primary_topic.subfield.display_name	Statistics, Probability and Uncertainty
primary_topic.display_name	Meta-analysis and systematic reviews
related_works	https://openalex.org/W2345835657, https://openalex.org/W2293871551, https://openalex.org/W4200142686, https://openalex.org/W3017109063, https://openalex.org/W2921772030, https://openalex.org/W3006255515, https://openalex.org/W3120228629, https://openalex.org/W4386356460, https://openalex.org/W4297803948, https://openalex.org/W3093267682
cited_by_count	12
counts_by_year[0].year	2025
counts_by_year[0].cited_by_count	3
counts_by_year[1].year	2024
counts_by_year[1].cited_by_count	6
counts_by_year[2].year	2023
counts_by_year[2].cited_by_count	1
counts_by_year[3].year	2022
counts_by_year[3].cited_by_count	1
counts_by_year[4].year	2021
counts_by_year[4].cited_by_count	1
locations_count	4
best_oa_location.id	doi:10.12688/f1000research.26594.1
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4210239046
best_oa_location.source.issn	2046-1402
best_oa_location.source.type	journal
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l	2046-1402
best_oa_location.source.is_core	True
best_oa_location.source.is_in_doaj	True
best_oa_location.source.display_name	F1000Research
best_oa_location.source.host_organization	https://openalex.org/P4310320357
best_oa_location.source.host_organization_name	Faculty of 1000
best_oa_location.source.host_organization_lineage	https://openalex.org/P4310320357
best_oa_location.source.host_organization_lineage_names	Faculty of 1000
best_oa_location.license	cc-by
best_oa_location.pdf_url	https://f1000research.com/articles/9-1235/v1/pdf
best_oa_location.version	publishedVersion
best_oa_location.raw_type	journal-article
best_oa_location.license_id	https://openalex.org/licenses/cc-by
best_oa_location.is_accepted	True
best_oa_location.is_published	True
best_oa_location.raw_source_name	F1000Research
best_oa_location.landing_page_url	https://doi.org/10.12688/f1000research.26594.1
primary_location.id	doi:10.12688/f1000research.26594.1
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4210239046
primary_location.source.issn	2046-1402
primary_location.source.type	journal
primary_location.source.is_oa	True
primary_location.source.issn_l	2046-1402
primary_location.source.is_core	True
primary_location.source.is_in_doaj	True
primary_location.source.display_name	F1000Research
primary_location.source.host_organization	https://openalex.org/P4310320357
primary_location.source.host_organization_name	Faculty of 1000
primary_location.source.host_organization_lineage	https://openalex.org/P4310320357
primary_location.source.host_organization_lineage_names	Faculty of 1000
primary_location.license	cc-by
primary_location.pdf_url	https://f1000research.com/articles/9-1235/v1/pdf
primary_location.version	publishedVersion
primary_location.raw_type	journal-article
primary_location.license_id	https://openalex.org/licenses/cc-by
primary_location.is_accepted	True
primary_location.is_published	True
primary_location.raw_source_name	F1000Research
primary_location.landing_page_url	https://doi.org/10.12688/f1000research.26594.1
publication_date	2020-10-14
publication_year	2020
referenced_works	https://openalex.org/W2345555849, https://openalex.org/W2943216811, https://openalex.org/W2008966430, https://openalex.org/W1968382311, https://openalex.org/W2022160389, https://openalex.org/W1665897291, https://openalex.org/W1979916898, https://openalex.org/W2079233726, https://openalex.org/W1913246108, https://openalex.org/W2060863409, https://openalex.org/W2709416302, https://openalex.org/W2055222843, https://openalex.org/W2789550191, https://openalex.org/W2781950503, https://openalex.org/W2948658587, https://openalex.org/W2969655117, https://openalex.org/W2790042388, https://openalex.org/W2403850581, https://openalex.org/W2119274873, https://openalex.org/W2669242161, https://openalex.org/W2112194552, https://openalex.org/W2918372257, https://openalex.org/W2148405837, https://openalex.org/W2101264903, https://openalex.org/W2754718739, https://openalex.org/W2572857085, https://openalex.org/W121960715, https://openalex.org/W2048060485, https://openalex.org/W2905049168, https://openalex.org/W2118364654, https://openalex.org/W1592189332, https://openalex.org/W2018443796, https://openalex.org/W2790858109, https://openalex.org/W6617397750, https://openalex.org/W2318114766, https://openalex.org/W2779283988, https://openalex.org/W2414283705, https://openalex.org/W631266573, https://openalex.org/W4242732983, https://openalex.org/W2162696461, https://openalex.org/W2807389295, https://openalex.org/W2809457638, https://openalex.org/W2169214811, https://openalex.org/W1767553509, https://openalex.org/W2740649455, https://openalex.org/W2011360123, https://openalex.org/W4241077770, https://openalex.org/W2066751996, https://openalex.org/W2918695271, https://openalex.org/W3021485906, https://openalex.org/W4256332921, https://openalex.org/W1953064436, https://openalex.org/W3037533174, https://openalex.org/W6636501135, https://openalex.org/W1663021616, https://openalex.org/W2279621333, https://openalex.org/W4249665717, https://openalex.org/W2102381912, https://openalex.org/W2793111149, https://openalex.org/W2801860740, https://openalex.org/W4240085069, https://openalex.org/W2944196828, https://openalex.org/W2139598698, https://openalex.org/W2783929487, https://openalex.org/W2148203699, https://openalex.org/W1897139626, https://openalex.org/W1825120118, https://openalex.org/W6826703204, https://openalex.org/W3043305063, https://openalex.org/W1982716150, https://openalex.org/W1969456350, https://openalex.org/W2085815446, https://openalex.org/W2062187425, https://openalex.org/W1986467665, https://openalex.org/W2935352943, https://openalex.org/W2995661940, https://openalex.org/W2103604968, https://openalex.org/W1523437445, https://openalex.org/W2152445141, https://openalex.org/W2060017626, https://openalex.org/W3022423927, https://openalex.org/W2992292486, https://openalex.org/W615440381, https://openalex.org/W2901028957, https://openalex.org/W2267831916, https://openalex.org/W3197147551, https://openalex.org/W3124333825, https://openalex.org/W4302805728, https://openalex.org/W2263593861, https://openalex.org/W4235094003, https://openalex.org/W4213312140, https://openalex.org/W2890369537, https://openalex.org/W2513016233, https://openalex.org/W4241152157, https://openalex.org/W2406493898, https://openalex.org/W1618499061
referenced_works_count	96
abstract_inverted_index.a	3, 19, 32, 114
abstract_inverted_index.Do	198, 228
abstract_inverted_index.Is	244
abstract_inverted_index.at	23
abstract_inverted_index.be	275
abstract_inverted_index.do	206
abstract_inverted_index.in	34, 65, 74, 148, 185, 216
abstract_inverted_index.is	178
abstract_inverted_index.it	245
abstract_inverted_index.of	2, 21, 28, 92, 116, 183, 224, 239
abstract_inverted_index.on	289
abstract_inverted_index.to	6, 40, 49, 56, 122, 158, 230, 247, 254, 269, 277, 284
abstract_inverted_index.we	207
abstract_inverted_index.(1)	69, 133
abstract_inverted_index.(2)	76, 144
abstract_inverted_index.(3)	85, 153
abstract_inverted_index.(4)	166
abstract_inverted_index.(5)	176
abstract_inverted_index.(6)	197
abstract_inverted_index.(7)	204
abstract_inverted_index.(8)	212
abstract_inverted_index.(9)	227
abstract_inverted_index.Can	134, 145
abstract_inverted_index.How	167, 205, 213
abstract_inverted_index.RRT	45
abstract_inverted_index.The	111
abstract_inverted_index.aid	231
abstract_inverted_index.all	123
abstract_inverted_index.and	13, 53, 63, 80, 84, 90, 104, 137, 181, 193, 222, 237, 242, 251, 258, 264, 281
abstract_inverted_index.can	168
abstract_inverted_index.for	44, 125
abstract_inverted_index.key	42
abstract_inverted_index.may	274
abstract_inverted_index.the	24, 98, 107, 128, 162, 179, 278
abstract_inverted_index.use	223
abstract_inverted_index.(10)	243
abstract_inverted_index.2019	36
abstract_inverted_index.Does	154
abstract_inverted_index.RRT,	75
abstract_inverted_index.RRT.	271
abstract_inverted_index.This	272
abstract_inverted_index.What	177
abstract_inverted_index.does	214
abstract_inverted_index.face	157
abstract_inverted_index.gave	61
abstract_inverted_index.help	171
abstract_inverted_index.list	115, 273
abstract_inverted_index.more	174
abstract_inverted_index.part	1
abstract_inverted_index.team	20
abstract_inverted_index.were	120
abstract_inverted_index.with	37
abstract_inverted_index.&	72
abstract_inverted_index.(RRT)	15
abstract_inverted_index.(e.g.	156
abstract_inverted_index.(i.e.	287
abstract_inverted_index.aimed	48
abstract_inverted_index.error	199, 210
abstract_inverted_index.face,	159
abstract_inverted_index.norms	252
abstract_inverted_index.other	194
abstract_inverted_index.their	102
abstract_inverted_index.three	66
abstract_inverted_index.tools	229
abstract_inverted_index.which	119
abstract_inverted_index.(e.g.,	188
abstract_inverted_index.Errors	79
abstract_inverted_index.Public	29
abstract_inverted_index.School	27
abstract_inverted_index.across	16
abstract_inverted_index.affect	161, 219
abstract_inverted_index.around	10
abstract_inverted_index.effort	5
abstract_inverted_index.errors	173, 184
abstract_inverted_index.expand	7
abstract_inverted_index.groups	100
abstract_inverted_index.hosted	31
abstract_inverted_index.impact	182
abstract_inverted_index.reduce	202
abstract_inverted_index.rigor,	11
abstract_inverted_index.useful	276
abstract_inverted_index.values	250
abstract_inverted_index.Indiana	25
abstract_inverted_index.Looking	86
abstract_inverted_index.October	35
abstract_inverted_index.advance	285
abstract_inverted_index.clarity	238
abstract_inverted_index.courses	140
abstract_inverted_index.discuss	41
abstract_inverted_index.errors,	192
abstract_inverted_index.groups:	68
abstract_inverted_index.improve	141, 151
abstract_inverted_index.leaders	39
abstract_inverted_index.online)	160
abstract_inverted_index.related	55, 253, 268
abstract_inverted_index.reports	234
abstract_inverted_index.seeking	283
abstract_inverted_index.working	67, 99
abstract_inverted_index.writing	150, 232
abstract_inverted_index.Accuracy	91
abstract_inverted_index.Analytic	82
abstract_inverted_index.Outward:	87
abstract_inverted_index.Reducing	77
abstract_inverted_index.Research	93
abstract_inverted_index.Training	73
abstract_inverted_index.activity	9
abstract_inverted_index.analytic	189
abstract_inverted_index.compiled	113
abstract_inverted_index.efficacy	163
abstract_inverted_index.errors?;	203
abstract_inverted_index.identify	50, 172
abstract_inverted_index.increase	235
abstract_inverted_index.modality	155
abstract_inverted_index.modeling	139
abstract_inverted_index.possible	246
abstract_inverted_index.priority	130
abstract_inverted_index.programs	170
abstract_inverted_index.research	8, 51, 108, 117, 131, 217, 225, 233, 240, 288
abstract_inverted_index.training	147
abstract_inverted_index.workshop	33, 47, 59
abstract_inverted_index.Education	71
abstract_inverted_index.Following	95
abstract_inverted_index.Improving	70
abstract_inverted_index.accurate,	257
abstract_inverted_index.attendees	60
abstract_inverted_index.automated	169
abstract_inverted_index.community	280
abstract_inverted_index.discussed	106
abstract_inverted_index.encourage	208
abstract_inverted_index.errors)?;	196
abstract_inverted_index.findings,	103
abstract_inverted_index.following	129
abstract_inverted_index.important	263
abstract_inverted_index.improving	270
abstract_inverted_index.inculcate	248
abstract_inverted_index.objective	195
abstract_inverted_index.presented	101
abstract_inverted_index.questions	267
abstract_inverted_index.reports?;	241
abstract_inverted_index.rigorous,	256
abstract_inverted_index.truthful,	255
abstract_inverted_index.two-days,	58
abstract_inverted_index.workflows	201
abstract_inverted_index.Increasing	81, 88
abstract_inverted_index.University	26
abstract_inverted_index.circulated	121
abstract_inverted_index.evidence?;	226
abstract_inverted_index.identified	127, 262
abstract_inverted_index.practice?;	143
abstract_inverted_index.prevalence	180
abstract_inverted_index.prevention	200
abstract_inverted_index.priorities	52
abstract_inverted_index.questions:	132
abstract_inverted_index.relatively	265
abstract_inverted_index.scientific	17, 149, 186, 249, 260, 279
abstract_inverted_index.statistics	136, 142
abstract_inverted_index.unexplored	266
abstract_inverted_index.‘spin’	215
abstract_inverted_index.RRT-focused	135
abstract_inverted_index.RRT-related	164
abstract_inverted_index.Statistical	78
abstract_inverted_index.coordinated	4
abstract_inverted_index.education?;	165
abstract_inverted_index.identified.	110
abstract_inverted_index.priorities,	118
abstract_inverted_index.small-group	96
abstract_inverted_index.specialized	146
abstract_inverted_index.stakeholder	220
abstract_inverted_index.statistical	191
abstract_inverted_index.Truthfulness	89
abstract_inverted_index.correction?;	211
abstract_inverted_index.disciplines,	18
abstract_inverted_index.discussions,	97
abstract_inverted_index.mathematical	138
abstract_inverted_index.meta-science	286
abstract_inverted_index.participants	105, 124
abstract_inverted_index.participated	64
abstract_inverted_index.publications	187
abstract_inverted_index.transparency	14
abstract_inverted_index.Transparency,	83
abstract_inverted_index.communication	218
abstract_inverted_index.comprehensive	259
abstract_inverted_index.efficiently?;	175
abstract_inverted_index.international	38
abstract_inverted_index.investigators	22, 112, 282
abstract_inverted_index.opportunities	43, 54, 109
abstract_inverted_index.presentations	62
abstract_inverted_index.understanding	221
abstract_inverted_index.transparency?;	152
abstract_inverted_index.Communications.	94
abstract_inverted_index.inconsistencies,	190
abstract_inverted_index.post-publication	209
abstract_inverted_index.reproducibility,	12
abstract_inverted_index.comprehensiveness	236
abstract_inverted_index.Health-Bloomington	30
abstract_inverted_index.research).</ns3:p>	290
abstract_inverted_index.<ns3:p><ns3:bold>Background:</ns3:bold>As	0
abstract_inverted_index.RRT.</ns3:p><ns3:p/><ns3:p><ns3:bold>Design:</ns3:bold>Over	57
abstract_inverted_index.research.</ns3:p><ns3:p/><ns3:p><ns3:bold>Objective:</ns3:bold>The	46
abstract_inverted_index.feedback.</ns3:p><ns3:p/><ns3:p><ns3:bold>Results:</ns3:bold>Participants	126
abstract_inverted_index.reporting?</ns3:p><ns3:p/><ns3:p><ns3:bold>Conclusion:</ns3:bold>Participants	261
cited_by_percentile_year.max	98
cited_by_percentile_year.min	89
countries_distinct_count	1
institutions_distinct_count	10
sustainable_development_goals[0].id	https://metadata.un.org/sdg/4
sustainable_development_goals[0].score	0.75
sustainable_development_goals[0].display_name	Quality Education
citation_normalized_percentile.value	0.71951868
citation_normalized_percentile.is_in_top_1_percent	False
citation_normalized_percentile.is_in_top_10_percent	False