Monitoring Fidelity of Online Reinforcement Learning Algorithms in Clinical Trials Article Swipe
Anna L. Trella
,
Kelly Zhang
,
Inbal Nahum‐Shani
,
Vivek Shetty
,
Iris Yan
,
Finale Doshi‐Velez
,
Susan A. Murphy
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2402.17003
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2402.17003
Online reinforcement learning (RL) algorithms offer great potential for personalizing treatment for participants in clinical trials. However, deploying an online, autonomous algorithm in the high-stakes healthcare setting makes quality control and data quality especially difficult to achieve. This paper proposes algorithm fidelity as a critical requirement for deploying online RL algorithms in clinical trials. It emphasizes the responsibility of the algorithm to (1) safeguard participants and (2) preserve the scientific utility of the data for post-trial analyses. We also present a framework for pre-deployment planning and real-time monitoring to help algorithm developers and clinical researchers ensure algorithm fidelity. To illustrate our framework's practical application, we present real-world examples from the Oralytics clinical trial. Since Spring 2023, this trial successfully deployed an autonomous, online RL algorithm to personalize behavioral interventions for participants at risk for dental disease.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2402.17003
- https://arxiv.org/pdf/2402.17003
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4392270708
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4392270708Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2402.17003Digital Object Identifier
- Title
-
Monitoring Fidelity of Online Reinforcement Learning Algorithms in Clinical TrialsWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-02-26Full publication date if available
- Authors
-
Anna L. Trella, Kelly Zhang, Inbal Nahum‐Shani, Vivek Shetty, Iris Yan, Finale Doshi‐Velez, Susan A. MurphyList of authors in order
- Landing page
-
https://arxiv.org/abs/2402.17003Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2402.17003Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2402.17003Direct OA link when available
- Concepts
-
Reinforcement learning, Fidelity, Computer science, Artificial intelligence, Machine learning, Reinforcement, Algorithm, Psychology, Social psychology, TelecommunicationsTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4392270708 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2402.17003 |
| ids.doi | https://doi.org/10.48550/arxiv.2402.17003 |
| ids.openalex | https://openalex.org/W4392270708 |
| fwci | |
| type | preprint |
| title | Monitoring Fidelity of Online Reinforcement Learning Algorithms in Clinical Trials |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T11235 |
| topics[0].field.id | https://openalex.org/fields/26 |
| topics[0].field.display_name | Mathematics |
| topics[0].score | 0.7644000053405762 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/2613 |
| topics[0].subfield.display_name | Statistics and Probability |
| topics[0].display_name | Statistical Methods in Clinical Trials |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C97541855 |
| concepts[0].level | 2 |
| concepts[0].score | 0.7805043458938599 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q830687 |
| concepts[0].display_name | Reinforcement learning |
| concepts[1].id | https://openalex.org/C2776459999 |
| concepts[1].level | 2 |
| concepts[1].score | 0.7489950656890869 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q2119376 |
| concepts[1].display_name | Fidelity |
| concepts[2].id | https://openalex.org/C41008148 |
| concepts[2].level | 0 |
| concepts[2].score | 0.6422865390777588 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[2].display_name | Computer science |
| concepts[3].id | https://openalex.org/C154945302 |
| concepts[3].level | 1 |
| concepts[3].score | 0.4946405589580536 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[3].display_name | Artificial intelligence |
| concepts[4].id | https://openalex.org/C119857082 |
| concepts[4].level | 1 |
| concepts[4].score | 0.48856648802757263 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q2539 |
| concepts[4].display_name | Machine learning |
| concepts[5].id | https://openalex.org/C67203356 |
| concepts[5].level | 2 |
| concepts[5].score | 0.43714195489883423 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q1321905 |
| concepts[5].display_name | Reinforcement |
| concepts[6].id | https://openalex.org/C11413529 |
| concepts[6].level | 1 |
| concepts[6].score | 0.40659987926483154 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q8366 |
| concepts[6].display_name | Algorithm |
| concepts[7].id | https://openalex.org/C15744967 |
| concepts[7].level | 0 |
| concepts[7].score | 0.2441801130771637 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q9418 |
| concepts[7].display_name | Psychology |
| concepts[8].id | https://openalex.org/C77805123 |
| concepts[8].level | 1 |
| concepts[8].score | 0.0811537504196167 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q161272 |
| concepts[8].display_name | Social psychology |
| concepts[9].id | https://openalex.org/C76155785 |
| concepts[9].level | 1 |
| concepts[9].score | 0.06267058849334717 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q418 |
| concepts[9].display_name | Telecommunications |
| keywords[0].id | https://openalex.org/keywords/reinforcement-learning |
| keywords[0].score | 0.7805043458938599 |
| keywords[0].display_name | Reinforcement learning |
| keywords[1].id | https://openalex.org/keywords/fidelity |
| keywords[1].score | 0.7489950656890869 |
| keywords[1].display_name | Fidelity |
| keywords[2].id | https://openalex.org/keywords/computer-science |
| keywords[2].score | 0.6422865390777588 |
| keywords[2].display_name | Computer science |
| keywords[3].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[3].score | 0.4946405589580536 |
| keywords[3].display_name | Artificial intelligence |
| keywords[4].id | https://openalex.org/keywords/machine-learning |
| keywords[4].score | 0.48856648802757263 |
| keywords[4].display_name | Machine learning |
| keywords[5].id | https://openalex.org/keywords/reinforcement |
| keywords[5].score | 0.43714195489883423 |
| keywords[5].display_name | Reinforcement |
| keywords[6].id | https://openalex.org/keywords/algorithm |
| keywords[6].score | 0.40659987926483154 |
| keywords[6].display_name | Algorithm |
| keywords[7].id | https://openalex.org/keywords/psychology |
| keywords[7].score | 0.2441801130771637 |
| keywords[7].display_name | Psychology |
| keywords[8].id | https://openalex.org/keywords/social-psychology |
| keywords[8].score | 0.0811537504196167 |
| keywords[8].display_name | Social psychology |
| keywords[9].id | https://openalex.org/keywords/telecommunications |
| keywords[9].score | 0.06267058849334717 |
| keywords[9].display_name | Telecommunications |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2402.17003 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2402.17003 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2402.17003 |
| locations[1].id | doi:10.48550/arxiv.2402.17003 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | cc-by |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | https://openalex.org/licenses/cc-by |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2402.17003 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5039583451 |
| authorships[0].author.orcid | https://orcid.org/0000-0003-4779-9115 |
| authorships[0].author.display_name | Anna L. Trella |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Trella, Anna L. |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5101990414 |
| authorships[1].author.orcid | https://orcid.org/0000-0003-0850-4978 |
| authorships[1].author.display_name | Kelly Zhang |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Zhang, Kelly W. |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5034549483 |
| authorships[2].author.orcid | https://orcid.org/0000-0001-6138-9089 |
| authorships[2].author.display_name | Inbal Nahum‐Shani |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Nahum-Shani, Inbal |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5047665844 |
| authorships[3].author.orcid | https://orcid.org/0000-0002-3167-3318 |
| authorships[3].author.display_name | Vivek Shetty |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Shetty, Vivek |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5051733612 |
| authorships[4].author.orcid | |
| authorships[4].author.display_name | Iris Yan |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Yan, Iris |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5038771285 |
| authorships[5].author.orcid | https://orcid.org/0000-0003-2886-3898 |
| authorships[5].author.display_name | Finale Doshi‐Velez |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Doshi-Velez, Finale |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5070589969 |
| authorships[6].author.orcid | https://orcid.org/0000-0002-2032-4286 |
| authorships[6].author.display_name | Susan A. Murphy |
| authorships[6].author_position | last |
| authorships[6].raw_author_name | Murphy, Susan A. |
| authorships[6].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2402.17003 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2024-03-05T00:00:00 |
| display_name | Monitoring Fidelity of Online Reinforcement Learning Algorithms in Clinical Trials |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T11235 |
| primary_topic.field.id | https://openalex.org/fields/26 |
| primary_topic.field.display_name | Mathematics |
| primary_topic.score | 0.7644000053405762 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/2613 |
| primary_topic.subfield.display_name | Statistics and Probability |
| primary_topic.display_name | Statistical Methods in Clinical Trials |
| related_works | https://openalex.org/W2920061524, https://openalex.org/W4310083477, https://openalex.org/W1977959518, https://openalex.org/W2038908348, https://openalex.org/W2381850946, https://openalex.org/W4380449851, https://openalex.org/W2107890255, https://openalex.org/W2106552856, https://openalex.org/W2145821588, https://openalex.org/W2086122291 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2402.17003 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2402.17003 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2402.17003 |
| primary_location.id | pmh:oai:arXiv.org:2402.17003 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2402.17003 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2402.17003 |
| publication_date | 2024-02-26 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 43, 80 |
| abstract_inverted_index.It | 54 |
| abstract_inverted_index.RL | 49, 123 |
| abstract_inverted_index.To | 98 |
| abstract_inverted_index.We | 77 |
| abstract_inverted_index.an | 18, 120 |
| abstract_inverted_index.as | 42 |
| abstract_inverted_index.at | 131 |
| abstract_inverted_index.in | 13, 22, 51 |
| abstract_inverted_index.of | 58, 71 |
| abstract_inverted_index.to | 35, 61, 88, 125 |
| abstract_inverted_index.we | 104 |
| abstract_inverted_index.(1) | 62 |
| abstract_inverted_index.(2) | 66 |
| abstract_inverted_index.and | 30, 65, 85, 92 |
| abstract_inverted_index.for | 8, 11, 46, 74, 82, 129, 133 |
| abstract_inverted_index.our | 100 |
| abstract_inverted_index.the | 23, 56, 59, 68, 72, 109 |
| abstract_inverted_index.(RL) | 3 |
| abstract_inverted_index.This | 37 |
| abstract_inverted_index.also | 78 |
| abstract_inverted_index.data | 31, 73 |
| abstract_inverted_index.from | 108 |
| abstract_inverted_index.help | 89 |
| abstract_inverted_index.risk | 132 |
| abstract_inverted_index.this | 116 |
| abstract_inverted_index.2023, | 115 |
| abstract_inverted_index.Since | 113 |
| abstract_inverted_index.great | 6 |
| abstract_inverted_index.makes | 27 |
| abstract_inverted_index.offer | 5 |
| abstract_inverted_index.paper | 38 |
| abstract_inverted_index.trial | 117 |
| abstract_inverted_index.Online | 0 |
| abstract_inverted_index.Spring | 114 |
| abstract_inverted_index.dental | 134 |
| abstract_inverted_index.ensure | 95 |
| abstract_inverted_index.online | 48, 122 |
| abstract_inverted_index.trial. | 112 |
| abstract_inverted_index.control | 29 |
| abstract_inverted_index.online, | 19 |
| abstract_inverted_index.present | 79, 105 |
| abstract_inverted_index.quality | 28, 32 |
| abstract_inverted_index.setting | 26 |
| abstract_inverted_index.trials. | 15, 53 |
| abstract_inverted_index.utility | 70 |
| abstract_inverted_index.However, | 16 |
| abstract_inverted_index.achieve. | 36 |
| abstract_inverted_index.clinical | 14, 52, 93, 111 |
| abstract_inverted_index.critical | 44 |
| abstract_inverted_index.deployed | 119 |
| abstract_inverted_index.disease. | 135 |
| abstract_inverted_index.examples | 107 |
| abstract_inverted_index.fidelity | 41 |
| abstract_inverted_index.learning | 2 |
| abstract_inverted_index.planning | 84 |
| abstract_inverted_index.preserve | 67 |
| abstract_inverted_index.proposes | 39 |
| abstract_inverted_index.Oralytics | 110 |
| abstract_inverted_index.algorithm | 21, 40, 60, 90, 96, 124 |
| abstract_inverted_index.analyses. | 76 |
| abstract_inverted_index.deploying | 17, 47 |
| abstract_inverted_index.difficult | 34 |
| abstract_inverted_index.fidelity. | 97 |
| abstract_inverted_index.framework | 81 |
| abstract_inverted_index.potential | 7 |
| abstract_inverted_index.practical | 102 |
| abstract_inverted_index.real-time | 86 |
| abstract_inverted_index.safeguard | 63 |
| abstract_inverted_index.treatment | 10 |
| abstract_inverted_index.algorithms | 4, 50 |
| abstract_inverted_index.autonomous | 20 |
| abstract_inverted_index.behavioral | 127 |
| abstract_inverted_index.developers | 91 |
| abstract_inverted_index.emphasizes | 55 |
| abstract_inverted_index.especially | 33 |
| abstract_inverted_index.healthcare | 25 |
| abstract_inverted_index.illustrate | 99 |
| abstract_inverted_index.monitoring | 87 |
| abstract_inverted_index.post-trial | 75 |
| abstract_inverted_index.real-world | 106 |
| abstract_inverted_index.scientific | 69 |
| abstract_inverted_index.autonomous, | 121 |
| abstract_inverted_index.framework's | 101 |
| abstract_inverted_index.high-stakes | 24 |
| abstract_inverted_index.personalize | 126 |
| abstract_inverted_index.requirement | 45 |
| abstract_inverted_index.researchers | 94 |
| abstract_inverted_index.application, | 103 |
| abstract_inverted_index.participants | 12, 64, 130 |
| abstract_inverted_index.successfully | 118 |
| abstract_inverted_index.interventions | 128 |
| abstract_inverted_index.personalizing | 9 |
| abstract_inverted_index.reinforcement | 1 |
| abstract_inverted_index.pre-deployment | 83 |
| abstract_inverted_index.responsibility | 57 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 7 |
| citation_normalized_percentile |