Learning to predict with supporting evidence Article Swipe
Aniruddh Raghu
,
John V. Guttag
,
Katherine Young
,
Eugene Pomerantsev
,
Adrian V. Dalca
,
Collin M. Stultz
·
YOU?
·
· 2021
· Open Access
·
· DOI: https://doi.org/10.1145/3450439.3451869
YOU?
·
· 2021
· Open Access
·
· DOI: https://doi.org/10.1145/3450439.3451869
The impact of machine learning models on healthcare will depend on the degree of trust that healthcare professionals place in the predictions made by these models. In this paper, we present a method to provide individuals with clinical expertise with domain-relevant evidence about why a prediction should be trusted. We first design a probabilistic model that relates meaningful latent concepts to prediction targets and observed data. Inference of latent variables in this model corresponds to both making a prediction and providing supporting evidence for that prediction. We present a two-step process to efficiently approximate inference: (i) estimating model parameters using variational learning, and (ii) approximating maximum a posteriori estimation of latent variables in the model using a neural network, trained with an objective derived from the probabilistic model. We demonstrate the method on the task of predicting mortality risk for patients with cardiovascular disease. Specifically, using electrocardiogram and tabular data as input, we show that our approach provides appropriate domain-relevant supporting evidence for accurate predictions.
Related Topics
Concepts
Metadata
- Type
- article
- Language
- en
- Landing Page
- https://doi.org/10.1145/3450439.3451869
- https://dl.acm.org/doi/pdf/10.1145/3450439.3451869
- OA Status
- gold
- Cited By
- 3
- References
- 40
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W3137065808
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W3137065808Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.1145/3450439.3451869Digital Object Identifier
- Title
-
Learning to predict with supporting evidenceWork title
- Type
-
articleOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2021Year of publication
- Publication date
-
2021-03-23Full publication date if available
- Authors
-
Aniruddh Raghu, John V. Guttag, Katherine Young, Eugene Pomerantsev, Adrian V. Dalca, Collin M. StultzList of authors in order
- Landing page
-
https://doi.org/10.1145/3450439.3451869Publisher landing page
- PDF URL
-
https://dl.acm.org/doi/pdf/10.1145/3450439.3451869Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
goldOpen access status per OpenAlex
- OA URL
-
https://dl.acm.org/doi/pdf/10.1145/3450439.3451869Direct OA link when available
- Concepts
-
Computer scienceTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
3Total citation count in OpenAlex
- Citations by year (recent)
-
2023: 1, 2021: 2Per-year citation counts (last 5 years)
- References (count)
-
40Number of works referenced by this work
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W3137065808 |
|---|---|
| doi | https://doi.org/10.1145/3450439.3451869 |
| ids.doi | https://doi.org/10.1145/3450439.3451869 |
| ids.mag | 3137065808 |
| ids.openalex | https://openalex.org/W3137065808 |
| fwci | 0.42331073 |
| type | article |
| title | Learning to predict with supporting evidence |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | 104 |
| biblio.first_page | 95 |
| topics[0].id | https://openalex.org/T13702 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9993000030517578 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Machine Learning in Healthcare |
| topics[1].id | https://openalex.org/T12026 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9980000257492065 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Explainable Artificial Intelligence (XAI) |
| topics[2].id | https://openalex.org/T12574 |
| topics[2].field.id | https://openalex.org/fields/27 |
| topics[2].field.display_name | Medicine |
| topics[2].score | 0.9933000206947327 |
| topics[2].domain.id | https://openalex.org/domains/4 |
| topics[2].domain.display_name | Health Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/2714 |
| topics[2].subfield.display_name | Family Practice |
| topics[2].display_name | Clinical Reasoning and Diagnostic Skills |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C41008148 |
| concepts[0].level | 0 |
| concepts[0].score | 0.6658539772033691 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[0].display_name | Computer science |
| keywords[0].id | https://openalex.org/keywords/computer-science |
| keywords[0].score | 0.6658539772033691 |
| keywords[0].display_name | Computer science |
| language | en |
| locations[0].id | doi:10.1145/3450439.3451869 |
| locations[0].is_oa | True |
| locations[0].source | |
| locations[0].license | |
| locations[0].pdf_url | https://dl.acm.org/doi/pdf/10.1145/3450439.3451869 |
| locations[0].version | publishedVersion |
| locations[0].raw_type | proceedings-article |
| locations[0].license_id | |
| locations[0].is_accepted | True |
| locations[0].is_published | True |
| locations[0].raw_source_name | Proceedings of the Conference on Health, Inference, and Learning |
| locations[0].landing_page_url | https://doi.org/10.1145/3450439.3451869 |
| indexed_in | crossref |
| authorships[0].author.id | https://openalex.org/A5067494217 |
| authorships[0].author.orcid | |
| authorships[0].author.display_name | Aniruddh Raghu |
| authorships[0].countries | US |
| authorships[0].affiliations[0].institution_ids | https://openalex.org/I63966007 |
| authorships[0].affiliations[0].raw_affiliation_string | Massachusetts Institute of Technology |
| authorships[0].institutions[0].id | https://openalex.org/I63966007 |
| authorships[0].institutions[0].ror | https://ror.org/042nb2s44 |
| authorships[0].institutions[0].type | education |
| authorships[0].institutions[0].lineage | https://openalex.org/I63966007 |
| authorships[0].institutions[0].country_code | US |
| authorships[0].institutions[0].display_name | Massachusetts Institute of Technology |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Aniruddh Raghu |
| authorships[0].is_corresponding | False |
| authorships[0].raw_affiliation_strings | Massachusetts Institute of Technology |
| authorships[1].author.id | https://openalex.org/A5007282049 |
| authorships[1].author.orcid | https://orcid.org/0000-0003-0992-0906 |
| authorships[1].author.display_name | John V. Guttag |
| authorships[1].countries | US |
| authorships[1].affiliations[0].institution_ids | https://openalex.org/I63966007 |
| authorships[1].affiliations[0].raw_affiliation_string | Massachusetts Institute of Technology |
| authorships[1].institutions[0].id | https://openalex.org/I63966007 |
| authorships[1].institutions[0].ror | https://ror.org/042nb2s44 |
| authorships[1].institutions[0].type | education |
| authorships[1].institutions[0].lineage | https://openalex.org/I63966007 |
| authorships[1].institutions[0].country_code | US |
| authorships[1].institutions[0].display_name | Massachusetts Institute of Technology |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | John Guttag |
| authorships[1].is_corresponding | False |
| authorships[1].raw_affiliation_strings | Massachusetts Institute of Technology |
| authorships[2].author.id | https://openalex.org/A5015347146 |
| authorships[2].author.orcid | https://orcid.org/0000-0003-1808-5658 |
| authorships[2].author.display_name | Katherine Young |
| authorships[2].countries | US |
| authorships[2].affiliations[0].institution_ids | https://openalex.org/I63966007 |
| authorships[2].affiliations[0].raw_affiliation_string | Massachusetts Institute of Technology |
| authorships[2].institutions[0].id | https://openalex.org/I63966007 |
| authorships[2].institutions[0].ror | https://ror.org/042nb2s44 |
| authorships[2].institutions[0].type | education |
| authorships[2].institutions[0].lineage | https://openalex.org/I63966007 |
| authorships[2].institutions[0].country_code | US |
| authorships[2].institutions[0].display_name | Massachusetts Institute of Technology |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Katherine Young |
| authorships[2].is_corresponding | False |
| authorships[2].raw_affiliation_strings | Massachusetts Institute of Technology |
| authorships[3].author.id | https://openalex.org/A5054363576 |
| authorships[3].author.orcid | |
| authorships[3].author.display_name | Eugene Pomerantsev |
| authorships[3].countries | US |
| authorships[3].affiliations[0].institution_ids | https://openalex.org/I4210087915 |
| authorships[3].affiliations[0].raw_affiliation_string | Massachusetts General Hospital |
| authorships[3].institutions[0].id | https://openalex.org/I4210087915 |
| authorships[3].institutions[0].ror | https://ror.org/002pd6e78 |
| authorships[3].institutions[0].type | healthcare |
| authorships[3].institutions[0].lineage | https://openalex.org/I4210087915, https://openalex.org/I48633490 |
| authorships[3].institutions[0].country_code | US |
| authorships[3].institutions[0].display_name | Massachusetts General Hospital |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Eugene Pomerantsev |
| authorships[3].is_corresponding | False |
| authorships[3].raw_affiliation_strings | Massachusetts General Hospital |
| authorships[4].author.id | https://openalex.org/A5091409910 |
| authorships[4].author.orcid | https://orcid.org/0000-0002-8422-0136 |
| authorships[4].author.display_name | Adrian V. Dalca |
| authorships[4].countries | US |
| authorships[4].affiliations[0].institution_ids | https://openalex.org/I4210087915 |
| authorships[4].affiliations[0].raw_affiliation_string | Massachusetts General Hospital |
| authorships[4].institutions[0].id | https://openalex.org/I4210087915 |
| authorships[4].institutions[0].ror | https://ror.org/002pd6e78 |
| authorships[4].institutions[0].type | healthcare |
| authorships[4].institutions[0].lineage | https://openalex.org/I4210087915, https://openalex.org/I48633490 |
| authorships[4].institutions[0].country_code | US |
| authorships[4].institutions[0].display_name | Massachusetts General Hospital |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Adrian V. Dalca |
| authorships[4].is_corresponding | False |
| authorships[4].raw_affiliation_strings | Massachusetts General Hospital |
| authorships[5].author.id | https://openalex.org/A5024941370 |
| authorships[5].author.orcid | https://orcid.org/0000-0002-3415-242X |
| authorships[5].author.display_name | Collin M. Stultz |
| authorships[5].countries | US |
| authorships[5].affiliations[0].institution_ids | https://openalex.org/I4210087915 |
| authorships[5].affiliations[0].raw_affiliation_string | Massachusetts General Hospital |
| authorships[5].institutions[0].id | https://openalex.org/I4210087915 |
| authorships[5].institutions[0].ror | https://ror.org/002pd6e78 |
| authorships[5].institutions[0].type | healthcare |
| authorships[5].institutions[0].lineage | https://openalex.org/I4210087915, https://openalex.org/I48633490 |
| authorships[5].institutions[0].country_code | US |
| authorships[5].institutions[0].display_name | Massachusetts General Hospital |
| authorships[5].author_position | last |
| authorships[5].raw_author_name | Collin M. Stultz |
| authorships[5].is_corresponding | False |
| authorships[5].raw_affiliation_strings | Massachusetts General Hospital |
| has_content.pdf | True |
| has_content.grobid_xml | True |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://dl.acm.org/doi/pdf/10.1145/3450439.3451869 |
| open_access.oa_status | gold |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Learning to predict with supporting evidence |
| has_fulltext | True |
| is_retracted | False |
| updated_date | 2025-11-06T03:46:38.306776 |
| primary_topic.id | https://openalex.org/T13702 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9993000030517578 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Machine Learning in Healthcare |
| related_works | https://openalex.org/W2748952813, https://openalex.org/W2390279801, https://openalex.org/W2358668433, https://openalex.org/W2376932109, https://openalex.org/W2001405890, https://openalex.org/W2382290278, https://openalex.org/W2478288626, https://openalex.org/W2350741829, https://openalex.org/W2530322880, https://openalex.org/W1596801655 |
| cited_by_count | 3 |
| counts_by_year[0].year | 2023 |
| counts_by_year[0].cited_by_count | 1 |
| counts_by_year[1].year | 2021 |
| counts_by_year[1].cited_by_count | 2 |
| locations_count | 1 |
| best_oa_location.id | doi:10.1145/3450439.3451869 |
| best_oa_location.is_oa | True |
| best_oa_location.source | |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://dl.acm.org/doi/pdf/10.1145/3450439.3451869 |
| best_oa_location.version | publishedVersion |
| best_oa_location.raw_type | proceedings-article |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | True |
| best_oa_location.is_published | True |
| best_oa_location.raw_source_name | Proceedings of the Conference on Health, Inference, and Learning |
| best_oa_location.landing_page_url | https://doi.org/10.1145/3450439.3451869 |
| primary_location.id | doi:10.1145/3450439.3451869 |
| primary_location.is_oa | True |
| primary_location.source | |
| primary_location.license | |
| primary_location.pdf_url | https://dl.acm.org/doi/pdf/10.1145/3450439.3451869 |
| primary_location.version | publishedVersion |
| primary_location.raw_type | proceedings-article |
| primary_location.license_id | |
| primary_location.is_accepted | True |
| primary_location.is_published | True |
| primary_location.raw_source_name | Proceedings of the Conference on Health, Inference, and Learning |
| primary_location.landing_page_url | https://doi.org/10.1145/3450439.3451869 |
| publication_date | 2021-03-23 |
| publication_year | 2021 |
| referenced_works | https://openalex.org/W2034690624, https://openalex.org/W2918341242, https://openalex.org/W2979699329, https://openalex.org/W3007910004, https://openalex.org/W2960259221, https://openalex.org/W2796885425, https://openalex.org/W1994606570, https://openalex.org/W2155965977, https://openalex.org/W2484192585, https://openalex.org/W2963095307, https://openalex.org/W2018700684, https://openalex.org/W2979691446, https://openalex.org/W2062841580, https://openalex.org/W2127083673, https://openalex.org/W2280404143, https://openalex.org/W2283619910, https://openalex.org/W4254170669, https://openalex.org/W4206159607, https://openalex.org/W1849277567, https://openalex.org/W2970030610, https://openalex.org/W2133979711, https://openalex.org/W2949197630, https://openalex.org/W2178738676, https://openalex.org/W1909320841, https://openalex.org/W3104075323, https://openalex.org/W3141140719, https://openalex.org/W2951885001, https://openalex.org/W2964180856, https://openalex.org/W3104831984, https://openalex.org/W2618851150, https://openalex.org/W1974943571, https://openalex.org/W2963483561, https://openalex.org/W1487357205, https://openalex.org/W2944817757, https://openalex.org/W4285719527, https://openalex.org/W3002682974, https://openalex.org/W1502922572, https://openalex.org/W2183507150, https://openalex.org/W2913705661, https://openalex.org/W2944218525 |
| referenced_works_count | 40 |
| abstract_inverted_index.a | 31, 44, 52, 77, 88, 106, 116 |
| abstract_inverted_index.In | 26 |
| abstract_inverted_index.We | 49, 86, 128 |
| abstract_inverted_index.an | 121 |
| abstract_inverted_index.as | 150 |
| abstract_inverted_index.be | 47 |
| abstract_inverted_index.by | 23 |
| abstract_inverted_index.in | 19, 70, 112 |
| abstract_inverted_index.of | 2, 13, 67, 109, 135 |
| abstract_inverted_index.on | 6, 10, 132 |
| abstract_inverted_index.to | 33, 60, 74, 91 |
| abstract_inverted_index.we | 29, 152 |
| abstract_inverted_index.(i) | 95 |
| abstract_inverted_index.The | 0 |
| abstract_inverted_index.and | 63, 79, 102, 147 |
| abstract_inverted_index.for | 83, 139, 162 |
| abstract_inverted_index.our | 155 |
| abstract_inverted_index.the | 11, 20, 113, 125, 130, 133 |
| abstract_inverted_index.why | 43 |
| abstract_inverted_index.(ii) | 103 |
| abstract_inverted_index.both | 75 |
| abstract_inverted_index.data | 149 |
| abstract_inverted_index.from | 124 |
| abstract_inverted_index.made | 22 |
| abstract_inverted_index.risk | 138 |
| abstract_inverted_index.show | 153 |
| abstract_inverted_index.task | 134 |
| abstract_inverted_index.that | 15, 55, 84, 154 |
| abstract_inverted_index.this | 27, 71 |
| abstract_inverted_index.will | 8 |
| abstract_inverted_index.with | 36, 39, 120, 141 |
| abstract_inverted_index.about | 42 |
| abstract_inverted_index.data. | 65 |
| abstract_inverted_index.first | 50 |
| abstract_inverted_index.model | 54, 72, 97, 114 |
| abstract_inverted_index.place | 18 |
| abstract_inverted_index.these | 24 |
| abstract_inverted_index.trust | 14 |
| abstract_inverted_index.using | 99, 115, 145 |
| abstract_inverted_index.degree | 12 |
| abstract_inverted_index.depend | 9 |
| abstract_inverted_index.design | 51 |
| abstract_inverted_index.impact | 1 |
| abstract_inverted_index.input, | 151 |
| abstract_inverted_index.latent | 58, 68, 110 |
| abstract_inverted_index.making | 76 |
| abstract_inverted_index.method | 32, 131 |
| abstract_inverted_index.model. | 127 |
| abstract_inverted_index.models | 5 |
| abstract_inverted_index.neural | 117 |
| abstract_inverted_index.paper, | 28 |
| abstract_inverted_index.should | 46 |
| abstract_inverted_index.derived | 123 |
| abstract_inverted_index.machine | 3 |
| abstract_inverted_index.maximum | 105 |
| abstract_inverted_index.models. | 25 |
| abstract_inverted_index.present | 30, 87 |
| abstract_inverted_index.process | 90 |
| abstract_inverted_index.provide | 34 |
| abstract_inverted_index.relates | 56 |
| abstract_inverted_index.tabular | 148 |
| abstract_inverted_index.targets | 62 |
| abstract_inverted_index.trained | 119 |
| abstract_inverted_index.accurate | 163 |
| abstract_inverted_index.approach | 156 |
| abstract_inverted_index.clinical | 37 |
| abstract_inverted_index.concepts | 59 |
| abstract_inverted_index.disease. | 143 |
| abstract_inverted_index.evidence | 41, 82, 161 |
| abstract_inverted_index.learning | 4 |
| abstract_inverted_index.network, | 118 |
| abstract_inverted_index.observed | 64 |
| abstract_inverted_index.patients | 140 |
| abstract_inverted_index.provides | 157 |
| abstract_inverted_index.trusted. | 48 |
| abstract_inverted_index.two-step | 89 |
| abstract_inverted_index.Inference | 66 |
| abstract_inverted_index.expertise | 38 |
| abstract_inverted_index.learning, | 101 |
| abstract_inverted_index.mortality | 137 |
| abstract_inverted_index.objective | 122 |
| abstract_inverted_index.providing | 80 |
| abstract_inverted_index.variables | 69, 111 |
| abstract_inverted_index.estimating | 96 |
| abstract_inverted_index.estimation | 108 |
| abstract_inverted_index.healthcare | 7, 16 |
| abstract_inverted_index.inference: | 94 |
| abstract_inverted_index.meaningful | 57 |
| abstract_inverted_index.parameters | 98 |
| abstract_inverted_index.posteriori | 107 |
| abstract_inverted_index.predicting | 136 |
| abstract_inverted_index.prediction | 45, 61, 78 |
| abstract_inverted_index.supporting | 81, 160 |
| abstract_inverted_index.appropriate | 158 |
| abstract_inverted_index.approximate | 93 |
| abstract_inverted_index.corresponds | 73 |
| abstract_inverted_index.demonstrate | 129 |
| abstract_inverted_index.efficiently | 92 |
| abstract_inverted_index.individuals | 35 |
| abstract_inverted_index.prediction. | 85 |
| abstract_inverted_index.predictions | 21 |
| abstract_inverted_index.variational | 100 |
| abstract_inverted_index.predictions. | 164 |
| abstract_inverted_index.Specifically, | 144 |
| abstract_inverted_index.approximating | 104 |
| abstract_inverted_index.probabilistic | 53, 126 |
| abstract_inverted_index.professionals | 17 |
| abstract_inverted_index.cardiovascular | 142 |
| abstract_inverted_index.domain-relevant | 40, 159 |
| abstract_inverted_index.electrocardiogram | 146 |
| cited_by_percentile_year.max | 95 |
| cited_by_percentile_year.min | 89 |
| countries_distinct_count | 1 |
| institutions_distinct_count | 6 |
| sustainable_development_goals[0].id | https://metadata.un.org/sdg/3 |
| sustainable_development_goals[0].score | 0.8500000238418579 |
| sustainable_development_goals[0].display_name | Good health and well-being |
| citation_normalized_percentile.value | 0.67877525 |
| citation_normalized_percentile.is_in_top_1_percent | False |
| citation_normalized_percentile.is_in_top_10_percent | False |