Improving Methods of Identifying Anaphylaxis for Medical Product Safety Surveillance Using Natural Language Processing and Machine Learning Article Swipe
David Carrell
,
Susan Gruber
,
James S. Floyd
,
Maralyssa Bann
,
Kara L. Cushing‐Haugen
,
Ron L. Johnson
,
Vina Graham
,
David Cronkite
,
Brian Hazlehurst
,
Andrew H. Felcher
,
Cosmin A. Bejan
,
Adee Kennedy
,
Mayura Shinde
,
Sara Karami
,
Yong Ma
,
Danijela Stojanović
,
Yueqin Zhao
,
Robert Ball
,
Jennifer C. Nelson
·
YOU?
·
· 2022
· Open Access
·
· DOI: https://doi.org/10.1093/aje/kwac182
YOU?
·
· 2022
· Open Access
·
· DOI: https://doi.org/10.1093/aje/kwac182
We sought to determine whether machine learning and natural language processing (NLP) applied to electronic medical records could improve performance of automated health-care claims-based algorithms to identify anaphylaxis events using data on 516 patients with outpatient, emergency department, or inpatient anaphylaxis diagnosis codes during 2015–2019 in 2 integrated health-care institutions in the Northwest United States. We used one site’s manually reviewed gold-standard outcomes data for model development and the other’s for external validation based on cross-validated area under the receiver operating characteristic curve (AUC), positive predictive value (PPV), and sensitivity. In the development site 154 (64%) of 239 potential events met adjudication criteria for anaphylaxis compared with 180 (65%) of 277 in the validation site. Logistic regression models using only structured claims data achieved a cross-validated AUC of 0.58 (95% CI: 0.54, 0.63). Machine learning improved cross-validated AUC to 0.62 (0.58, 0.66); incorporating NLP-derived covariates further increased cross-validated AUCs to 0.70 (0.66, 0.75) in development and 0.67 (0.63, 0.71) in external validation data. A classification threshold with cross-validated PPV of 79% and cross-validated sensitivity of 66% in development data had cross-validated PPV of 78% and cross-validated sensitivity of 56% in external data. Machine learning and NLP-derived data improved identification of validated anaphylaxis events.
Related Topics
Concepts
Medicine
Receiver operating characteristic
Adjudication
Cross-validation
Emergency department
Machine learning
Artificial intelligence
Logistic regression
Random forest
Area under the curve
Gold standard (test)
Acute care
Sensitivity (control systems)
Health care
Emergency medicine
Medical emergency
Computer science
Internal medicine
Engineering
Electronic engineering
Psychiatry
Political science
Law
Economic growth
Economics
Metadata
- Type
- article
- Language
- en
- Landing Page
- https://doi.org/10.1093/aje/kwac182
- https://academic.oup.com/aje/article-pdf/192/2/283/49088177/kwac182.pdf
- OA Status
- bronze
- Cited By
- 26
- References
- 67
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4308184843
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4308184843Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.1093/aje/kwac182Digital Object Identifier
- Title
-
Improving Methods of Identifying Anaphylaxis for Medical Product Safety Surveillance Using Natural Language Processing and Machine LearningWork title
- Type
-
articleOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2022Year of publication
- Publication date
-
2022-11-02Full publication date if available
- Authors
-
David Carrell, Susan Gruber, James S. Floyd, Maralyssa Bann, Kara L. Cushing‐Haugen, Ron L. Johnson, Vina Graham, David Cronkite, Brian Hazlehurst, Andrew H. Felcher, Cosmin A. Bejan, Adee Kennedy, Mayura Shinde, Sara Karami, Yong Ma, Danijela Stojanović, Yueqin Zhao, Robert Ball, Jennifer C. NelsonList of authors in order
- Landing page
-
https://doi.org/10.1093/aje/kwac182Publisher landing page
- PDF URL
-
https://academic.oup.com/aje/article-pdf/192/2/283/49088177/kwac182.pdfDirect link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
bronzeOpen access status per OpenAlex
- OA URL
-
https://academic.oup.com/aje/article-pdf/192/2/283/49088177/kwac182.pdfDirect OA link when available
- Concepts
-
Medicine, Receiver operating characteristic, Adjudication, Cross-validation, Emergency department, Machine learning, Artificial intelligence, Logistic regression, Random forest, Area under the curve, Gold standard (test), Acute care, Sensitivity (control systems), Health care, Emergency medicine, Medical emergency, Computer science, Internal medicine, Engineering, Electronic engineering, Psychiatry, Political science, Law, Economic growth, EconomicsTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
26Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 8, 2024: 12, 2023: 6Per-year citation counts (last 5 years)
- References (count)
-
67Number of works referenced by this work
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4308184843 |
|---|---|
| doi | https://doi.org/10.1093/aje/kwac182 |
| ids.doi | https://doi.org/10.1093/aje/kwac182 |
| ids.pmid | https://pubmed.ncbi.nlm.nih.gov/36331289 |
| ids.openalex | https://openalex.org/W4308184843 |
| fwci | 5.16352974 |
| mesh[0].qualifier_ui | |
| mesh[0].descriptor_ui | D006801 |
| mesh[0].is_major_topic | False |
| mesh[0].qualifier_name | |
| mesh[0].descriptor_name | Humans |
| mesh[1].qualifier_ui | |
| mesh[1].descriptor_ui | D009323 |
| mesh[1].is_major_topic | True |
| mesh[1].qualifier_name | |
| mesh[1].descriptor_name | Natural Language Processing |
| mesh[2].qualifier_ui | Q000175 |
| mesh[2].descriptor_ui | D000707 |
| mesh[2].is_major_topic | True |
| mesh[2].qualifier_name | diagnosis |
| mesh[2].descriptor_name | Anaphylaxis |
| mesh[3].qualifier_ui | Q000453 |
| mesh[3].descriptor_ui | D000707 |
| mesh[3].is_major_topic | True |
| mesh[3].qualifier_name | epidemiology |
| mesh[3].descriptor_name | Anaphylaxis |
| mesh[4].qualifier_ui | |
| mesh[4].descriptor_ui | D000069550 |
| mesh[4].is_major_topic | False |
| mesh[4].qualifier_name | |
| mesh[4].descriptor_name | Machine Learning |
| mesh[5].qualifier_ui | |
| mesh[5].descriptor_ui | D000465 |
| mesh[5].is_major_topic | False |
| mesh[5].qualifier_name | |
| mesh[5].descriptor_name | Algorithms |
| mesh[6].qualifier_ui | |
| mesh[6].descriptor_ui | D004636 |
| mesh[6].is_major_topic | False |
| mesh[6].qualifier_name | |
| mesh[6].descriptor_name | Emergency Service, Hospital |
| mesh[7].qualifier_ui | |
| mesh[7].descriptor_ui | D057286 |
| mesh[7].is_major_topic | False |
| mesh[7].qualifier_name | |
| mesh[7].descriptor_name | Electronic Health Records |
| mesh[8].qualifier_ui | |
| mesh[8].descriptor_ui | D006801 |
| mesh[8].is_major_topic | False |
| mesh[8].qualifier_name | |
| mesh[8].descriptor_name | Humans |
| mesh[9].qualifier_ui | |
| mesh[9].descriptor_ui | D009323 |
| mesh[9].is_major_topic | True |
| mesh[9].qualifier_name | |
| mesh[9].descriptor_name | Natural Language Processing |
| mesh[10].qualifier_ui | Q000175 |
| mesh[10].descriptor_ui | D000707 |
| mesh[10].is_major_topic | True |
| mesh[10].qualifier_name | diagnosis |
| mesh[10].descriptor_name | Anaphylaxis |
| mesh[11].qualifier_ui | Q000453 |
| mesh[11].descriptor_ui | D000707 |
| mesh[11].is_major_topic | True |
| mesh[11].qualifier_name | epidemiology |
| mesh[11].descriptor_name | Anaphylaxis |
| mesh[12].qualifier_ui | |
| mesh[12].descriptor_ui | D000069550 |
| mesh[12].is_major_topic | False |
| mesh[12].qualifier_name | |
| mesh[12].descriptor_name | Machine Learning |
| mesh[13].qualifier_ui | |
| mesh[13].descriptor_ui | D000465 |
| mesh[13].is_major_topic | False |
| mesh[13].qualifier_name | |
| mesh[13].descriptor_name | Algorithms |
| mesh[14].qualifier_ui | |
| mesh[14].descriptor_ui | D004636 |
| mesh[14].is_major_topic | False |
| mesh[14].qualifier_name | |
| mesh[14].descriptor_name | Emergency Service, Hospital |
| mesh[15].qualifier_ui | |
| mesh[15].descriptor_ui | D057286 |
| mesh[15].is_major_topic | False |
| mesh[15].qualifier_name | |
| mesh[15].descriptor_name | Electronic Health Records |
| mesh[16].qualifier_ui | |
| mesh[16].descriptor_ui | D006801 |
| mesh[16].is_major_topic | False |
| mesh[16].qualifier_name | |
| mesh[16].descriptor_name | Humans |
| mesh[17].qualifier_ui | |
| mesh[17].descriptor_ui | D009323 |
| mesh[17].is_major_topic | True |
| mesh[17].qualifier_name | |
| mesh[17].descriptor_name | Natural Language Processing |
| mesh[18].qualifier_ui | Q000175 |
| mesh[18].descriptor_ui | D000707 |
| mesh[18].is_major_topic | True |
| mesh[18].qualifier_name | diagnosis |
| mesh[18].descriptor_name | Anaphylaxis |
| mesh[19].qualifier_ui | Q000453 |
| mesh[19].descriptor_ui | D000707 |
| mesh[19].is_major_topic | True |
| mesh[19].qualifier_name | epidemiology |
| mesh[19].descriptor_name | Anaphylaxis |
| mesh[20].qualifier_ui | |
| mesh[20].descriptor_ui | D000069550 |
| mesh[20].is_major_topic | False |
| mesh[20].qualifier_name | |
| mesh[20].descriptor_name | Machine Learning |
| mesh[21].qualifier_ui | |
| mesh[21].descriptor_ui | D000465 |
| mesh[21].is_major_topic | False |
| mesh[21].qualifier_name | |
| mesh[21].descriptor_name | Algorithms |
| mesh[22].qualifier_ui | |
| mesh[22].descriptor_ui | D004636 |
| mesh[22].is_major_topic | False |
| mesh[22].qualifier_name | |
| mesh[22].descriptor_name | Emergency Service, Hospital |
| mesh[23].qualifier_ui | |
| mesh[23].descriptor_ui | D057286 |
| mesh[23].is_major_topic | False |
| mesh[23].qualifier_name | |
| mesh[23].descriptor_name | Electronic Health Records |
| mesh[24].qualifier_ui | |
| mesh[24].descriptor_ui | D006801 |
| mesh[24].is_major_topic | False |
| mesh[24].qualifier_name | |
| mesh[24].descriptor_name | Humans |
| mesh[25].qualifier_ui | |
| mesh[25].descriptor_ui | D009323 |
| mesh[25].is_major_topic | True |
| mesh[25].qualifier_name | |
| mesh[25].descriptor_name | Natural Language Processing |
| mesh[26].qualifier_ui | Q000175 |
| mesh[26].descriptor_ui | D000707 |
| mesh[26].is_major_topic | True |
| mesh[26].qualifier_name | diagnosis |
| mesh[26].descriptor_name | Anaphylaxis |
| mesh[27].qualifier_ui | Q000453 |
| mesh[27].descriptor_ui | D000707 |
| mesh[27].is_major_topic | True |
| mesh[27].qualifier_name | epidemiology |
| mesh[27].descriptor_name | Anaphylaxis |
| mesh[28].qualifier_ui | |
| mesh[28].descriptor_ui | D000069550 |
| mesh[28].is_major_topic | False |
| mesh[28].qualifier_name | |
| mesh[28].descriptor_name | Machine Learning |
| mesh[29].qualifier_ui | |
| mesh[29].descriptor_ui | D000465 |
| mesh[29].is_major_topic | False |
| mesh[29].qualifier_name | |
| mesh[29].descriptor_name | Algorithms |
| mesh[30].qualifier_ui | |
| mesh[30].descriptor_ui | D004636 |
| mesh[30].is_major_topic | False |
| mesh[30].qualifier_name | |
| mesh[30].descriptor_name | Emergency Service, Hospital |
| mesh[31].qualifier_ui | |
| mesh[31].descriptor_ui | D057286 |
| mesh[31].is_major_topic | False |
| mesh[31].qualifier_name | |
| mesh[31].descriptor_name | Electronic Health Records |
| type | article |
| title | Improving Methods of Identifying Anaphylaxis for Medical Product Safety Surveillance Using Natural Language Processing and Machine Learning |
| biblio.issue | 2 |
| biblio.volume | 192 |
| biblio.last_page | 295 |
| biblio.first_page | 283 |
| topics[0].id | https://openalex.org/T10634 |
| topics[0].field.id | https://openalex.org/fields/27 |
| topics[0].field.display_name | Medicine |
| topics[0].score | 0.9993000030517578 |
| topics[0].domain.id | https://openalex.org/domains/4 |
| topics[0].domain.display_name | Health Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/2723 |
| topics[0].subfield.display_name | Immunology and Allergy |
| topics[0].display_name | Food Allergy and Anaphylaxis Research |
| topics[1].id | https://openalex.org/T10877 |
| topics[1].field.id | https://openalex.org/fields/27 |
| topics[1].field.display_name | Medicine |
| topics[1].score | 0.9937999844551086 |
| topics[1].domain.id | https://openalex.org/domains/4 |
| topics[1].domain.display_name | Health Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/2723 |
| topics[1].subfield.display_name | Immunology and Allergy |
| topics[1].display_name | Allergic Rhinitis and Sensitization |
| topics[2].id | https://openalex.org/T10051 |
| topics[2].field.id | https://openalex.org/fields/27 |
| topics[2].field.display_name | Medicine |
| topics[2].score | 0.9929999709129333 |
| topics[2].domain.id | https://openalex.org/domains/4 |
| topics[2].domain.display_name | Health Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/2737 |
| topics[2].subfield.display_name | Physiology |
| topics[2].display_name | Asthma and respiratory diseases |
| is_xpac | False |
| apc_list.value | 4450 |
| apc_list.currency | USD |
| apc_list.value_usd | 4450 |
| apc_paid | |
| concepts[0].id | https://openalex.org/C71924100 |
| concepts[0].level | 0 |
| concepts[0].score | 0.7216527462005615 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q11190 |
| concepts[0].display_name | Medicine |
| concepts[1].id | https://openalex.org/C58471807 |
| concepts[1].level | 2 |
| concepts[1].score | 0.6696189045906067 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q327120 |
| concepts[1].display_name | Receiver operating characteristic |
| concepts[2].id | https://openalex.org/C204434341 |
| concepts[2].level | 2 |
| concepts[2].score | 0.6156911849975586 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q357789 |
| concepts[2].display_name | Adjudication |
| concepts[3].id | https://openalex.org/C27181475 |
| concepts[3].level | 2 |
| concepts[3].score | 0.6141530871391296 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q541014 |
| concepts[3].display_name | Cross-validation |
| concepts[4].id | https://openalex.org/C2780724011 |
| concepts[4].level | 2 |
| concepts[4].score | 0.5623398423194885 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q1295316 |
| concepts[4].display_name | Emergency department |
| concepts[5].id | https://openalex.org/C119857082 |
| concepts[5].level | 1 |
| concepts[5].score | 0.5621964335441589 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q2539 |
| concepts[5].display_name | Machine learning |
| concepts[6].id | https://openalex.org/C154945302 |
| concepts[6].level | 1 |
| concepts[6].score | 0.5608813166618347 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[6].display_name | Artificial intelligence |
| concepts[7].id | https://openalex.org/C151956035 |
| concepts[7].level | 2 |
| concepts[7].score | 0.5571447014808655 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q1132755 |
| concepts[7].display_name | Logistic regression |
| concepts[8].id | https://openalex.org/C169258074 |
| concepts[8].level | 2 |
| concepts[8].score | 0.46088242530822754 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q245748 |
| concepts[8].display_name | Random forest |
| concepts[9].id | https://openalex.org/C76318530 |
| concepts[9].level | 2 |
| concepts[9].score | 0.4591379463672638 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q16833590 |
| concepts[9].display_name | Area under the curve |
| concepts[10].id | https://openalex.org/C40993552 |
| concepts[10].level | 2 |
| concepts[10].score | 0.44375649094581604 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q514654 |
| concepts[10].display_name | Gold standard (test) |
| concepts[11].id | https://openalex.org/C2781385661 |
| concepts[11].level | 3 |
| concepts[11].score | 0.4418397545814514 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q4677918 |
| concepts[11].display_name | Acute care |
| concepts[12].id | https://openalex.org/C21200559 |
| concepts[12].level | 2 |
| concepts[12].score | 0.44182753562927246 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q7451068 |
| concepts[12].display_name | Sensitivity (control systems) |
| concepts[13].id | https://openalex.org/C160735492 |
| concepts[13].level | 2 |
| concepts[13].score | 0.39573973417282104 |
| concepts[13].wikidata | https://www.wikidata.org/wiki/Q31207 |
| concepts[13].display_name | Health care |
| concepts[14].id | https://openalex.org/C194828623 |
| concepts[14].level | 1 |
| concepts[14].score | 0.3890921473503113 |
| concepts[14].wikidata | https://www.wikidata.org/wiki/Q2861470 |
| concepts[14].display_name | Emergency medicine |
| concepts[15].id | https://openalex.org/C545542383 |
| concepts[15].level | 1 |
| concepts[15].score | 0.35907524824142456 |
| concepts[15].wikidata | https://www.wikidata.org/wiki/Q2751242 |
| concepts[15].display_name | Medical emergency |
| concepts[16].id | https://openalex.org/C41008148 |
| concepts[16].level | 0 |
| concepts[16].score | 0.33641132712364197 |
| concepts[16].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[16].display_name | Computer science |
| concepts[17].id | https://openalex.org/C126322002 |
| concepts[17].level | 1 |
| concepts[17].score | 0.18139952421188354 |
| concepts[17].wikidata | https://www.wikidata.org/wiki/Q11180 |
| concepts[17].display_name | Internal medicine |
| concepts[18].id | https://openalex.org/C127413603 |
| concepts[18].level | 0 |
| concepts[18].score | 0.10169681906700134 |
| concepts[18].wikidata | https://www.wikidata.org/wiki/Q11023 |
| concepts[18].display_name | Engineering |
| concepts[19].id | https://openalex.org/C24326235 |
| concepts[19].level | 1 |
| concepts[19].score | 0.0 |
| concepts[19].wikidata | https://www.wikidata.org/wiki/Q126095 |
| concepts[19].display_name | Electronic engineering |
| concepts[20].id | https://openalex.org/C118552586 |
| concepts[20].level | 1 |
| concepts[20].score | 0.0 |
| concepts[20].wikidata | https://www.wikidata.org/wiki/Q7867 |
| concepts[20].display_name | Psychiatry |
| concepts[21].id | https://openalex.org/C17744445 |
| concepts[21].level | 0 |
| concepts[21].score | 0.0 |
| concepts[21].wikidata | https://www.wikidata.org/wiki/Q36442 |
| concepts[21].display_name | Political science |
| concepts[22].id | https://openalex.org/C199539241 |
| concepts[22].level | 1 |
| concepts[22].score | 0.0 |
| concepts[22].wikidata | https://www.wikidata.org/wiki/Q7748 |
| concepts[22].display_name | Law |
| concepts[23].id | https://openalex.org/C50522688 |
| concepts[23].level | 1 |
| concepts[23].score | 0.0 |
| concepts[23].wikidata | https://www.wikidata.org/wiki/Q189833 |
| concepts[23].display_name | Economic growth |
| concepts[24].id | https://openalex.org/C162324750 |
| concepts[24].level | 0 |
| concepts[24].score | 0.0 |
| concepts[24].wikidata | https://www.wikidata.org/wiki/Q8134 |
| concepts[24].display_name | Economics |
| keywords[0].id | https://openalex.org/keywords/medicine |
| keywords[0].score | 0.7216527462005615 |
| keywords[0].display_name | Medicine |
| keywords[1].id | https://openalex.org/keywords/receiver-operating-characteristic |
| keywords[1].score | 0.6696189045906067 |
| keywords[1].display_name | Receiver operating characteristic |
| keywords[2].id | https://openalex.org/keywords/adjudication |
| keywords[2].score | 0.6156911849975586 |
| keywords[2].display_name | Adjudication |
| keywords[3].id | https://openalex.org/keywords/cross-validation |
| keywords[3].score | 0.6141530871391296 |
| keywords[3].display_name | Cross-validation |
| keywords[4].id | https://openalex.org/keywords/emergency-department |
| keywords[4].score | 0.5623398423194885 |
| keywords[4].display_name | Emergency department |
| keywords[5].id | https://openalex.org/keywords/machine-learning |
| keywords[5].score | 0.5621964335441589 |
| keywords[5].display_name | Machine learning |
| keywords[6].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[6].score | 0.5608813166618347 |
| keywords[6].display_name | Artificial intelligence |
| keywords[7].id | https://openalex.org/keywords/logistic-regression |
| keywords[7].score | 0.5571447014808655 |
| keywords[7].display_name | Logistic regression |
| keywords[8].id | https://openalex.org/keywords/random-forest |
| keywords[8].score | 0.46088242530822754 |
| keywords[8].display_name | Random forest |
| keywords[9].id | https://openalex.org/keywords/area-under-the-curve |
| keywords[9].score | 0.4591379463672638 |
| keywords[9].display_name | Area under the curve |
| keywords[10].id | https://openalex.org/keywords/gold-standard |
| keywords[10].score | 0.44375649094581604 |
| keywords[10].display_name | Gold standard (test) |
| keywords[11].id | https://openalex.org/keywords/acute-care |
| keywords[11].score | 0.4418397545814514 |
| keywords[11].display_name | Acute care |
| keywords[12].id | https://openalex.org/keywords/sensitivity |
| keywords[12].score | 0.44182753562927246 |
| keywords[12].display_name | Sensitivity (control systems) |
| keywords[13].id | https://openalex.org/keywords/health-care |
| keywords[13].score | 0.39573973417282104 |
| keywords[13].display_name | Health care |
| keywords[14].id | https://openalex.org/keywords/emergency-medicine |
| keywords[14].score | 0.3890921473503113 |
| keywords[14].display_name | Emergency medicine |
| keywords[15].id | https://openalex.org/keywords/medical-emergency |
| keywords[15].score | 0.35907524824142456 |
| keywords[15].display_name | Medical emergency |
| keywords[16].id | https://openalex.org/keywords/computer-science |
| keywords[16].score | 0.33641132712364197 |
| keywords[16].display_name | Computer science |
| keywords[17].id | https://openalex.org/keywords/internal-medicine |
| keywords[17].score | 0.18139952421188354 |
| keywords[17].display_name | Internal medicine |
| keywords[18].id | https://openalex.org/keywords/engineering |
| keywords[18].score | 0.10169681906700134 |
| keywords[18].display_name | Engineering |
| language | en |
| locations[0].id | doi:10.1093/aje/kwac182 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S170967050 |
| locations[0].source.issn | 0002-9262, 1476-6256 |
| locations[0].source.type | journal |
| locations[0].source.is_oa | False |
| locations[0].source.issn_l | 0002-9262 |
| locations[0].source.is_core | True |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | American Journal of Epidemiology |
| locations[0].source.host_organization | https://openalex.org/P4310311648 |
| locations[0].source.host_organization_name | Oxford University Press |
| locations[0].source.host_organization_lineage | https://openalex.org/P4310311648, https://openalex.org/P4310311647 |
| locations[0].source.host_organization_lineage_names | Oxford University Press, University of Oxford |
| locations[0].license | |
| locations[0].pdf_url | https://academic.oup.com/aje/article-pdf/192/2/283/49088177/kwac182.pdf |
| locations[0].version | publishedVersion |
| locations[0].raw_type | journal-article |
| locations[0].license_id | |
| locations[0].is_accepted | True |
| locations[0].is_published | True |
| locations[0].raw_source_name | American Journal of Epidemiology |
| locations[0].landing_page_url | https://doi.org/10.1093/aje/kwac182 |
| locations[1].id | pmid:36331289 |
| locations[1].is_oa | False |
| locations[1].source.id | https://openalex.org/S4306525036 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | False |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | PubMed |
| locations[1].source.host_organization | https://openalex.org/I1299303238 |
| locations[1].source.host_organization_name | National Institutes of Health |
| locations[1].source.host_organization_lineage | https://openalex.org/I1299303238 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | publishedVersion |
| locations[1].raw_type | |
| locations[1].license_id | |
| locations[1].is_accepted | True |
| locations[1].is_published | True |
| locations[1].raw_source_name | American journal of epidemiology |
| locations[1].landing_page_url | https://pubmed.ncbi.nlm.nih.gov/36331289 |
| locations[2].id | pmh:oai:pubmedcentral.nih.gov:9896464 |
| locations[2].is_oa | True |
| locations[2].source.id | https://openalex.org/S2764455111 |
| locations[2].source.issn | |
| locations[2].source.type | repository |
| locations[2].source.is_oa | False |
| locations[2].source.issn_l | |
| locations[2].source.is_core | False |
| locations[2].source.is_in_doaj | False |
| locations[2].source.display_name | PubMed Central |
| locations[2].source.host_organization | https://openalex.org/I1299303238 |
| locations[2].source.host_organization_name | National Institutes of Health |
| locations[2].source.host_organization_lineage | https://openalex.org/I1299303238 |
| locations[2].license | other-oa |
| locations[2].pdf_url | |
| locations[2].version | submittedVersion |
| locations[2].raw_type | Text |
| locations[2].license_id | https://openalex.org/licenses/other-oa |
| locations[2].is_accepted | False |
| locations[2].is_published | False |
| locations[2].raw_source_name | Am J Epidemiol |
| locations[2].landing_page_url | https://www.ncbi.nlm.nih.gov/pmc/articles/9896464 |
| indexed_in | crossref, pubmed |
| authorships[0].author.id | https://openalex.org/A5041212198 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-8471-0928 |
| authorships[0].author.display_name | David Carrell |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | David S Carrell |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5048956065 |
| authorships[1].author.orcid | https://orcid.org/0000-0003-0127-0099 |
| authorships[1].author.display_name | Susan Gruber |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Susan Gruber |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5060145270 |
| authorships[2].author.orcid | https://orcid.org/0000-0002-0365-9041 |
| authorships[2].author.display_name | James S. Floyd |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | James S Floyd |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5061109493 |
| authorships[3].author.orcid | https://orcid.org/0000-0002-5893-950X |
| authorships[3].author.display_name | Maralyssa Bann |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Maralyssa A Bann |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5023546258 |
| authorships[4].author.orcid | https://orcid.org/0000-0003-2993-0191 |
| authorships[4].author.display_name | Kara L. Cushing‐Haugen |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Kara L Cushing-Haugen |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5045855600 |
| authorships[5].author.orcid | https://orcid.org/0000-0002-4163-577X |
| authorships[5].author.display_name | Ron L. Johnson |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Ron L Johnson |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5023818336 |
| authorships[6].author.orcid | |
| authorships[6].author.display_name | Vina Graham |
| authorships[6].author_position | middle |
| authorships[6].raw_author_name | Vina Graham |
| authorships[6].is_corresponding | False |
| authorships[7].author.id | https://openalex.org/A5054315152 |
| authorships[7].author.orcid | https://orcid.org/0000-0002-2797-5102 |
| authorships[7].author.display_name | David Cronkite |
| authorships[7].author_position | middle |
| authorships[7].raw_author_name | David J Cronkite |
| authorships[7].is_corresponding | False |
| authorships[8].author.id | https://openalex.org/A5039253208 |
| authorships[8].author.orcid | https://orcid.org/0000-0001-9365-3964 |
| authorships[8].author.display_name | Brian Hazlehurst |
| authorships[8].author_position | middle |
| authorships[8].raw_author_name | Brian L Hazlehurst |
| authorships[8].is_corresponding | False |
| authorships[9].author.id | https://openalex.org/A5009328822 |
| authorships[9].author.orcid | |
| authorships[9].author.display_name | Andrew H. Felcher |
| authorships[9].author_position | middle |
| authorships[9].raw_author_name | Andrew H Felcher |
| authorships[9].is_corresponding | False |
| authorships[10].author.id | https://openalex.org/A5054117089 |
| authorships[10].author.orcid | https://orcid.org/0000-0001-5107-0584 |
| authorships[10].author.display_name | Cosmin A. Bejan |
| authorships[10].author_position | middle |
| authorships[10].raw_author_name | Cosmin A Bejan |
| authorships[10].is_corresponding | False |
| authorships[11].author.id | https://openalex.org/A5083755872 |
| authorships[11].author.orcid | |
| authorships[11].author.display_name | Adee Kennedy |
| authorships[11].author_position | middle |
| authorships[11].raw_author_name | Adee Kennedy |
| authorships[11].is_corresponding | False |
| authorships[12].author.id | https://openalex.org/A5103864914 |
| authorships[12].author.orcid | |
| authorships[12].author.display_name | Mayura Shinde |
| authorships[12].author_position | middle |
| authorships[12].raw_author_name | Mayura U Shinde |
| authorships[12].is_corresponding | False |
| authorships[13].author.id | https://openalex.org/A5020712297 |
| authorships[13].author.orcid | https://orcid.org/0000-0003-0107-7707 |
| authorships[13].author.display_name | Sara Karami |
| authorships[13].author_position | middle |
| authorships[13].raw_author_name | Sara Karami |
| authorships[13].is_corresponding | False |
| authorships[14].author.id | https://openalex.org/A5102907985 |
| authorships[14].author.orcid | https://orcid.org/0000-0002-9161-3077 |
| authorships[14].author.display_name | Yong Ma |
| authorships[14].author_position | middle |
| authorships[14].raw_author_name | Yong Ma |
| authorships[14].is_corresponding | False |
| authorships[15].author.id | https://openalex.org/A5102905653 |
| authorships[15].author.orcid | https://orcid.org/0000-0001-7814-7625 |
| authorships[15].author.display_name | Danijela Stojanović |
| authorships[15].author_position | middle |
| authorships[15].raw_author_name | Danijela Stojanovic |
| authorships[15].is_corresponding | False |
| authorships[16].author.id | https://openalex.org/A5021380975 |
| authorships[16].author.orcid | |
| authorships[16].author.display_name | Yueqin Zhao |
| authorships[16].author_position | middle |
| authorships[16].raw_author_name | Yueqin Zhao |
| authorships[16].is_corresponding | False |
| authorships[17].author.id | https://openalex.org/A5036429950 |
| authorships[17].author.orcid | https://orcid.org/0000-0002-1609-7420 |
| authorships[17].author.display_name | Robert Ball |
| authorships[17].author_position | middle |
| authorships[17].raw_author_name | Robert Ball |
| authorships[17].is_corresponding | False |
| authorships[18].author.id | https://openalex.org/A5040276088 |
| authorships[18].author.orcid | https://orcid.org/0000-0002-2782-2050 |
| authorships[18].author.display_name | Jennifer C. Nelson |
| authorships[18].author_position | last |
| authorships[18].raw_author_name | Jennifer C Nelson |
| authorships[18].is_corresponding | False |
| has_content.pdf | True |
| has_content.grobid_xml | True |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://academic.oup.com/aje/article-pdf/192/2/283/49088177/kwac182.pdf |
| open_access.oa_status | bronze |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Improving Methods of Identifying Anaphylaxis for Medical Product Safety Surveillance Using Natural Language Processing and Machine Learning |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T03:46:38.306776 |
| primary_topic.id | https://openalex.org/T10634 |
| primary_topic.field.id | https://openalex.org/fields/27 |
| primary_topic.field.display_name | Medicine |
| primary_topic.score | 0.9993000030517578 |
| primary_topic.domain.id | https://openalex.org/domains/4 |
| primary_topic.domain.display_name | Health Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/2723 |
| primary_topic.subfield.display_name | Immunology and Allergy |
| primary_topic.display_name | Food Allergy and Anaphylaxis Research |
| related_works | https://openalex.org/W2884884929, https://openalex.org/W2360072188, https://openalex.org/W864029991, https://openalex.org/W2058868407, https://openalex.org/W2130378978, https://openalex.org/W2373192550, https://openalex.org/W3123742489, https://openalex.org/W2119185745, https://openalex.org/W2000473249, https://openalex.org/W2791534013 |
| cited_by_count | 26 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 8 |
| counts_by_year[1].year | 2024 |
| counts_by_year[1].cited_by_count | 12 |
| counts_by_year[2].year | 2023 |
| counts_by_year[2].cited_by_count | 6 |
| locations_count | 3 |
| best_oa_location.id | doi:10.1093/aje/kwac182 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S170967050 |
| best_oa_location.source.issn | 0002-9262, 1476-6256 |
| best_oa_location.source.type | journal |
| best_oa_location.source.is_oa | False |
| best_oa_location.source.issn_l | 0002-9262 |
| best_oa_location.source.is_core | True |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | American Journal of Epidemiology |
| best_oa_location.source.host_organization | https://openalex.org/P4310311648 |
| best_oa_location.source.host_organization_name | Oxford University Press |
| best_oa_location.source.host_organization_lineage | https://openalex.org/P4310311648, https://openalex.org/P4310311647 |
| best_oa_location.source.host_organization_lineage_names | Oxford University Press, University of Oxford |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://academic.oup.com/aje/article-pdf/192/2/283/49088177/kwac182.pdf |
| best_oa_location.version | publishedVersion |
| best_oa_location.raw_type | journal-article |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | True |
| best_oa_location.is_published | True |
| best_oa_location.raw_source_name | American Journal of Epidemiology |
| best_oa_location.landing_page_url | https://doi.org/10.1093/aje/kwac182 |
| primary_location.id | doi:10.1093/aje/kwac182 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S170967050 |
| primary_location.source.issn | 0002-9262, 1476-6256 |
| primary_location.source.type | journal |
| primary_location.source.is_oa | False |
| primary_location.source.issn_l | 0002-9262 |
| primary_location.source.is_core | True |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | American Journal of Epidemiology |
| primary_location.source.host_organization | https://openalex.org/P4310311648 |
| primary_location.source.host_organization_name | Oxford University Press |
| primary_location.source.host_organization_lineage | https://openalex.org/P4310311648, https://openalex.org/P4310311647 |
| primary_location.source.host_organization_lineage_names | Oxford University Press, University of Oxford |
| primary_location.license | |
| primary_location.pdf_url | https://academic.oup.com/aje/article-pdf/192/2/283/49088177/kwac182.pdf |
| primary_location.version | publishedVersion |
| primary_location.raw_type | journal-article |
| primary_location.license_id | |
| primary_location.is_accepted | True |
| primary_location.is_published | True |
| primary_location.raw_source_name | American Journal of Epidemiology |
| primary_location.landing_page_url | https://doi.org/10.1093/aje/kwac182 |
| publication_date | 2022-11-02 |
| publication_year | 2022 |
| referenced_works | https://openalex.org/W1674711689, https://openalex.org/W2085577106, https://openalex.org/W1992517276, https://openalex.org/W2901040498, https://openalex.org/W2066117481, https://openalex.org/W2105945068, https://openalex.org/W2116323965, https://openalex.org/W2753708538, https://openalex.org/W2615116167, https://openalex.org/W2107508737, https://openalex.org/W2030147903, https://openalex.org/W6635343491, https://openalex.org/W2902452806, https://openalex.org/W1484976590, https://openalex.org/W2762621503, https://openalex.org/W2132096417, https://openalex.org/W3016229786, https://openalex.org/W2052951012, https://openalex.org/W1885171326, https://openalex.org/W2097360283, https://openalex.org/W3157034752, https://openalex.org/W2168769804, https://openalex.org/W2990471161, https://openalex.org/W2044944166, https://openalex.org/W3132568032, https://openalex.org/W2735580341, https://openalex.org/W2345195116, https://openalex.org/W2997514453, https://openalex.org/W2889377580, https://openalex.org/W2889217150, https://openalex.org/W2889242407, https://openalex.org/W4292807770, https://openalex.org/W6767839947, https://openalex.org/W2124119575, https://openalex.org/W2889124355, https://openalex.org/W2992885821, https://openalex.org/W2893396084, https://openalex.org/W6704284518, https://openalex.org/W1983745852, https://openalex.org/W2105637130, https://openalex.org/W2159583324, https://openalex.org/W2021075383, https://openalex.org/W2146089916, https://openalex.org/W1964625659, https://openalex.org/W6643104708, https://openalex.org/W2016871293, https://openalex.org/W1863753810, https://openalex.org/W2169818249, https://openalex.org/W196776705, https://openalex.org/W138398095, https://openalex.org/W4294541781, https://openalex.org/W2126292488, https://openalex.org/W2117812871, https://openalex.org/W3194280623, https://openalex.org/W1144300234, https://openalex.org/W2520392019, https://openalex.org/W2765693998, https://openalex.org/W2991319174, https://openalex.org/W2606769376, https://openalex.org/W3160137267, https://openalex.org/W2078881122, https://openalex.org/W1970169482, https://openalex.org/W1594888675, https://openalex.org/W1972099155, https://openalex.org/W1041910, https://openalex.org/W2342068386, https://openalex.org/W2973032093 |
| referenced_works_count | 67 |
| abstract_inverted_index.2 | 47 |
| abstract_inverted_index.A | 164 |
| abstract_inverted_index.a | 125 |
| abstract_inverted_index.In | 91 |
| abstract_inverted_index.We | 1, 56 |
| abstract_inverted_index.in | 46, 51, 112, 154, 160, 177, 190 |
| abstract_inverted_index.of | 21, 97, 110, 128, 170, 175, 183, 188, 200 |
| abstract_inverted_index.on | 32, 75 |
| abstract_inverted_index.or | 39 |
| abstract_inverted_index.to | 3, 14, 26, 139, 150 |
| abstract_inverted_index.154 | 95 |
| abstract_inverted_index.180 | 108 |
| abstract_inverted_index.239 | 98 |
| abstract_inverted_index.277 | 111 |
| abstract_inverted_index.516 | 33 |
| abstract_inverted_index.56% | 189 |
| abstract_inverted_index.66% | 176 |
| abstract_inverted_index.78% | 184 |
| abstract_inverted_index.79% | 171 |
| abstract_inverted_index.AUC | 127, 138 |
| abstract_inverted_index.CI: | 131 |
| abstract_inverted_index.PPV | 169, 182 |
| abstract_inverted_index.and | 8, 68, 89, 156, 172, 185, 195 |
| abstract_inverted_index.for | 65, 71, 104 |
| abstract_inverted_index.had | 180 |
| abstract_inverted_index.met | 101 |
| abstract_inverted_index.one | 58 |
| abstract_inverted_index.the | 52, 69, 79, 92, 113 |
| abstract_inverted_index.(95% | 130 |
| abstract_inverted_index.0.58 | 129 |
| abstract_inverted_index.0.62 | 140 |
| abstract_inverted_index.0.67 | 157 |
| abstract_inverted_index.0.70 | 151 |
| abstract_inverted_index.AUCs | 149 |
| abstract_inverted_index.area | 77 |
| abstract_inverted_index.data | 31, 64, 123, 179, 197 |
| abstract_inverted_index.only | 120 |
| abstract_inverted_index.site | 94 |
| abstract_inverted_index.used | 57 |
| abstract_inverted_index.with | 35, 107, 167 |
| abstract_inverted_index.(64%) | 96 |
| abstract_inverted_index.(65%) | 109 |
| abstract_inverted_index.(NLP) | 12 |
| abstract_inverted_index.0.54, | 132 |
| abstract_inverted_index.0.71) | 159 |
| abstract_inverted_index.0.75) | 153 |
| abstract_inverted_index.based | 74 |
| abstract_inverted_index.codes | 43 |
| abstract_inverted_index.could | 18 |
| abstract_inverted_index.curve | 83 |
| abstract_inverted_index.data. | 163, 192 |
| abstract_inverted_index.model | 66 |
| abstract_inverted_index.site. | 115 |
| abstract_inverted_index.under | 78 |
| abstract_inverted_index.using | 30, 119 |
| abstract_inverted_index.value | 87 |
| abstract_inverted_index.(0.58, | 141 |
| abstract_inverted_index.(0.63, | 158 |
| abstract_inverted_index.(0.66, | 152 |
| abstract_inverted_index.(AUC), | 84 |
| abstract_inverted_index.(PPV), | 88 |
| abstract_inverted_index.0.63). | 133 |
| abstract_inverted_index.0.66); | 142 |
| abstract_inverted_index.United | 54 |
| abstract_inverted_index.claims | 122 |
| abstract_inverted_index.during | 44 |
| abstract_inverted_index.events | 29, 100 |
| abstract_inverted_index.models | 118 |
| abstract_inverted_index.sought | 2 |
| abstract_inverted_index.Machine | 134, 193 |
| abstract_inverted_index.States. | 55 |
| abstract_inverted_index.applied | 13 |
| abstract_inverted_index.events. | 203 |
| abstract_inverted_index.further | 146 |
| abstract_inverted_index.improve | 19 |
| abstract_inverted_index.machine | 6 |
| abstract_inverted_index.medical | 16 |
| abstract_inverted_index.natural | 9 |
| abstract_inverted_index.records | 17 |
| abstract_inverted_index.whether | 5 |
| abstract_inverted_index.Abstract | 0 |
| abstract_inverted_index.Logistic | 116 |
| abstract_inverted_index.achieved | 124 |
| abstract_inverted_index.compared | 106 |
| abstract_inverted_index.criteria | 103 |
| abstract_inverted_index.external | 72, 161, 191 |
| abstract_inverted_index.identify | 27 |
| abstract_inverted_index.improved | 136, 198 |
| abstract_inverted_index.language | 10 |
| abstract_inverted_index.learning | 7, 135, 194 |
| abstract_inverted_index.manually | 60 |
| abstract_inverted_index.outcomes | 63 |
| abstract_inverted_index.patients | 34 |
| abstract_inverted_index.positive | 85 |
| abstract_inverted_index.receiver | 80 |
| abstract_inverted_index.reviewed | 61 |
| abstract_inverted_index.site’s | 59 |
| abstract_inverted_index.Northwest | 53 |
| abstract_inverted_index.automated | 22 |
| abstract_inverted_index.determine | 4 |
| abstract_inverted_index.diagnosis | 42 |
| abstract_inverted_index.emergency | 37 |
| abstract_inverted_index.increased | 147 |
| abstract_inverted_index.inpatient | 40 |
| abstract_inverted_index.operating | 81 |
| abstract_inverted_index.other’s | 70 |
| abstract_inverted_index.potential | 99 |
| abstract_inverted_index.threshold | 166 |
| abstract_inverted_index.validated | 201 |
| abstract_inverted_index.algorithms | 25 |
| abstract_inverted_index.covariates | 145 |
| abstract_inverted_index.electronic | 15 |
| abstract_inverted_index.integrated | 48 |
| abstract_inverted_index.predictive | 86 |
| abstract_inverted_index.processing | 11 |
| abstract_inverted_index.regression | 117 |
| abstract_inverted_index.structured | 121 |
| abstract_inverted_index.validation | 73, 114, 162 |
| abstract_inverted_index.2015–2019 | 45 |
| abstract_inverted_index.NLP-derived | 144, 196 |
| abstract_inverted_index.anaphylaxis | 28, 41, 105, 202 |
| abstract_inverted_index.department, | 38 |
| abstract_inverted_index.development | 67, 93, 155, 178 |
| abstract_inverted_index.health-care | 23, 49 |
| abstract_inverted_index.outpatient, | 36 |
| abstract_inverted_index.performance | 20 |
| abstract_inverted_index.sensitivity | 174, 187 |
| abstract_inverted_index.adjudication | 102 |
| abstract_inverted_index.claims-based | 24 |
| abstract_inverted_index.institutions | 50 |
| abstract_inverted_index.sensitivity. | 90 |
| abstract_inverted_index.gold-standard | 62 |
| abstract_inverted_index.incorporating | 143 |
| abstract_inverted_index.characteristic | 82 |
| abstract_inverted_index.classification | 165 |
| abstract_inverted_index.identification | 199 |
| abstract_inverted_index.cross-validated | 76, 126, 137, 148, 168, 173, 181, 186 |
| cited_by_percentile_year.max | 99 |
| cited_by_percentile_year.min | 98 |
| countries_distinct_count | 0 |
| institutions_distinct_count | 19 |
| citation_normalized_percentile.value | 0.94368243 |
| citation_normalized_percentile.is_in_top_1_percent | False |
| citation_normalized_percentile.is_in_top_10_percent | True |