ChatG ‐PD ? Comparing large language model artificial intelligence and faculty rankings of the competitiveness of standardized letters of evaluation
Article Swipe
Benjamin Schnapp
,
Morgan Sehdev
,
Caitlin Schrepel
,
Sharon Bord
,
Alexis Pelletier‐Bui
,
Al’ai Alvarez
,
Nicole M. Dubosh
,
Yoon Soo Park
,
Eric Shappell
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.1002/aet2.11052
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.1002/aet2.11052
Background While faculty have previously been shown to have high levels of agreement about the competitiveness of emergency medicine (EM) standardized letters of evaluation (SLOEs), reviewing SLOEs remains a highly time‐intensive process for faculty. Artificial intelligence large language models (LLMs) have shown promise for effectively analyzing large volumes of data across a variety of contexts, but their ability to interpret SLOEs is unknown. Objective The objective was to evaluate the ability of LLMs to rate EM SLOEs on competitiveness compared to faculty consensus and previously developed algorithms. Methods Fifty mock SLOE letters were drafted and analyzed seven times by a data‐focused LLM with instructions to rank them based on desirability for residency. The LLM was also asked to use its own criteria to decide which characteristics are most important for residency and revise its ranking of the SLOEs. LLM‐generated rank lists were compared with faculty consensus rankings. Results There was a high degree of correlation ( r = 0.96) between the rank list initially generated by LLM consensus and the rank list generated by trained faculty. The correlation between the revised list generated by the LLM and the faculty consensus was lower ( r = 0.86). Conclusions The LLM generated rankings showed strong correlation with expert faculty consensus rankings with minimal input of faculty time and effort.
Related Topics
Concepts
Metadata
- Type
- article
- Language
- en
- Landing Page
- https://doi.org/10.1002/aet2.11052
- OA Status
- hybrid
- Cited By
- 2
- References
- 20
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4405216920
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4405216920Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.1002/aet2.11052Digital Object Identifier
- Title
-
ChatG ‐PD ? Comparing large language model artificial intelligence and faculty rankings of the competitiveness of standardized letters of evaluationWork title - Type
-
articleOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-12-01Full publication date if available
- Authors
-
Benjamin Schnapp, Morgan Sehdev, Caitlin Schrepel, Sharon Bord, Alexis Pelletier‐Bui, Al’ai Alvarez, Nicole M. Dubosh, Yoon Soo Park, Eric ShappellList of authors in order
- Landing page
-
https://doi.org/10.1002/aet2.11052Publisher landing page
- Open access
-
YesWhether a free full text is available
- OA status
-
hybridOpen access status per OpenAlex
- OA URL
-
https://doi.org/10.1002/aet2.11052Direct OA link when available
- Concepts
-
Variety (cybernetics), Artificial intelligence, Computer science, Operations research, Psychology, Natural language processing, EngineeringTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
2Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 2Per-year citation counts (last 5 years)
- References (count)
-
20Number of works referenced by this work
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4405216920 |
|---|---|
| doi | https://doi.org/10.1002/aet2.11052 |
| ids.doi | https://doi.org/10.1002/aet2.11052 |
| ids.pmid | https://pubmed.ncbi.nlm.nih.gov/39660298 |
| ids.openalex | https://openalex.org/W4405216920 |
| fwci | 0.48679942 |
| type | article |
| title | |
| biblio.issue | 6 |
| biblio.volume | 8 |
| biblio.last_page | e11052 |
| biblio.first_page | e11052 |
| topics[0].id | https://openalex.org/T11636 |
| topics[0].field.id | https://openalex.org/fields/27 |
| topics[0].field.display_name | Medicine |
| topics[0].score | 0.9883000254631042 |
| topics[0].domain.id | https://openalex.org/domains/4 |
| topics[0].domain.display_name | Health Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/2718 |
| topics[0].subfield.display_name | Health Informatics |
| topics[0].display_name | Artificial Intelligence in Healthcare and Education |
| topics[1].id | https://openalex.org/T11894 |
| topics[1].field.id | https://openalex.org/fields/27 |
| topics[1].field.display_name | Medicine |
| topics[1].score | 0.9848999977111816 |
| topics[1].domain.id | https://openalex.org/domains/4 |
| topics[1].domain.display_name | Health Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/2741 |
| topics[1].subfield.display_name | Radiology, Nuclear Medicine and Imaging |
| topics[1].display_name | Radiology practices and education |
| topics[2].id | https://openalex.org/T12574 |
| topics[2].field.id | https://openalex.org/fields/27 |
| topics[2].field.display_name | Medicine |
| topics[2].score | 0.972000002861023 |
| topics[2].domain.id | https://openalex.org/domains/4 |
| topics[2].domain.display_name | Health Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/2714 |
| topics[2].subfield.display_name | Family Practice |
| topics[2].display_name | Clinical Reasoning and Diagnostic Skills |
| is_xpac | False |
| apc_list.value | 3140 |
| apc_list.currency | USD |
| apc_list.value_usd | 3140 |
| apc_paid.value | 3140 |
| apc_paid.currency | USD |
| apc_paid.value_usd | 3140 |
| concepts[0].id | https://openalex.org/C136197465 |
| concepts[0].level | 2 |
| concepts[0].score | 0.6566786766052246 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q1729295 |
| concepts[0].display_name | Variety (cybernetics) |
| concepts[1].id | https://openalex.org/C154945302 |
| concepts[1].level | 1 |
| concepts[1].score | 0.437171995639801 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[1].display_name | Artificial intelligence |
| concepts[2].id | https://openalex.org/C41008148 |
| concepts[2].level | 0 |
| concepts[2].score | 0.4105480909347534 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[2].display_name | Computer science |
| concepts[3].id | https://openalex.org/C42475967 |
| concepts[3].level | 1 |
| concepts[3].score | 0.3852594494819641 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q194292 |
| concepts[3].display_name | Operations research |
| concepts[4].id | https://openalex.org/C15744967 |
| concepts[4].level | 0 |
| concepts[4].score | 0.33933892846107483 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q9418 |
| concepts[4].display_name | Psychology |
| concepts[5].id | https://openalex.org/C204321447 |
| concepts[5].level | 1 |
| concepts[5].score | 0.3259899616241455 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q30642 |
| concepts[5].display_name | Natural language processing |
| concepts[6].id | https://openalex.org/C127413603 |
| concepts[6].level | 0 |
| concepts[6].score | 0.21944105625152588 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q11023 |
| concepts[6].display_name | Engineering |
| keywords[0].id | https://openalex.org/keywords/variety |
| keywords[0].score | 0.6566786766052246 |
| keywords[0].display_name | Variety (cybernetics) |
| keywords[1].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[1].score | 0.437171995639801 |
| keywords[1].display_name | Artificial intelligence |
| keywords[2].id | https://openalex.org/keywords/computer-science |
| keywords[2].score | 0.4105480909347534 |
| keywords[2].display_name | Computer science |
| keywords[3].id | https://openalex.org/keywords/operations-research |
| keywords[3].score | 0.3852594494819641 |
| keywords[3].display_name | Operations research |
| keywords[4].id | https://openalex.org/keywords/psychology |
| keywords[4].score | 0.33933892846107483 |
| keywords[4].display_name | Psychology |
| keywords[5].id | https://openalex.org/keywords/natural-language-processing |
| keywords[5].score | 0.3259899616241455 |
| keywords[5].display_name | Natural language processing |
| keywords[6].id | https://openalex.org/keywords/engineering |
| keywords[6].score | 0.21944105625152588 |
| keywords[6].display_name | Engineering |
| language | en |
| locations[0].id | doi:10.1002/aet2.11052 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4210216950 |
| locations[0].source.issn | 2472-5390 |
| locations[0].source.type | journal |
| locations[0].source.is_oa | False |
| locations[0].source.issn_l | 2472-5390 |
| locations[0].source.is_core | True |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | AEM Education and Training |
| locations[0].source.host_organization | https://openalex.org/P4310320595 |
| locations[0].source.host_organization_name | Wiley |
| locations[0].source.host_organization_lineage | https://openalex.org/P4310320595 |
| locations[0].source.host_organization_lineage_names | Wiley |
| locations[0].license | cc-by-nc |
| locations[0].pdf_url | |
| locations[0].version | publishedVersion |
| locations[0].raw_type | journal-article |
| locations[0].license_id | https://openalex.org/licenses/cc-by-nc |
| locations[0].is_accepted | True |
| locations[0].is_published | True |
| locations[0].raw_source_name | AEM Education and Training |
| locations[0].landing_page_url | https://doi.org/10.1002/aet2.11052 |
| locations[1].id | pmid:39660298 |
| locations[1].is_oa | False |
| locations[1].source.id | https://openalex.org/S4306525036 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | False |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | PubMed |
| locations[1].source.host_organization | https://openalex.org/I1299303238 |
| locations[1].source.host_organization_name | National Institutes of Health |
| locations[1].source.host_organization_lineage | https://openalex.org/I1299303238 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | publishedVersion |
| locations[1].raw_type | |
| locations[1].license_id | |
| locations[1].is_accepted | True |
| locations[1].is_published | True |
| locations[1].raw_source_name | AEM education and training |
| locations[1].landing_page_url | https://pubmed.ncbi.nlm.nih.gov/39660298 |
| locations[2].id | pmh:oai:pubmedcentral.nih.gov:11628426 |
| locations[2].is_oa | True |
| locations[2].source.id | https://openalex.org/S2764455111 |
| locations[2].source.issn | |
| locations[2].source.type | repository |
| locations[2].source.is_oa | False |
| locations[2].source.issn_l | |
| locations[2].source.is_core | False |
| locations[2].source.is_in_doaj | False |
| locations[2].source.display_name | PubMed Central |
| locations[2].source.host_organization | https://openalex.org/I1299303238 |
| locations[2].source.host_organization_name | National Institutes of Health |
| locations[2].source.host_organization_lineage | https://openalex.org/I1299303238 |
| locations[2].license | other-oa |
| locations[2].pdf_url | |
| locations[2].version | submittedVersion |
| locations[2].raw_type | Text |
| locations[2].license_id | https://openalex.org/licenses/other-oa |
| locations[2].is_accepted | False |
| locations[2].is_published | False |
| locations[2].raw_source_name | AEM Educ Train |
| locations[2].landing_page_url | https://www.ncbi.nlm.nih.gov/pmc/articles/11628426 |
| indexed_in | crossref, pubmed |
| authorships[0].author.id | https://openalex.org/A5091561751 |
| authorships[0].author.orcid | https://orcid.org/0000-0001-5031-8269 |
| authorships[0].author.display_name | Benjamin Schnapp |
| authorships[0].countries | US |
| authorships[0].affiliations[0].institution_ids | https://openalex.org/I135310074 |
| authorships[0].affiliations[0].raw_affiliation_string | BerbeeWalsh Department of Emergency Medicine, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin, USA |
| authorships[0].institutions[0].id | https://openalex.org/I135310074 |
| authorships[0].institutions[0].ror | https://ror.org/01y2jtd41 |
| authorships[0].institutions[0].type | education |
| authorships[0].institutions[0].lineage | https://openalex.org/I135310074 |
| authorships[0].institutions[0].country_code | US |
| authorships[0].institutions[0].display_name | University of Wisconsin–Madison |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Benjamin Schnapp |
| authorships[0].is_corresponding | True |
| authorships[0].raw_affiliation_strings | BerbeeWalsh Department of Emergency Medicine, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin, USA |
| authorships[1].author.id | https://openalex.org/A5016719301 |
| authorships[1].author.orcid | https://orcid.org/0000-0002-2981-0512 |
| authorships[1].author.display_name | Morgan Sehdev |
| authorships[1].countries | US |
| authorships[1].affiliations[0].institution_ids | https://openalex.org/I1283280774, https://openalex.org/I4210087915, https://openalex.org/I4210089597 |
| authorships[1].affiliations[0].raw_affiliation_string | Harvard-Affiliated Emergency Medicine Residency, Brigham and Women's Hospital/Massachusetts General Hospital, Boston, Massachusetts, USA |
| authorships[1].institutions[0].id | https://openalex.org/I1283280774 |
| authorships[1].institutions[0].ror | https://ror.org/04b6nzv94 |
| authorships[1].institutions[0].type | healthcare |
| authorships[1].institutions[0].lineage | https://openalex.org/I1283280774, https://openalex.org/I48633490 |
| authorships[1].institutions[0].country_code | US |
| authorships[1].institutions[0].display_name | Brigham and Women's Hospital |
| authorships[1].institutions[1].id | https://openalex.org/I4210089597 |
| authorships[1].institutions[1].ror | https://ror.org/005m2at17 |
| authorships[1].institutions[1].type | other |
| authorships[1].institutions[1].lineage | https://openalex.org/I1283280774, https://openalex.org/I4210087915, https://openalex.org/I4210089597, https://openalex.org/I48633490 |
| authorships[1].institutions[1].country_code | US |
| authorships[1].institutions[1].display_name | Harvard Affiliated Emergency Medicine Residency |
| authorships[1].institutions[2].id | https://openalex.org/I4210087915 |
| authorships[1].institutions[2].ror | https://ror.org/002pd6e78 |
| authorships[1].institutions[2].type | healthcare |
| authorships[1].institutions[2].lineage | https://openalex.org/I4210087915, https://openalex.org/I48633490 |
| authorships[1].institutions[2].country_code | US |
| authorships[1].institutions[2].display_name | Massachusetts General Hospital |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Morgan Sehdev |
| authorships[1].is_corresponding | False |
| authorships[1].raw_affiliation_strings | Harvard-Affiliated Emergency Medicine Residency, Brigham and Women's Hospital/Massachusetts General Hospital, Boston, Massachusetts, USA |
| authorships[2].author.id | https://openalex.org/A5029508560 |
| authorships[2].author.orcid | https://orcid.org/0000-0003-0204-7657 |
| authorships[2].author.display_name | Caitlin Schrepel |
| authorships[2].countries | US |
| authorships[2].affiliations[0].institution_ids | https://openalex.org/I201448701 |
| authorships[2].affiliations[0].raw_affiliation_string | Department of Emergency Medicine, University of Washington, Seattle, Washington, USA |
| authorships[2].institutions[0].id | https://openalex.org/I201448701 |
| authorships[2].institutions[0].ror | https://ror.org/00cvxb145 |
| authorships[2].institutions[0].type | education |
| authorships[2].institutions[0].lineage | https://openalex.org/I201448701 |
| authorships[2].institutions[0].country_code | US |
| authorships[2].institutions[0].display_name | University of Washington |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Caitlin Schrepel |
| authorships[2].is_corresponding | False |
| authorships[2].raw_affiliation_strings | Department of Emergency Medicine, University of Washington, Seattle, Washington, USA |
| authorships[3].author.id | https://openalex.org/A5062260462 |
| authorships[3].author.orcid | https://orcid.org/0009-0008-5987-4139 |
| authorships[3].author.display_name | Sharon Bord |
| authorships[3].countries | US |
| authorships[3].affiliations[0].institution_ids | https://openalex.org/I145311948, https://openalex.org/I2799853436 |
| authorships[3].affiliations[0].raw_affiliation_string | Department of Emergency Medicine, The Johns Hopkins University School of Medicine, Baltimore, Maryland, USA |
| authorships[3].institutions[0].id | https://openalex.org/I2799853436 |
| authorships[3].institutions[0].ror | https://ror.org/037zgn354 |
| authorships[3].institutions[0].type | healthcare |
| authorships[3].institutions[0].lineage | https://openalex.org/I2799853436 |
| authorships[3].institutions[0].country_code | US |
| authorships[3].institutions[0].display_name | Johns Hopkins Medicine |
| authorships[3].institutions[1].id | https://openalex.org/I145311948 |
| authorships[3].institutions[1].ror | https://ror.org/00za53h95 |
| authorships[3].institutions[1].type | education |
| authorships[3].institutions[1].lineage | https://openalex.org/I145311948 |
| authorships[3].institutions[1].country_code | US |
| authorships[3].institutions[1].display_name | Johns Hopkins University |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Sharon Bord |
| authorships[3].is_corresponding | False |
| authorships[3].raw_affiliation_strings | Department of Emergency Medicine, The Johns Hopkins University School of Medicine, Baltimore, Maryland, USA |
| authorships[4].author.id | https://openalex.org/A5087002482 |
| authorships[4].author.orcid | https://orcid.org/0000-0002-9969-4123 |
| authorships[4].author.display_name | Alexis Pelletier‐Bui |
| authorships[4].countries | US |
| authorships[4].affiliations[0].institution_ids | https://openalex.org/I4210089330 |
| authorships[4].affiliations[0].raw_affiliation_string | Department of Emergency Medicine, Cooper Medical School of Rowan University, Camden, New Jersey, USA |
| authorships[4].institutions[0].id | https://openalex.org/I4210089330 |
| authorships[4].institutions[0].ror | https://ror.org/007evha27 |
| authorships[4].institutions[0].type | education |
| authorships[4].institutions[0].lineage | https://openalex.org/I4210089330, https://openalex.org/I4210157085, https://openalex.org/I44265643 |
| authorships[4].institutions[0].country_code | US |
| authorships[4].institutions[0].display_name | Cooper Medical School of Rowan University |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Alexis Pelletier‐Bui |
| authorships[4].is_corresponding | False |
| authorships[4].raw_affiliation_strings | Department of Emergency Medicine, Cooper Medical School of Rowan University, Camden, New Jersey, USA |
| authorships[5].author.id | https://openalex.org/A5041893941 |
| authorships[5].author.orcid | https://orcid.org/0000-0002-5438-2476 |
| authorships[5].author.display_name | Al’ai Alvarez |
| authorships[5].countries | US |
| authorships[5].affiliations[0].institution_ids | https://openalex.org/I97018004 |
| authorships[5].affiliations[0].raw_affiliation_string | Department of Emergency Medicine, Stanford University, Palo Alto, California, USA |
| authorships[5].institutions[0].id | https://openalex.org/I97018004 |
| authorships[5].institutions[0].ror | https://ror.org/00f54p054 |
| authorships[5].institutions[0].type | education |
| authorships[5].institutions[0].lineage | https://openalex.org/I97018004 |
| authorships[5].institutions[0].country_code | US |
| authorships[5].institutions[0].display_name | Stanford University |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Al’ai Alvarez |
| authorships[5].is_corresponding | False |
| authorships[5].raw_affiliation_strings | Department of Emergency Medicine, Stanford University, Palo Alto, California, USA |
| authorships[6].author.id | https://openalex.org/A5063244053 |
| authorships[6].author.orcid | https://orcid.org/0000-0001-8674-5334 |
| authorships[6].author.display_name | Nicole M. Dubosh |
| authorships[6].countries | US |
| authorships[6].affiliations[0].institution_ids | https://openalex.org/I1316535847, https://openalex.org/I136199984 |
| authorships[6].affiliations[0].raw_affiliation_string | Department of Emergency Medicine, Beth Israel Deaconess Medical Center/Harvard Medical School, Boston, Massachusetts, USA |
| authorships[6].institutions[0].id | https://openalex.org/I1316535847 |
| authorships[6].institutions[0].ror | https://ror.org/04drvxt59 |
| authorships[6].institutions[0].type | healthcare |
| authorships[6].institutions[0].lineage | https://openalex.org/I1316535847 |
| authorships[6].institutions[0].country_code | US |
| authorships[6].institutions[0].display_name | Beth Israel Deaconess Medical Center |
| authorships[6].institutions[1].id | https://openalex.org/I136199984 |
| authorships[6].institutions[1].ror | https://ror.org/03vek6s52 |
| authorships[6].institutions[1].type | education |
| authorships[6].institutions[1].lineage | https://openalex.org/I136199984 |
| authorships[6].institutions[1].country_code | US |
| authorships[6].institutions[1].display_name | Harvard University |
| authorships[6].author_position | middle |
| authorships[6].raw_author_name | Nicole M. Dubosh |
| authorships[6].is_corresponding | False |
| authorships[6].raw_affiliation_strings | Department of Emergency Medicine, Beth Israel Deaconess Medical Center/Harvard Medical School, Boston, Massachusetts, USA |
| authorships[7].author.id | https://openalex.org/A5100622818 |
| authorships[7].author.orcid | https://orcid.org/0000-0001-8583-4335 |
| authorships[7].author.display_name | Yoon Soo Park |
| authorships[7].countries | US |
| authorships[7].affiliations[0].institution_ids | https://openalex.org/I39422238 |
| authorships[7].affiliations[0].raw_affiliation_string | Department of Medical Education, University of Illinois at Chicago, Chicago, Illinois, USA |
| authorships[7].institutions[0].id | https://openalex.org/I39422238 |
| authorships[7].institutions[0].ror | https://ror.org/02mpq6x41 |
| authorships[7].institutions[0].type | education |
| authorships[7].institutions[0].lineage | https://openalex.org/I39422238 |
| authorships[7].institutions[0].country_code | US |
| authorships[7].institutions[0].display_name | University of Illinois Chicago |
| authorships[7].author_position | middle |
| authorships[7].raw_author_name | Yoon Soo Park |
| authorships[7].is_corresponding | False |
| authorships[7].raw_affiliation_strings | Department of Medical Education, University of Illinois at Chicago, Chicago, Illinois, USA |
| authorships[8].author.id | https://openalex.org/A5082396551 |
| authorships[8].author.orcid | https://orcid.org/0000-0003-0281-3219 |
| authorships[8].author.display_name | Eric Shappell |
| authorships[8].countries | US |
| authorships[8].affiliations[0].institution_ids | https://openalex.org/I136199984, https://openalex.org/I4210087915 |
| authorships[8].affiliations[0].raw_affiliation_string | Department of Emergency Medicine, Massachusetts General Hospital/Harvard Medical School, Boston, Massachusetts, USA |
| authorships[8].institutions[0].id | https://openalex.org/I136199984 |
| authorships[8].institutions[0].ror | https://ror.org/03vek6s52 |
| authorships[8].institutions[0].type | education |
| authorships[8].institutions[0].lineage | https://openalex.org/I136199984 |
| authorships[8].institutions[0].country_code | US |
| authorships[8].institutions[0].display_name | Harvard University |
| authorships[8].institutions[1].id | https://openalex.org/I4210087915 |
| authorships[8].institutions[1].ror | https://ror.org/002pd6e78 |
| authorships[8].institutions[1].type | healthcare |
| authorships[8].institutions[1].lineage | https://openalex.org/I4210087915, https://openalex.org/I48633490 |
| authorships[8].institutions[1].country_code | US |
| authorships[8].institutions[1].display_name | Massachusetts General Hospital |
| authorships[8].author_position | last |
| authorships[8].raw_author_name | Eric Shappell |
| authorships[8].is_corresponding | False |
| authorships[8].raw_affiliation_strings | Department of Emergency Medicine, Massachusetts General Hospital/Harvard Medical School, Boston, Massachusetts, USA |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://doi.org/10.1002/aet2.11052 |
| open_access.oa_status | hybrid |
| open_access.any_repository_has_fulltext | False |
| created_date | 2024-12-11T00:00:00 |
| display_name | |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T03:46:38.306776 |
| primary_topic.id | https://openalex.org/T11636 |
| primary_topic.field.id | https://openalex.org/fields/27 |
| primary_topic.field.display_name | Medicine |
| primary_topic.score | 0.9883000254631042 |
| primary_topic.domain.id | https://openalex.org/domains/4 |
| primary_topic.domain.display_name | Health Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/2718 |
| primary_topic.subfield.display_name | Health Informatics |
| primary_topic.display_name | Artificial Intelligence in Healthcare and Education |
| related_works | https://openalex.org/W4391375266, https://openalex.org/W2748952813, https://openalex.org/W2032233321, https://openalex.org/W3121970507, https://openalex.org/W2110028391, https://openalex.org/W54497855, https://openalex.org/W217960748, https://openalex.org/W3125814499, https://openalex.org/W2090827041, https://openalex.org/W3204019825 |
| cited_by_count | 2 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 2 |
| locations_count | 3 |
| best_oa_location.id | doi:10.1002/aet2.11052 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4210216950 |
| best_oa_location.source.issn | 2472-5390 |
| best_oa_location.source.type | journal |
| best_oa_location.source.is_oa | False |
| best_oa_location.source.issn_l | 2472-5390 |
| best_oa_location.source.is_core | True |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | AEM Education and Training |
| best_oa_location.source.host_organization | https://openalex.org/P4310320595 |
| best_oa_location.source.host_organization_name | Wiley |
| best_oa_location.source.host_organization_lineage | https://openalex.org/P4310320595 |
| best_oa_location.source.host_organization_lineage_names | Wiley |
| best_oa_location.license | cc-by-nc |
| best_oa_location.pdf_url | |
| best_oa_location.version | publishedVersion |
| best_oa_location.raw_type | journal-article |
| best_oa_location.license_id | https://openalex.org/licenses/cc-by-nc |
| best_oa_location.is_accepted | True |
| best_oa_location.is_published | True |
| best_oa_location.raw_source_name | AEM Education and Training |
| best_oa_location.landing_page_url | https://doi.org/10.1002/aet2.11052 |
| primary_location.id | doi:10.1002/aet2.11052 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4210216950 |
| primary_location.source.issn | 2472-5390 |
| primary_location.source.type | journal |
| primary_location.source.is_oa | False |
| primary_location.source.issn_l | 2472-5390 |
| primary_location.source.is_core | True |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | AEM Education and Training |
| primary_location.source.host_organization | https://openalex.org/P4310320595 |
| primary_location.source.host_organization_name | Wiley |
| primary_location.source.host_organization_lineage | https://openalex.org/P4310320595 |
| primary_location.source.host_organization_lineage_names | Wiley |
| primary_location.license | cc-by-nc |
| primary_location.pdf_url | |
| primary_location.version | publishedVersion |
| primary_location.raw_type | journal-article |
| primary_location.license_id | https://openalex.org/licenses/cc-by-nc |
| primary_location.is_accepted | True |
| primary_location.is_published | True |
| primary_location.raw_source_name | AEM Education and Training |
| primary_location.landing_page_url | https://doi.org/10.1002/aet2.11052 |
| publication_date | 2024-12-01 |
| publication_year | 2024 |
| referenced_works | https://openalex.org/W4391443830, https://openalex.org/W4385780164, https://openalex.org/W2924089730, https://openalex.org/W4311273561, https://openalex.org/W4360802167, https://openalex.org/W4390617165, https://openalex.org/W4385827730, https://openalex.org/W4379376212, https://openalex.org/W4327908244, https://openalex.org/W4394747666, https://openalex.org/W4402528009, https://openalex.org/W4386794197, https://openalex.org/W2204262283, https://openalex.org/W4401851573, https://openalex.org/W2924813521, https://openalex.org/W4376643691, https://openalex.org/W4386302153, https://openalex.org/W2995424678, https://openalex.org/W2931350392, https://openalex.org/W2616788784 |
| referenced_works_count | 20 |
| abstract_inverted_index.( | 156, 193 |
| abstract_inverted_index.= | 158, 195 |
| abstract_inverted_index.a | 29, 52, 100, 151 |
| abstract_inverted_index.r | 157, 194 |
| abstract_inverted_index.EM | 76 |
| abstract_inverted_index.by | 99, 166, 174, 184 |
| abstract_inverted_index.is | 62 |
| abstract_inverted_index.of | 12, 17, 23, 49, 54, 72, 136, 154, 213 |
| abstract_inverted_index.on | 78, 109 |
| abstract_inverted_index.to | 8, 59, 68, 74, 81, 105, 118, 123 |
| abstract_inverted_index.LLM | 102, 114, 167, 186, 199 |
| abstract_inverted_index.The | 65, 113, 177, 198 |
| abstract_inverted_index.and | 84, 95, 132, 169, 187, 216 |
| abstract_inverted_index.are | 127 |
| abstract_inverted_index.but | 56 |
| abstract_inverted_index.for | 33, 44, 111, 130 |
| abstract_inverted_index.its | 120, 134 |
| abstract_inverted_index.own | 121 |
| abstract_inverted_index.the | 15, 70, 137, 161, 170, 180, 185, 188 |
| abstract_inverted_index.use | 119 |
| abstract_inverted_index.was | 67, 115, 150, 191 |
| abstract_inverted_index.(EM) | 20 |
| abstract_inverted_index.LLMs | 73 |
| abstract_inverted_index.SLOE | 91 |
| abstract_inverted_index.also | 116 |
| abstract_inverted_index.been | 6 |
| abstract_inverted_index.data | 50 |
| abstract_inverted_index.have | 4, 9, 41 |
| abstract_inverted_index.high | 10, 152 |
| abstract_inverted_index.list | 163, 172, 182 |
| abstract_inverted_index.mock | 90 |
| abstract_inverted_index.most | 128 |
| abstract_inverted_index.rank | 106, 140, 162, 171 |
| abstract_inverted_index.rate | 75 |
| abstract_inverted_index.them | 107 |
| abstract_inverted_index.time | 215 |
| abstract_inverted_index.were | 93, 142 |
| abstract_inverted_index.with | 103, 144, 205, 210 |
| abstract_inverted_index.0.96) | 159 |
| abstract_inverted_index.Fifty | 89 |
| abstract_inverted_index.SLOEs | 27, 61, 77 |
| abstract_inverted_index.There | 149 |
| abstract_inverted_index.While | 2 |
| abstract_inverted_index.about | 14 |
| abstract_inverted_index.asked | 117 |
| abstract_inverted_index.based | 108 |
| abstract_inverted_index.input | 212 |
| abstract_inverted_index.large | 37, 47 |
| abstract_inverted_index.lists | 141 |
| abstract_inverted_index.lower | 192 |
| abstract_inverted_index.seven | 97 |
| abstract_inverted_index.shown | 7, 42 |
| abstract_inverted_index.their | 57 |
| abstract_inverted_index.times | 98 |
| abstract_inverted_index.which | 125 |
| abstract_inverted_index.(LLMs) | 40 |
| abstract_inverted_index.0.86). | 196 |
| abstract_inverted_index.SLOEs. | 138 |
| abstract_inverted_index.across | 51 |
| abstract_inverted_index.decide | 124 |
| abstract_inverted_index.degree | 153 |
| abstract_inverted_index.expert | 206 |
| abstract_inverted_index.highly | 30 |
| abstract_inverted_index.levels | 11 |
| abstract_inverted_index.models | 39 |
| abstract_inverted_index.revise | 133 |
| abstract_inverted_index.showed | 202 |
| abstract_inverted_index.strong | 203 |
| abstract_inverted_index.Methods | 88 |
| abstract_inverted_index.Results | 148 |
| abstract_inverted_index.ability | 58, 71 |
| abstract_inverted_index.between | 160, 179 |
| abstract_inverted_index.drafted | 94 |
| abstract_inverted_index.effort. | 217 |
| abstract_inverted_index.faculty | 3, 82, 145, 189, 207, 214 |
| abstract_inverted_index.letters | 22, 92 |
| abstract_inverted_index.minimal | 211 |
| abstract_inverted_index.process | 32 |
| abstract_inverted_index.promise | 43 |
| abstract_inverted_index.ranking | 135 |
| abstract_inverted_index.remains | 28 |
| abstract_inverted_index.revised | 181 |
| abstract_inverted_index.trained | 175 |
| abstract_inverted_index.variety | 53 |
| abstract_inverted_index.volumes | 48 |
| abstract_inverted_index.(SLOEs), | 25 |
| abstract_inverted_index.Abstract | 0 |
| abstract_inverted_index.analyzed | 96 |
| abstract_inverted_index.compared | 80, 143 |
| abstract_inverted_index.criteria | 122 |
| abstract_inverted_index.evaluate | 69 |
| abstract_inverted_index.faculty. | 34, 176 |
| abstract_inverted_index.language | 38 |
| abstract_inverted_index.medicine | 19 |
| abstract_inverted_index.rankings | 201, 209 |
| abstract_inverted_index.unknown. | 63 |
| abstract_inverted_index.Objective | 64 |
| abstract_inverted_index.agreement | 13 |
| abstract_inverted_index.analyzing | 46 |
| abstract_inverted_index.consensus | 83, 146, 168, 190, 208 |
| abstract_inverted_index.contexts, | 55 |
| abstract_inverted_index.developed | 86 |
| abstract_inverted_index.emergency | 18 |
| abstract_inverted_index.generated | 165, 173, 183, 200 |
| abstract_inverted_index.important | 129 |
| abstract_inverted_index.initially | 164 |
| abstract_inverted_index.interpret | 60 |
| abstract_inverted_index.objective | 66 |
| abstract_inverted_index.rankings. | 147 |
| abstract_inverted_index.residency | 131 |
| abstract_inverted_index.reviewing | 26 |
| abstract_inverted_index.Artificial | 35 |
| abstract_inverted_index.Background | 1 |
| abstract_inverted_index.evaluation | 24 |
| abstract_inverted_index.previously | 5, 85 |
| abstract_inverted_index.residency. | 112 |
| abstract_inverted_index.Conclusions | 197 |
| abstract_inverted_index.algorithms. | 87 |
| abstract_inverted_index.correlation | 155, 178, 204 |
| abstract_inverted_index.effectively | 45 |
| abstract_inverted_index.desirability | 110 |
| abstract_inverted_index.instructions | 104 |
| abstract_inverted_index.intelligence | 36 |
| abstract_inverted_index.standardized | 21 |
| abstract_inverted_index.data‐focused | 101 |
| abstract_inverted_index.LLM‐generated | 139 |
| abstract_inverted_index.characteristics | 126 |
| abstract_inverted_index.competitiveness | 16, 79 |
| abstract_inverted_index.time‐intensive | 31 |
| cited_by_percentile_year.max | 97 |
| cited_by_percentile_year.min | 95 |
| corresponding_author_ids | https://openalex.org/A5091561751 |
| countries_distinct_count | 1 |
| institutions_distinct_count | 9 |
| corresponding_institution_ids | https://openalex.org/I135310074 |
| sustainable_development_goals[0].id | https://metadata.un.org/sdg/4 |
| sustainable_development_goals[0].score | 0.7400000095367432 |
| sustainable_development_goals[0].display_name | Quality Education |
| citation_normalized_percentile.value | 0.63310318 |
| citation_normalized_percentile.is_in_top_1_percent | False |
| citation_normalized_percentile.is_in_top_10_percent | False |