Diagnosing Hallucination Risk in AI Surgical Decision-Support: A Sequential Framework for Sequential Validation Article Swipe
Dong Chen
,
Yumei Wei
,
Zhiyang He
,
Guan‐Ming Kuang
,
Chao Ye
,
M. R. An
,
Hui Peng
,
Yong Hu
,
Huiren Tao
,
Kenneth Cheung
·
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2511.00588
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2511.00588
Large language models (LLMs) offer transformative potential for clinical decision support in spine surgery but pose significant risks through hallucinations, which are factually inconsistent or contextually misaligned outputs that may compromise patient safety. This study introduces a clinician-centered framework to quantify hallucination risks by evaluating diagnostic precision, recommendation quality, reasoning robustness, output coherence, and knowledge alignment. We assessed six leading LLMs across 30 expert-validated spinal cases. DeepSeek-R1 demonstrated superior overall performance (total score: 86.03 $\pm$ 2.08), particularly in high-stakes domains such as trauma and infection. A critical finding reveals that reasoning-enhanced model variants did not uniformly outperform standard counterparts: Claude-3.7-Sonnet's extended thinking mode underperformed relative to its standard version (80.79 $\pm$ 1.83 vs. 81.56 $\pm$ 1.92), indicating extended chain-of-thought reasoning alone is insufficient for clinical reliability. Multidimensional stress-testing exposed model-specific vulnerabilities, with recommendation quality degrading by 7.4% under amplified complexity. This decline contrasted with marginal improvements in rationality (+2.0%), readability (+1.7%) and diagnosis (+4.7%), highlighting a concerning divergence between perceived coherence and actionable guidance. Our findings advocate integrating interpretability mechanisms (e.g., reasoning chain visualization) into clinical workflows and establish a safety-aware validation framework for surgical LLM deployment.
Related Topics
Concepts
No concepts available.
Metadata
- Type
- preprint
- Landing Page
- http://arxiv.org/abs/2511.00588
- https://arxiv.org/pdf/2511.00588
- OA Status
- green
- OpenAlex ID
- https://openalex.org/W4415938315
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4415938315Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2511.00588Digital Object Identifier
- Title
-
Diagnosing Hallucination Risk in AI Surgical Decision-Support: A Sequential Framework for Sequential ValidationWork title
- Type
-
preprintOpenAlex work type
- Publication year
-
2025Year of publication
- Publication date
-
2025-11-01Full publication date if available
- Authors
-
Dong Chen, Yumei Wei, Zhiyang He, Guan‐Ming Kuang, Chao Ye, M. R. An, Hui Peng, Yong Hu, Huiren Tao, Kenneth CheungList of authors in order
- Landing page
-
https://arxiv.org/abs/2511.00588Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2511.00588Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2511.00588Direct OA link when available
- Cited by
-
0Total citation count in OpenAlex
Full payload
| id | https://openalex.org/W4415938315 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2511.00588 |
| ids.doi | https://doi.org/10.48550/arxiv.2511.00588 |
| ids.openalex | https://openalex.org/W4415938315 |
| fwci | |
| type | preprint |
| title | Diagnosing Hallucination Risk in AI Surgical Decision-Support: A Sequential Framework for Sequential Validation |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| language | |
| locations[0].id | pmh:oai:arXiv.org:2511.00588 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2511.00588 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2511.00588 |
| locations[1].id | doi:10.48550/arxiv.2511.00588 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | cc-by |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | https://openalex.org/licenses/cc-by |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2511.00588 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5100319473 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-9496-1464 |
| authorships[0].author.display_name | Dong Chen |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Chen, Dong |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5101492317 |
| authorships[1].author.orcid | |
| authorships[1].author.display_name | Yumei Wei |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Wei, Yanzhe |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5114430031 |
| authorships[2].author.orcid | https://orcid.org/0009-0007-6280-8653 |
| authorships[2].author.display_name | Zhiyang He |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | He, Zonglin |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5017504070 |
| authorships[3].author.orcid | https://orcid.org/0000-0003-4976-4151 |
| authorships[3].author.display_name | Guan‐Ming Kuang |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Kuang, Guan-Ming |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5110870535 |
| authorships[4].author.orcid | https://orcid.org/0009-0002-9755-4335 |
| authorships[4].author.display_name | Chao Ye |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Ye, Canhua |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5015529510 |
| authorships[5].author.orcid | https://orcid.org/0009-0004-1603-5248 |
| authorships[5].author.display_name | M. R. An |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | An, Meiru |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5101463302 |
| authorships[6].author.orcid | https://orcid.org/0000-0002-9794-9225 |
| authorships[6].author.display_name | Hui Peng |
| authorships[6].author_position | middle |
| authorships[6].raw_author_name | Peng, Huili |
| authorships[6].is_corresponding | False |
| authorships[7].author.id | https://openalex.org/A5042083053 |
| authorships[7].author.orcid | https://orcid.org/0000-0003-0305-5616 |
| authorships[7].author.display_name | Yong Hu |
| authorships[7].author_position | middle |
| authorships[7].raw_author_name | Hu, Yong |
| authorships[7].is_corresponding | False |
| authorships[8].author.id | https://openalex.org/A5045656085 |
| authorships[8].author.orcid | https://orcid.org/0000-0002-6373-2968 |
| authorships[8].author.display_name | Huiren Tao |
| authorships[8].author_position | middle |
| authorships[8].raw_author_name | Tao, Huiren |
| authorships[8].is_corresponding | False |
| authorships[9].author.id | https://openalex.org/A5033243610 |
| authorships[9].author.orcid | https://orcid.org/0000-0003-2387-8011 |
| authorships[9].author.display_name | Kenneth Cheung |
| authorships[9].author_position | last |
| authorships[9].raw_author_name | Cheung, Kenneth MC |
| authorships[9].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2511.00588 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-11-05T00:00:00 |
| display_name | Diagnosing Hallucination Risk in AI Surgical Decision-Support: A Sequential Framework for Sequential Validation |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-07T23:20:04.922697 |
| primary_topic | |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2511.00588 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2511.00588 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2511.00588 |
| primary_location.id | pmh:oai:arXiv.org:2511.00588 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2511.00588 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2511.00588 |
| publication_date | 2025-11-01 |
| publication_year | 2025 |
| referenced_works_count | 0 |
| abstract_inverted_index.A | 85 |
| abstract_inverted_index.a | 36, 155, 179 |
| abstract_inverted_index.30 | 62 |
| abstract_inverted_index.We | 56 |
| abstract_inverted_index.as | 81 |
| abstract_inverted_index.by | 43, 135 |
| abstract_inverted_index.in | 11, 77, 146 |
| abstract_inverted_index.is | 121 |
| abstract_inverted_index.or | 24 |
| abstract_inverted_index.to | 39, 105 |
| abstract_inverted_index.LLM | 185 |
| abstract_inverted_index.Our | 164 |
| abstract_inverted_index.and | 53, 83, 151, 161, 177 |
| abstract_inverted_index.are | 21 |
| abstract_inverted_index.but | 14 |
| abstract_inverted_index.did | 93 |
| abstract_inverted_index.for | 7, 123, 183 |
| abstract_inverted_index.its | 106 |
| abstract_inverted_index.may | 29 |
| abstract_inverted_index.not | 94 |
| abstract_inverted_index.six | 58 |
| abstract_inverted_index.vs. | 112 |
| abstract_inverted_index.1.83 | 111 |
| abstract_inverted_index.7.4% | 136 |
| abstract_inverted_index.LLMs | 60 |
| abstract_inverted_index.This | 33, 140 |
| abstract_inverted_index.into | 174 |
| abstract_inverted_index.mode | 102 |
| abstract_inverted_index.pose | 15 |
| abstract_inverted_index.such | 80 |
| abstract_inverted_index.that | 28, 89 |
| abstract_inverted_index.with | 131, 143 |
| abstract_inverted_index.$\pm$ | 74, 110, 114 |
| abstract_inverted_index.81.56 | 113 |
| abstract_inverted_index.86.03 | 73 |
| abstract_inverted_index.Large | 0 |
| abstract_inverted_index.alone | 120 |
| abstract_inverted_index.chain | 172 |
| abstract_inverted_index.model | 91 |
| abstract_inverted_index.offer | 4 |
| abstract_inverted_index.risks | 17, 42 |
| abstract_inverted_index.spine | 12 |
| abstract_inverted_index.study | 34 |
| abstract_inverted_index.under | 137 |
| abstract_inverted_index.which | 20 |
| abstract_inverted_index.(80.79 | 109 |
| abstract_inverted_index.(LLMs) | 3 |
| abstract_inverted_index.(e.g., | 170 |
| abstract_inverted_index.(total | 71 |
| abstract_inverted_index.1.92), | 115 |
| abstract_inverted_index.2.08), | 75 |
| abstract_inverted_index.across | 61 |
| abstract_inverted_index.cases. | 65 |
| abstract_inverted_index.models | 2 |
| abstract_inverted_index.output | 51 |
| abstract_inverted_index.score: | 72 |
| abstract_inverted_index.spinal | 64 |
| abstract_inverted_index.trauma | 82 |
| abstract_inverted_index.(+1.7%) | 150 |
| abstract_inverted_index.between | 158 |
| abstract_inverted_index.decline | 141 |
| abstract_inverted_index.domains | 79 |
| abstract_inverted_index.exposed | 128 |
| abstract_inverted_index.finding | 87 |
| abstract_inverted_index.leading | 59 |
| abstract_inverted_index.outputs | 27 |
| abstract_inverted_index.overall | 69 |
| abstract_inverted_index.patient | 31 |
| abstract_inverted_index.quality | 133 |
| abstract_inverted_index.reveals | 88 |
| abstract_inverted_index.safety. | 32 |
| abstract_inverted_index.support | 10 |
| abstract_inverted_index.surgery | 13 |
| abstract_inverted_index.through | 18 |
| abstract_inverted_index.version | 108 |
| abstract_inverted_index.(+2.0%), | 148 |
| abstract_inverted_index.(+4.7%), | 153 |
| abstract_inverted_index.advocate | 166 |
| abstract_inverted_index.assessed | 57 |
| abstract_inverted_index.clinical | 8, 124, 175 |
| abstract_inverted_index.critical | 86 |
| abstract_inverted_index.decision | 9 |
| abstract_inverted_index.extended | 100, 117 |
| abstract_inverted_index.findings | 165 |
| abstract_inverted_index.language | 1 |
| abstract_inverted_index.marginal | 144 |
| abstract_inverted_index.quality, | 48 |
| abstract_inverted_index.quantify | 40 |
| abstract_inverted_index.relative | 104 |
| abstract_inverted_index.standard | 97, 107 |
| abstract_inverted_index.superior | 68 |
| abstract_inverted_index.surgical | 184 |
| abstract_inverted_index.thinking | 101 |
| abstract_inverted_index.variants | 92 |
| abstract_inverted_index.amplified | 138 |
| abstract_inverted_index.coherence | 160 |
| abstract_inverted_index.degrading | 134 |
| abstract_inverted_index.diagnosis | 152 |
| abstract_inverted_index.establish | 178 |
| abstract_inverted_index.factually | 22 |
| abstract_inverted_index.framework | 38, 182 |
| abstract_inverted_index.guidance. | 163 |
| abstract_inverted_index.knowledge | 54 |
| abstract_inverted_index.perceived | 159 |
| abstract_inverted_index.potential | 6 |
| abstract_inverted_index.reasoning | 49, 119, 171 |
| abstract_inverted_index.uniformly | 95 |
| abstract_inverted_index.workflows | 176 |
| abstract_inverted_index.actionable | 162 |
| abstract_inverted_index.alignment. | 55 |
| abstract_inverted_index.coherence, | 52 |
| abstract_inverted_index.compromise | 30 |
| abstract_inverted_index.concerning | 156 |
| abstract_inverted_index.contrasted | 142 |
| abstract_inverted_index.diagnostic | 45 |
| abstract_inverted_index.divergence | 157 |
| abstract_inverted_index.evaluating | 44 |
| abstract_inverted_index.indicating | 116 |
| abstract_inverted_index.infection. | 84 |
| abstract_inverted_index.introduces | 35 |
| abstract_inverted_index.mechanisms | 169 |
| abstract_inverted_index.misaligned | 26 |
| abstract_inverted_index.outperform | 96 |
| abstract_inverted_index.precision, | 46 |
| abstract_inverted_index.validation | 181 |
| abstract_inverted_index.DeepSeek-R1 | 66 |
| abstract_inverted_index.complexity. | 139 |
| abstract_inverted_index.deployment. | 186 |
| abstract_inverted_index.high-stakes | 78 |
| abstract_inverted_index.integrating | 167 |
| abstract_inverted_index.performance | 70 |
| abstract_inverted_index.rationality | 147 |
| abstract_inverted_index.readability | 149 |
| abstract_inverted_index.robustness, | 50 |
| abstract_inverted_index.significant | 16 |
| abstract_inverted_index.contextually | 25 |
| abstract_inverted_index.demonstrated | 67 |
| abstract_inverted_index.highlighting | 154 |
| abstract_inverted_index.improvements | 145 |
| abstract_inverted_index.inconsistent | 23 |
| abstract_inverted_index.insufficient | 122 |
| abstract_inverted_index.particularly | 76 |
| abstract_inverted_index.reliability. | 125 |
| abstract_inverted_index.safety-aware | 180 |
| abstract_inverted_index.counterparts: | 98 |
| abstract_inverted_index.hallucination | 41 |
| abstract_inverted_index.model-specific | 129 |
| abstract_inverted_index.recommendation | 47, 132 |
| abstract_inverted_index.stress-testing | 127 |
| abstract_inverted_index.transformative | 5 |
| abstract_inverted_index.underperformed | 103 |
| abstract_inverted_index.visualization) | 173 |
| abstract_inverted_index.hallucinations, | 19 |
| abstract_inverted_index.Multidimensional | 126 |
| abstract_inverted_index.chain-of-thought | 118 |
| abstract_inverted_index.expert-validated | 63 |
| abstract_inverted_index.interpretability | 168 |
| abstract_inverted_index.vulnerabilities, | 130 |
| abstract_inverted_index.clinician-centered | 37 |
| abstract_inverted_index.reasoning-enhanced | 90 |
| abstract_inverted_index.Claude-3.7-Sonnet's | 99 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 10 |
| citation_normalized_percentile |