Few-shot clinical entity recognition in English, French and Spanish: masked language models outperform generative model prompting Article Swipe

MAKE IMPACT

Marco Naguib , Xavier Tannier , Aurélie Névéol · YOU? · · 2024 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2402.12801

Large language models (LLMs) have become the preferred solution for many natural language processing tasks. In low-resource environments such as specialized domains, their few-shot capabilities are expected to deliver high performance. Named Entity Recognition (NER) is a critical task in information extraction that is not covered in recent LLM benchmarks. There is a need for better understanding the performance of LLMs for NER in a variety of settings including languages other than English. This study aims to evaluate generative LLMs, employed through prompt engineering, for few-shot clinical NER. %from the perspective of F1 performance and environmental impact. We compare 13 auto-regressive models using prompting and 16 masked models using fine-tuning on 14 NER datasets covering English, French and Spanish. While prompt-based auto-regressive models achieve competitive F1 for general NER, they are outperformed within the clinical domain by lighter biLSTM-CRF taggers based on masked models. Additionally, masked models exhibit lower environmental impact compared to auto-regressive models. Findings are consistent across the three languages studied, which suggests that LLM prompting is not yet suited for NER production in the clinical domain.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2402.12801

PDF

https://arxiv.org/pdf/2402.12801

OA Status

green

Cited By

1

Related Works

10

OpenAlex ID

https://openalex.org/W4392020057

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4392020057

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2402.12801

Digital Object Identifier

Title

Few-shot clinical entity recognition in English, French and Spanish: masked language models outperform generative model prompting

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2024

Year of publication

Publication date

2024-02-20

Full publication date if available

Authors

Marco Naguib, Xavier Tannier, Aurélie Névéol

List of authors in order

Landing page

https://arxiv.org/abs/2402.12801

Publisher landing page

PDF URL

https://arxiv.org/pdf/2402.12801

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2402.12801

Direct OA link when available

Concepts

Computer science, Natural language processing, Shot (pellet), Speech recognition, Linguistics, Artificial intelligence, Philosophy, Chemistry, Organic chemistry

Top concepts (fields/topics) attached by OpenAlex

Cited by

1

Total citation count in OpenAlex

Citations by year (recent)

2024: 1

Per-year citation counts (last 5 years)

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload