Sources of Hallucination by Large Language Models on Inference Tasks Article Swipe

MAKE IMPACT

Nick McKenna , Tianyi Li , Liang Cheng , Mohammad Javad Hosseini , Mark Johnson , Mark Steedman · YOU? · · 2023 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2305.14552

Large Language Models (LLMs) are claimed to be capable of Natural Language Inference (NLI), necessary for applied tasks like question answering and summarization. We present a series of behavioral studies on several LLM families (LLaMA, GPT-3.5, and PaLM) which probe their behavior using controlled experiments. We establish two biases originating from pretraining which predict much of their behavior, and show that these are major sources of hallucination in generative LLMs. First, memorization at the level of sentences: we show that, regardless of the premise, models falsely label NLI test samples as entailing when the hypothesis is attested in training data, and that entities are used as ``indices'' to access the memorized data. Second, statistical patterns of usage learned at the level of corpora: we further show a similar effect when the premise predicate is less frequent than that of the hypothesis in the training data, a bias following from previous studies. We demonstrate that LLMs perform significantly worse on NLI test samples which do not conform to these biases than those which do, and we offer these as valuable controls for future LLM evaluation.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2305.14552

PDF

https://arxiv.org/pdf/2305.14552

OA Status

green

Cited By

17

Related Works

10

OpenAlex ID

https://openalex.org/W4378509483

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4378509483

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2305.14552

Digital Object Identifier

Title

Sources of Hallucination by Large Language Models on Inference Tasks

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2023

Year of publication

Publication date

2023-05-23

Full publication date if available

Authors

Nick McKenna, Tianyi Li, Liang Cheng, Mohammad Javad Hosseini, Mark Johnson, Mark Steedman

List of authors in order

Landing page

https://arxiv.org/abs/2305.14552

Publisher landing page

PDF URL

https://arxiv.org/pdf/2305.14552

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2305.14552

Direct OA link when available

Concepts

Inference, Premise, Computer science, Automatic summarization, Memorization, Predicate (mathematical logic), Artificial intelligence, Natural language processing, Test (biology), Sentence, Cognitive psychology, Psychology, Generative model, Generative grammar, Linguistics, Philosophy, Programming language, Biology, Paleontology

Top concepts (fields/topics) attached by OpenAlex

Cited by

17

Total citation count in OpenAlex

Citations by year (recent)

2025: 3, 2024: 10, 2023: 4

Per-year citation counts (last 5 years)

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload