Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models Article Swipe

PDF

Huaixiu Zheng , Swaroop Mishra , Xinyun Chen , Heng-Tze Cheng , Ed H. , Quoc V. Le , Denny Zhou ·

YOU? · · 2023 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2310.06117

We present Step-Back Prompting, a simple prompting technique that enables LLMs to do abstractions to derive high-level concepts and first principles from instances containing specific details. Using the concepts and principles to guide reasoning, LLMs significantly improve their abilities in following a correct reasoning path towards the solution. We conduct experiments of Step-Back Prompting with PaLM-2L, GPT-4 and Llama2-70B models, and observe substantial performance gains on various challenging reasoning-intensive tasks including STEM, Knowledge QA, and Multi-Hop Reasoning. For instance, Step-Back Prompting improves PaLM-2L performance on MMLU (Physics and Chemistry) by 7% and 11% respectively, TimeQA by 27%, and MuSiQue by 7%.

Related Topics

Computer Science

Programming Language

Artificial Intelligence

Epistemology

Philosophy

Concepts

Abstraction Computer science Programming language Natural language processing Cognitive science Artificial intelligence Epistemology Psychology Philosophy

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2310.06117
PDF: https://arxiv.org/pdf/2310.06117
OA Status: green
Cited By: 20
Related Works: 10
OpenAlex ID: https://openalex.org/W4387560733

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4387560733

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2310.06117

Digital Object Identifier
Title: Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2023

Year of publication
Publication date: 2023-10-09

Full publication date if available
Authors: Huaixiu Zheng, Swaroop Mishra, Xinyun Chen, Heng-Tze Cheng, Ed H., Quoc V. Le, Denny Zhou

List of authors in order
Landing page: https://arxiv.org/abs/2310.06117

Publisher landing page
PDF URL: https://arxiv.org/pdf/2310.06117

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2310.06117

Direct OA link when available
Concepts: Abstraction, Computer science, Programming language, Natural language processing, Cognitive science, Artificial intelligence, Epistemology, Psychology, Philosophy

Top concepts (fields/topics) attached by OpenAlex
Cited by: 20

Total citation count in OpenAlex
Citations by year (recent): 2025: 12, 2024: 7, 2023: 1

Per-year citation counts (last 5 years)
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4387560733
doi	https://doi.org/10.48550/arxiv.2310.06117
ids.doi	https://doi.org/10.48550/arxiv.2310.06117
ids.openalex	https://openalex.org/W4387560733
fwci
type	preprint
title	Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T10028
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	0.9853000044822693
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1702
topics[0].subfield.display_name	Artificial Intelligence
topics[0].display_name	Topic Modeling
topics[1].id	https://openalex.org/T10181
topics[1].field.id	https://openalex.org/fields/17
topics[1].field.display_name	Computer Science
topics[1].score	0.9828000068664551
topics[1].domain.id	https://openalex.org/domains/3
topics[1].domain.display_name	Physical Sciences
topics[1].subfield.id	https://openalex.org/subfields/1702
topics[1].subfield.display_name	Artificial Intelligence
topics[1].display_name	Natural Language Processing Techniques
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C124304363
concepts[0].level	2
concepts[0].score	0.8015742897987366
concepts[0].wikidata	https://www.wikidata.org/wiki/Q673661
concepts[0].display_name	Abstraction
concepts[1].id	https://openalex.org/C41008148
concepts[1].level	0
concepts[1].score	0.5929064750671387
concepts[1].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[1].display_name	Computer science
concepts[2].id	https://openalex.org/C199360897
concepts[2].level	1
concepts[2].score	0.48612403869628906
concepts[2].wikidata	https://www.wikidata.org/wiki/Q9143
concepts[2].display_name	Programming language
concepts[3].id	https://openalex.org/C204321447
concepts[3].level	1
concepts[3].score	0.39655715227127075
concepts[3].wikidata	https://www.wikidata.org/wiki/Q30642
concepts[3].display_name	Natural language processing
concepts[4].id	https://openalex.org/C188147891
concepts[4].level	1
concepts[4].score	0.3935718834400177
concepts[4].wikidata	https://www.wikidata.org/wiki/Q147638
concepts[4].display_name	Cognitive science
concepts[5].id	https://openalex.org/C154945302
concepts[5].level	1
concepts[5].score	0.3926449716091156
concepts[5].wikidata	https://www.wikidata.org/wiki/Q11660
concepts[5].display_name	Artificial intelligence
concepts[6].id	https://openalex.org/C111472728
concepts[6].level	1
concepts[6].score	0.28263258934020996
concepts[6].wikidata	https://www.wikidata.org/wiki/Q9471
concepts[6].display_name	Epistemology
concepts[7].id	https://openalex.org/C15744967
concepts[7].level	0
concepts[7].score	0.19387337565422058
concepts[7].wikidata	https://www.wikidata.org/wiki/Q9418
concepts[7].display_name	Psychology
concepts[8].id	https://openalex.org/C138885662
concepts[8].level	0
concepts[8].score	0.14444252848625183
concepts[8].wikidata	https://www.wikidata.org/wiki/Q5891
concepts[8].display_name	Philosophy
keywords[0].id	https://openalex.org/keywords/abstraction
keywords[0].score	0.8015742897987366
keywords[0].display_name	Abstraction
keywords[1].id	https://openalex.org/keywords/computer-science
keywords[1].score	0.5929064750671387
keywords[1].display_name	Computer science
keywords[2].id	https://openalex.org/keywords/programming-language
keywords[2].score	0.48612403869628906
keywords[2].display_name	Programming language
keywords[3].id	https://openalex.org/keywords/natural-language-processing
keywords[3].score	0.39655715227127075
keywords[3].display_name	Natural language processing
keywords[4].id	https://openalex.org/keywords/cognitive-science
keywords[4].score	0.3935718834400177
keywords[4].display_name	Cognitive science
keywords[5].id	https://openalex.org/keywords/artificial-intelligence
keywords[5].score	0.3926449716091156
keywords[5].display_name	Artificial intelligence
keywords[6].id	https://openalex.org/keywords/epistemology
keywords[6].score	0.28263258934020996
keywords[6].display_name	Epistemology
keywords[7].id	https://openalex.org/keywords/psychology
keywords[7].score	0.19387337565422058
keywords[7].display_name	Psychology
keywords[8].id	https://openalex.org/keywords/philosophy
keywords[8].score	0.14444252848625183
keywords[8].display_name	Philosophy
language	en
locations[0].id	pmh:oai:arXiv.org:2310.06117
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/2310.06117
locations[0].version	submittedVersion
locations[0].raw_type	text
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2310.06117
locations[1].id	doi:10.48550/arxiv.2310.06117
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license	cc-by
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id	https://openalex.org/licenses/cc-by
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2310.06117
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5003807260
authorships[0].author.orcid
authorships[0].author.display_name	Huaixiu Zheng
authorships[0].author_position	first
authorships[0].raw_author_name	Zheng, Huaixiu Steven
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5063722751
authorships[1].author.orcid	https://orcid.org/0009-0001-6413-7001
authorships[1].author.display_name	Swaroop Mishra
authorships[1].author_position	middle
authorships[1].raw_author_name	Mishra, Swaroop
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5101977008
authorships[2].author.orcid	https://orcid.org/0009-0001-9700-8951
authorships[2].author.display_name	Xinyun Chen
authorships[2].author_position	middle
authorships[2].raw_author_name	Chen, Xinyun
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5054880622
authorships[3].author.orcid	https://orcid.org/0009-0007-3845-8796
authorships[3].author.display_name	Heng-Tze Cheng
authorships[3].author_position	middle
authorships[3].raw_author_name	Cheng, Heng-Tze
authorships[3].is_corresponding	False
authorships[4].author.id	https://openalex.org/A5028125399
authorships[4].author.orcid	https://orcid.org/0000-0003-3230-5338
authorships[4].author.display_name	Ed H.
authorships[4].author_position	middle
authorships[4].raw_author_name	Chi, Ed H.
authorships[4].is_corresponding	False
authorships[5].author.id	https://openalex.org/A5088551093
authorships[5].author.orcid	https://orcid.org/0000-0002-1087-2844
authorships[5].author.display_name	Quoc V. Le
authorships[5].author_position	middle
authorships[5].raw_author_name	Le, Quoc V
authorships[5].is_corresponding	False
authorships[6].author.id	https://openalex.org/A5061512999
authorships[6].author.orcid
authorships[6].author.display_name	Denny Zhou
authorships[6].author_position	last
authorships[6].raw_author_name	Zhou, Denny
authorships[6].is_corresponding	False
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2310.06117
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2023-10-12T00:00:00
display_name	Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T06:51:31.235846
primary_topic.id	https://openalex.org/T10028
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	0.9853000044822693
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1702
primary_topic.subfield.display_name	Artificial Intelligence
primary_topic.display_name	Topic Modeling
related_works	https://openalex.org/W2045155990, https://openalex.org/W4313163053, https://openalex.org/W3045811229, https://openalex.org/W4300973204, https://openalex.org/W1483408780, https://openalex.org/W4284884309, https://openalex.org/W2908749798, https://openalex.org/W2105980483, https://openalex.org/W4243842598, https://openalex.org/W3204019825
cited_by_count	20
counts_by_year[0].year	2025
counts_by_year[0].cited_by_count	12
counts_by_year[1].year	2024
counts_by_year[1].cited_by_count	7
counts_by_year[2].year	2023
counts_by_year[2].cited_by_count	1
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2310.06117
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/2310.06117
best_oa_location.version	submittedVersion
best_oa_location.raw_type	text
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2310.06117
primary_location.id	pmh:oai:arXiv.org:2310.06117
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/2310.06117
primary_location.version	submittedVersion
primary_location.raw_type	text
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2310.06117
publication_date	2023-10-09
publication_year	2023
referenced_works_count	0
abstract_inverted_index.a	4, 41
abstract_inverted_index.7%	90
abstract_inverted_index.We	0, 48
abstract_inverted_index.by	89, 95, 99
abstract_inverted_index.do	12
abstract_inverted_index.in	39
abstract_inverted_index.of	51
abstract_inverted_index.on	65, 84
abstract_inverted_index.to	11, 14, 31
abstract_inverted_index.11%	92
abstract_inverted_index.7%.	100
abstract_inverted_index.For	77
abstract_inverted_index.QA,	73
abstract_inverted_index.and	18, 29, 57, 60, 74, 87, 91, 97
abstract_inverted_index.the	27, 46
abstract_inverted_index.27%,	96
abstract_inverted_index.LLMs	10, 34
abstract_inverted_index.MMLU	85
abstract_inverted_index.from	21
abstract_inverted_index.path	44
abstract_inverted_index.that	8
abstract_inverted_index.with	54
abstract_inverted_index.GPT-4	56
abstract_inverted_index.STEM,	71
abstract_inverted_index.Using	26
abstract_inverted_index.first	19
abstract_inverted_index.gains	64
abstract_inverted_index.guide	32
abstract_inverted_index.tasks	69
abstract_inverted_index.their	37
abstract_inverted_index.TimeQA	94
abstract_inverted_index.derive	15
abstract_inverted_index.simple	5
abstract_inverted_index.MuSiQue	98
abstract_inverted_index.PaLM-2L	82
abstract_inverted_index.conduct	49
abstract_inverted_index.correct	42
abstract_inverted_index.enables	9
abstract_inverted_index.improve	36
abstract_inverted_index.models,	59
abstract_inverted_index.observe	61
abstract_inverted_index.present	1
abstract_inverted_index.towards	45
abstract_inverted_index.various	66
abstract_inverted_index.(Physics	86
abstract_inverted_index.PaLM-2L,	55
abstract_inverted_index.concepts	17, 28
abstract_inverted_index.details.	25
abstract_inverted_index.improves	81
abstract_inverted_index.specific	24
abstract_inverted_index.Knowledge	72
abstract_inverted_index.Multi-Hop	75
abstract_inverted_index.Prompting	53, 80
abstract_inverted_index.Step-Back	2, 52, 79
abstract_inverted_index.abilities	38
abstract_inverted_index.following	40
abstract_inverted_index.including	70
abstract_inverted_index.instance,	78
abstract_inverted_index.instances	22
abstract_inverted_index.prompting	6
abstract_inverted_index.reasoning	43
abstract_inverted_index.solution.	47
abstract_inverted_index.technique	7
abstract_inverted_index.Chemistry)	88
abstract_inverted_index.Llama2-70B	58
abstract_inverted_index.Prompting,	3
abstract_inverted_index.Reasoning.	76
abstract_inverted_index.containing	23
abstract_inverted_index.high-level	16
abstract_inverted_index.principles	20, 30
abstract_inverted_index.reasoning,	33
abstract_inverted_index.challenging	67
abstract_inverted_index.experiments	50
abstract_inverted_index.performance	63, 83
abstract_inverted_index.substantial	62
abstract_inverted_index.abstractions	13
abstract_inverted_index.respectively,	93
abstract_inverted_index.significantly	35
abstract_inverted_index.reasoning-intensive	68
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	7
sustainable_development_goals[0].id	https://metadata.un.org/sdg/4
sustainable_development_goals[0].score	0.6800000071525574
sustainable_development_goals[0].display_name	Quality Education
citation_normalized_percentile