Diversity of Thought Improves Reasoning Abilities of LLMs Article Swipe
Ranjita Naik
,
Varun Chandrasekaran
,
Mert Yüksekgönül
,
Hamid Palangi
,
Besmira Nushi
·
YOU?
·
· 2023
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2310.07088
YOU?
·
· 2023
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2310.07088
Large language models (LLMs) are documented to struggle in settings that require complex reasoning. Nevertheless, instructing the model to break down the problem into smaller reasoning steps, or ensembling various generations through modifying decoding steps boosts performance. However, these methods assume that the input prompt is fixed and expect the decoding strategies to introduce the diversity needed for ensembling. In this work, we discuss how one can create and leverage variations of the input prompt as a means of diversity of thought. We propose a method that automatically improves prompt diversity by soliciting feedback from the LLM to ideate approaches that are apt for the problem. We then ensemble the diverse prompts in our method DIVSE (DIVerse reasoning path Self-Ensemble) across multiple inference calls, or use diverse approaches within a single inference call; we call the latter IDIV-SE (In-call DIVerse reasoning path Self-Ensemble). Apart from our approaches outperforming prior work, DIV-SE(in particular) advances state-of-the-art performance on the challenging planning and graph coloring benchmarks. Our results improve the Pareto frontier of the accuracy-cost trade-off.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2310.07088
- https://arxiv.org/pdf/2310.07088
- OA Status
- green
- Cited By
- 3
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4387595998
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4387595998Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2310.07088Digital Object Identifier
- Title
-
Diversity of Thought Improves Reasoning Abilities of LLMsWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2023Year of publication
- Publication date
-
2023-10-11Full publication date if available
- Authors
-
Ranjita Naik, Varun Chandrasekaran, Mert Yüksekgönül, Hamid Palangi, Besmira NushiList of authors in order
- Landing page
-
https://arxiv.org/abs/2310.07088Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2310.07088Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2310.07088Direct OA link when available
- Concepts
-
Computer science, Leverage (statistics), Inference, Path (computing), Diversity (politics), Artificial intelligence, Graph, Machine learning, Theoretical computer science, Sociology, Programming language, AnthropologyTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
3Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 1, 2024: 2Per-year citation counts (last 5 years)
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4387595998 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2310.07088 |
| ids.doi | https://doi.org/10.48550/arxiv.2310.07088 |
| ids.openalex | https://openalex.org/W4387595998 |
| fwci | |
| type | preprint |
| title | Diversity of Thought Improves Reasoning Abilities of LLMs |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10028 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9991000294685364 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Topic Modeling |
| topics[1].id | https://openalex.org/T10181 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9968000054359436 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Natural Language Processing Techniques |
| topics[2].id | https://openalex.org/T11714 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9830999970436096 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1707 |
| topics[2].subfield.display_name | Computer Vision and Pattern Recognition |
| topics[2].display_name | Multimodal Machine Learning Applications |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C41008148 |
| concepts[0].level | 0 |
| concepts[0].score | 0.7111071348190308 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[0].display_name | Computer science |
| concepts[1].id | https://openalex.org/C153083717 |
| concepts[1].level | 2 |
| concepts[1].score | 0.6728684902191162 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q6535263 |
| concepts[1].display_name | Leverage (statistics) |
| concepts[2].id | https://openalex.org/C2776214188 |
| concepts[2].level | 2 |
| concepts[2].score | 0.6249194145202637 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q408386 |
| concepts[2].display_name | Inference |
| concepts[3].id | https://openalex.org/C2777735758 |
| concepts[3].level | 2 |
| concepts[3].score | 0.5017549991607666 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q817765 |
| concepts[3].display_name | Path (computing) |
| concepts[4].id | https://openalex.org/C2781316041 |
| concepts[4].level | 2 |
| concepts[4].score | 0.4892234802246094 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q1230584 |
| concepts[4].display_name | Diversity (politics) |
| concepts[5].id | https://openalex.org/C154945302 |
| concepts[5].level | 1 |
| concepts[5].score | 0.46247756481170654 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[5].display_name | Artificial intelligence |
| concepts[6].id | https://openalex.org/C132525143 |
| concepts[6].level | 2 |
| concepts[6].score | 0.43256402015686035 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q141488 |
| concepts[6].display_name | Graph |
| concepts[7].id | https://openalex.org/C119857082 |
| concepts[7].level | 1 |
| concepts[7].score | 0.40280774235725403 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q2539 |
| concepts[7].display_name | Machine learning |
| concepts[8].id | https://openalex.org/C80444323 |
| concepts[8].level | 1 |
| concepts[8].score | 0.29862678050994873 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q2878974 |
| concepts[8].display_name | Theoretical computer science |
| concepts[9].id | https://openalex.org/C144024400 |
| concepts[9].level | 0 |
| concepts[9].score | 0.10488459467887878 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q21201 |
| concepts[9].display_name | Sociology |
| concepts[10].id | https://openalex.org/C199360897 |
| concepts[10].level | 1 |
| concepts[10].score | 0.08849659562110901 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q9143 |
| concepts[10].display_name | Programming language |
| concepts[11].id | https://openalex.org/C19165224 |
| concepts[11].level | 1 |
| concepts[11].score | 0.0 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q23404 |
| concepts[11].display_name | Anthropology |
| keywords[0].id | https://openalex.org/keywords/computer-science |
| keywords[0].score | 0.7111071348190308 |
| keywords[0].display_name | Computer science |
| keywords[1].id | https://openalex.org/keywords/leverage |
| keywords[1].score | 0.6728684902191162 |
| keywords[1].display_name | Leverage (statistics) |
| keywords[2].id | https://openalex.org/keywords/inference |
| keywords[2].score | 0.6249194145202637 |
| keywords[2].display_name | Inference |
| keywords[3].id | https://openalex.org/keywords/path |
| keywords[3].score | 0.5017549991607666 |
| keywords[3].display_name | Path (computing) |
| keywords[4].id | https://openalex.org/keywords/diversity |
| keywords[4].score | 0.4892234802246094 |
| keywords[4].display_name | Diversity (politics) |
| keywords[5].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[5].score | 0.46247756481170654 |
| keywords[5].display_name | Artificial intelligence |
| keywords[6].id | https://openalex.org/keywords/graph |
| keywords[6].score | 0.43256402015686035 |
| keywords[6].display_name | Graph |
| keywords[7].id | https://openalex.org/keywords/machine-learning |
| keywords[7].score | 0.40280774235725403 |
| keywords[7].display_name | Machine learning |
| keywords[8].id | https://openalex.org/keywords/theoretical-computer-science |
| keywords[8].score | 0.29862678050994873 |
| keywords[8].display_name | Theoretical computer science |
| keywords[9].id | https://openalex.org/keywords/sociology |
| keywords[9].score | 0.10488459467887878 |
| keywords[9].display_name | Sociology |
| keywords[10].id | https://openalex.org/keywords/programming-language |
| keywords[10].score | 0.08849659562110901 |
| keywords[10].display_name | Programming language |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2310.07088 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2310.07088 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2310.07088 |
| locations[1].id | doi:10.48550/arxiv.2310.07088 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | cc-by |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | https://openalex.org/licenses/cc-by |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2310.07088 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5068870408 |
| authorships[0].author.orcid | https://orcid.org/0009-0001-4120-5251 |
| authorships[0].author.display_name | Ranjita Naik |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Naik, Ranjita |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5009092703 |
| authorships[1].author.orcid | https://orcid.org/0000-0003-0661-6378 |
| authorships[1].author.display_name | Varun Chandrasekaran |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Chandrasekaran, Varun |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5061433075 |
| authorships[2].author.orcid | https://orcid.org/0000-0002-9761-8178 |
| authorships[2].author.display_name | Mert Yüksekgönül |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Yuksekgonul, Mert |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5033846851 |
| authorships[3].author.orcid | https://orcid.org/0000-0003-2912-4579 |
| authorships[3].author.display_name | Hamid Palangi |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Palangi, Hamid |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5011998621 |
| authorships[4].author.orcid | https://orcid.org/0000-0002-7554-8586 |
| authorships[4].author.display_name | Besmira Nushi |
| authorships[4].author_position | last |
| authorships[4].raw_author_name | Nushi, Besmira |
| authorships[4].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2310.07088 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2023-10-13T00:00:00 |
| display_name | Diversity of Thought Improves Reasoning Abilities of LLMs |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10028 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9991000294685364 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Topic Modeling |
| related_works | https://openalex.org/W2055243143, https://openalex.org/W2787993192, https://openalex.org/W2158269427, https://openalex.org/W4381280689, https://openalex.org/W2847365777, https://openalex.org/W4321636575, https://openalex.org/W1986418932, https://openalex.org/W2357796999, https://openalex.org/W4236583520, https://openalex.org/W4379251913 |
| cited_by_count | 3 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 1 |
| counts_by_year[1].year | 2024 |
| counts_by_year[1].cited_by_count | 2 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2310.07088 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2310.07088 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2310.07088 |
| primary_location.id | pmh:oai:arXiv.org:2310.07088 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2310.07088 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2310.07088 |
| publication_date | 2023-10-11 |
| publication_year | 2023 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 76, 84, 129 |
| abstract_inverted_index.In | 59 |
| abstract_inverted_index.We | 82, 106 |
| abstract_inverted_index.as | 75 |
| abstract_inverted_index.by | 91 |
| abstract_inverted_index.in | 8, 112 |
| abstract_inverted_index.is | 45 |
| abstract_inverted_index.of | 71, 78, 80, 169 |
| abstract_inverted_index.on | 155 |
| abstract_inverted_index.or | 27, 124 |
| abstract_inverted_index.to | 6, 18, 52, 97 |
| abstract_inverted_index.we | 62, 133 |
| abstract_inverted_index.LLM | 96 |
| abstract_inverted_index.Our | 163 |
| abstract_inverted_index.and | 47, 68, 159 |
| abstract_inverted_index.apt | 102 |
| abstract_inverted_index.are | 4, 101 |
| abstract_inverted_index.can | 66 |
| abstract_inverted_index.for | 57, 103 |
| abstract_inverted_index.how | 64 |
| abstract_inverted_index.one | 65 |
| abstract_inverted_index.our | 113, 145 |
| abstract_inverted_index.the | 16, 21, 42, 49, 54, 72, 95, 104, 109, 135, 156, 166, 170 |
| abstract_inverted_index.use | 125 |
| abstract_inverted_index.call | 134 |
| abstract_inverted_index.down | 20 |
| abstract_inverted_index.from | 94, 144 |
| abstract_inverted_index.into | 23 |
| abstract_inverted_index.path | 118, 141 |
| abstract_inverted_index.that | 10, 41, 86, 100 |
| abstract_inverted_index.then | 107 |
| abstract_inverted_index.this | 60 |
| abstract_inverted_index.Apart | 143 |
| abstract_inverted_index.DIVSE | 115 |
| abstract_inverted_index.Large | 0 |
| abstract_inverted_index.break | 19 |
| abstract_inverted_index.call; | 132 |
| abstract_inverted_index.fixed | 46 |
| abstract_inverted_index.graph | 160 |
| abstract_inverted_index.input | 43, 73 |
| abstract_inverted_index.means | 77 |
| abstract_inverted_index.model | 17 |
| abstract_inverted_index.prior | 148 |
| abstract_inverted_index.steps | 34 |
| abstract_inverted_index.these | 38 |
| abstract_inverted_index.work, | 61, 149 |
| abstract_inverted_index.(LLMs) | 3 |
| abstract_inverted_index.Pareto | 167 |
| abstract_inverted_index.across | 120 |
| abstract_inverted_index.assume | 40 |
| abstract_inverted_index.boosts | 35 |
| abstract_inverted_index.calls, | 123 |
| abstract_inverted_index.create | 67 |
| abstract_inverted_index.expect | 48 |
| abstract_inverted_index.ideate | 98 |
| abstract_inverted_index.latter | 136 |
| abstract_inverted_index.method | 85, 114 |
| abstract_inverted_index.models | 2 |
| abstract_inverted_index.needed | 56 |
| abstract_inverted_index.prompt | 44, 74, 89 |
| abstract_inverted_index.single | 130 |
| abstract_inverted_index.steps, | 26 |
| abstract_inverted_index.within | 128 |
| abstract_inverted_index.DIVerse | 139 |
| abstract_inverted_index.IDIV-SE | 137 |
| abstract_inverted_index.complex | 12 |
| abstract_inverted_index.discuss | 63 |
| abstract_inverted_index.diverse | 110, 126 |
| abstract_inverted_index.improve | 165 |
| abstract_inverted_index.methods | 39 |
| abstract_inverted_index.problem | 22 |
| abstract_inverted_index.prompts | 111 |
| abstract_inverted_index.propose | 83 |
| abstract_inverted_index.require | 11 |
| abstract_inverted_index.results | 164 |
| abstract_inverted_index.smaller | 24 |
| abstract_inverted_index.through | 31 |
| abstract_inverted_index.various | 29 |
| abstract_inverted_index.(DIVerse | 116 |
| abstract_inverted_index.(In-call | 138 |
| abstract_inverted_index.However, | 37 |
| abstract_inverted_index.advances | 152 |
| abstract_inverted_index.coloring | 161 |
| abstract_inverted_index.decoding | 33, 50 |
| abstract_inverted_index.ensemble | 108 |
| abstract_inverted_index.feedback | 93 |
| abstract_inverted_index.frontier | 168 |
| abstract_inverted_index.improves | 88 |
| abstract_inverted_index.language | 1 |
| abstract_inverted_index.leverage | 69 |
| abstract_inverted_index.multiple | 121 |
| abstract_inverted_index.planning | 158 |
| abstract_inverted_index.problem. | 105 |
| abstract_inverted_index.settings | 9 |
| abstract_inverted_index.struggle | 7 |
| abstract_inverted_index.thought. | 81 |
| abstract_inverted_index.DIV-SE(in | 150 |
| abstract_inverted_index.diversity | 55, 79, 90 |
| abstract_inverted_index.inference | 122, 131 |
| abstract_inverted_index.introduce | 53 |
| abstract_inverted_index.modifying | 32 |
| abstract_inverted_index.reasoning | 25, 117, 140 |
| abstract_inverted_index.approaches | 99, 127, 146 |
| abstract_inverted_index.documented | 5 |
| abstract_inverted_index.ensembling | 28 |
| abstract_inverted_index.reasoning. | 13 |
| abstract_inverted_index.soliciting | 92 |
| abstract_inverted_index.strategies | 51 |
| abstract_inverted_index.trade-off. | 172 |
| abstract_inverted_index.variations | 70 |
| abstract_inverted_index.benchmarks. | 162 |
| abstract_inverted_index.challenging | 157 |
| abstract_inverted_index.ensembling. | 58 |
| abstract_inverted_index.generations | 30 |
| abstract_inverted_index.instructing | 15 |
| abstract_inverted_index.particular) | 151 |
| abstract_inverted_index.performance | 154 |
| abstract_inverted_index.performance. | 36 |
| abstract_inverted_index.Nevertheless, | 14 |
| abstract_inverted_index.accuracy-cost | 171 |
| abstract_inverted_index.automatically | 87 |
| abstract_inverted_index.outperforming | 147 |
| abstract_inverted_index.Self-Ensemble) | 119 |
| abstract_inverted_index.Self-Ensemble). | 142 |
| abstract_inverted_index.state-of-the-art | 153 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 5 |
| citation_normalized_percentile |