Towards Quantifying Commonsense Reasoning with Mechanistic Insights Article Swipe

PDF

Abhinav Joshi , Areeb Ahmad , D. K. Shukla , Ashutosh Modi ·

YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2504.10077

Commonsense reasoning deals with the implicit knowledge that is well understood by humans and typically acquired via interactions with the world. In recent times, commonsense reasoning and understanding of various LLMs have been evaluated using text-based tasks. In this work, we argue that a proxy of this understanding can be maintained as a graphical structure that can further help to perform a rigorous evaluation of commonsense reasoning abilities about various real-world activities. We create an annotation scheme for capturing this implicit knowledge in the form of a graphical structure for 37 daily human activities. We find that the created resource can be used to frame an enormous number of commonsense queries (~ 10^{17}), facilitating rigorous evaluation of commonsense reasoning in LLMs. Moreover, recently, the remarkable performance of LLMs has raised questions about whether these models are truly capable of reasoning in the wild and, in general, how reasoning occurs inside these models. In this resource paper, we bridge this gap by proposing design mechanisms that facilitate research in a similar direction. Our findings suggest that the reasoning components are localized in LLMs that play a prominent role in decision-making when prompted with a commonsense query.

Related Topics

Truth And Reconciliation Commission Of Canada

Reich Ministry Of Public Enlightenment And Propaganda

Rick Hurst

Mutual Aid

Concepts

No concepts available.

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2504.10077
PDF: https://arxiv.org/pdf/2504.10077
OA Status: green
OpenAlex ID: https://openalex.org/W4415159509

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4415159509

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2504.10077

Digital Object Identifier
Title: Towards Quantifying Commonsense Reasoning with Mechanistic Insights

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2025

Year of publication
Publication date: 2025-04-14

Full publication date if available
Authors: Abhinav Joshi, Areeb Ahmad, D. K. Shukla, Ashutosh Modi

List of authors in order
Landing page: https://arxiv.org/abs/2504.10077

Publisher landing page
PDF URL: https://arxiv.org/pdf/2504.10077

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2504.10077

Direct OA link when available
Cited by: 0

Total citation count in OpenAlex

Full payload

id	https://openalex.org/W4415159509
doi	https://doi.org/10.48550/arxiv.2504.10077
ids.doi	https://doi.org/10.48550/arxiv.2504.10077
ids.openalex	https://openalex.org/W4415159509
fwci
type	preprint
title	Towards Quantifying Commonsense Reasoning with Mechanistic Insights
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T11010
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	0.8651999831199646
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1702
topics[0].subfield.display_name	Artificial Intelligence
topics[0].display_name	Logic, Reasoning, and Knowledge
topics[1].id	https://openalex.org/T10215
topics[1].field.id	https://openalex.org/fields/17
topics[1].field.display_name	Computer Science
topics[1].score	0.8639000058174133
topics[1].domain.id	https://openalex.org/domains/3
topics[1].domain.display_name	Physical Sciences
topics[1].subfield.id	https://openalex.org/subfields/1702
topics[1].subfield.display_name	Artificial Intelligence
topics[1].display_name	Semantic Web and Ontologies
topics[2].id	https://openalex.org/T11303
topics[2].field.id	https://openalex.org/fields/17
topics[2].field.display_name	Computer Science
topics[2].score	0.8144000172615051
topics[2].domain.id	https://openalex.org/domains/3
topics[2].domain.display_name	Physical Sciences
topics[2].subfield.id	https://openalex.org/subfields/1702
topics[2].subfield.display_name	Artificial Intelligence
topics[2].display_name	Bayesian Modeling and Causal Inference
is_xpac	False
apc_list
apc_paid
language	en
locations[0].id	pmh:oai:arXiv.org:2504.10077
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license	cc-by-nc-sa
locations[0].pdf_url	https://arxiv.org/pdf/2504.10077
locations[0].version	submittedVersion
locations[0].raw_type	text
locations[0].license_id	https://openalex.org/licenses/cc-by-nc-sa
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2504.10077
locations[1].id	doi:10.48550/arxiv.2504.10077
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2504.10077
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5103253154
authorships[0].author.orcid	https://orcid.org/0000-0001-6756-1126
authorships[0].author.display_name	Abhinav Joshi
authorships[0].author_position	first
authorships[0].raw_author_name	Joshi, Abhinav
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5102533871
authorships[1].author.orcid
authorships[1].author.display_name	Areeb Ahmad
authorships[1].author_position	middle
authorships[1].raw_author_name	Ahmad, Areeb
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5031140914
authorships[2].author.orcid	https://orcid.org/0000-0001-8835-9269
authorships[2].author.display_name	D. K. Shukla
authorships[2].author_position	middle
authorships[2].raw_author_name	Shukla, Divyaksh
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5076043215
authorships[3].author.orcid	https://orcid.org/0000-0002-0962-8350
authorships[3].author.display_name	Ashutosh Modi
authorships[3].author_position	last
authorships[3].raw_author_name	Modi, Ashutosh
authorships[3].is_corresponding	False
has_content.pdf	True
has_content.grobid_xml	True
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2504.10077
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2025-10-14T00:00:00
display_name	Towards Quantifying Commonsense Reasoning with Mechanistic Insights
has_fulltext	True
is_retracted	False
updated_date	2025-11-06T06:51:31.235846
primary_topic.id	https://openalex.org/T11010
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	0.8651999831199646
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1702
primary_topic.subfield.display_name	Artificial Intelligence
primary_topic.display_name	Logic, Reasoning, and Knowledge
cited_by_count	0
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2504.10077
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license	cc-by-nc-sa
best_oa_location.pdf_url	https://arxiv.org/pdf/2504.10077
best_oa_location.version	submittedVersion
best_oa_location.raw_type	text
best_oa_location.license_id	https://openalex.org/licenses/cc-by-nc-sa
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2504.10077
primary_location.id	pmh:oai:arXiv.org:2504.10077
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license	cc-by-nc-sa
primary_location.pdf_url	https://arxiv.org/pdf/2504.10077
primary_location.version	submittedVersion
primary_location.raw_type	text
primary_location.license_id	https://openalex.org/licenses/cc-by-nc-sa
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2504.10077
publication_date	2025-04-14
publication_year	2025
referenced_works_count	0
abstract_inverted_index.a	43, 52, 61, 86, 168, 184, 192
abstract_inverted_index.(~	111
abstract_inverted_index.37	90
abstract_inverted_index.In	21, 37, 152
abstract_inverted_index.We	72, 94
abstract_inverted_index.an	74, 105
abstract_inverted_index.as	51
abstract_inverted_index.be	49, 101
abstract_inverted_index.by	11, 160
abstract_inverted_index.in	82, 119, 140, 144, 167, 180, 187
abstract_inverted_index.is	8
abstract_inverted_index.of	28, 45, 64, 85, 108, 116, 126, 138
abstract_inverted_index.to	59, 103
abstract_inverted_index.we	40, 156
abstract_inverted_index.Our	171
abstract_inverted_index.and	13, 26
abstract_inverted_index.are	135, 178
abstract_inverted_index.can	48, 56, 100
abstract_inverted_index.for	77, 89
abstract_inverted_index.gap	159
abstract_inverted_index.has	128
abstract_inverted_index.how	146
abstract_inverted_index.the	4, 19, 83, 97, 123, 141, 175
abstract_inverted_index.via	16
abstract_inverted_index.LLMs	30, 127, 181
abstract_inverted_index.and,	143
abstract_inverted_index.been	32
abstract_inverted_index.find	95
abstract_inverted_index.form	84
abstract_inverted_index.have	31
abstract_inverted_index.help	58
abstract_inverted_index.play	183
abstract_inverted_index.role	186
abstract_inverted_index.that	7, 42, 55, 96, 164, 174, 182
abstract_inverted_index.this	38, 46, 79, 153, 158
abstract_inverted_index.used	102
abstract_inverted_index.well	9
abstract_inverted_index.when	189
abstract_inverted_index.wild	142
abstract_inverted_index.with	3, 18, 191
abstract_inverted_index.LLMs.	120
abstract_inverted_index.about	68, 131
abstract_inverted_index.argue	41
abstract_inverted_index.daily	91
abstract_inverted_index.deals	2
abstract_inverted_index.frame	104
abstract_inverted_index.human	92
abstract_inverted_index.proxy	44
abstract_inverted_index.these	133, 150
abstract_inverted_index.truly	136
abstract_inverted_index.using	34
abstract_inverted_index.work,	39
abstract_inverted_index.bridge	157
abstract_inverted_index.create	73
abstract_inverted_index.design	162
abstract_inverted_index.humans	12
abstract_inverted_index.inside	149
abstract_inverted_index.models	134
abstract_inverted_index.number	107
abstract_inverted_index.occurs	148
abstract_inverted_index.paper,	155
abstract_inverted_index.query.	194
abstract_inverted_index.raised	129
abstract_inverted_index.recent	22
abstract_inverted_index.scheme	76
abstract_inverted_index.tasks.	36
abstract_inverted_index.times,	23
abstract_inverted_index.world.	20
abstract_inverted_index.capable	137
abstract_inverted_index.created	98
abstract_inverted_index.further	57
abstract_inverted_index.models.	151
abstract_inverted_index.perform	60
abstract_inverted_index.queries	110
abstract_inverted_index.similar	169
abstract_inverted_index.suggest	173
abstract_inverted_index.various	29, 69
abstract_inverted_index.whether	132
abstract_inverted_index.acquired	15
abstract_inverted_index.enormous	106
abstract_inverted_index.findings	172
abstract_inverted_index.general,	145
abstract_inverted_index.implicit	5, 80
abstract_inverted_index.prompted	190
abstract_inverted_index.research	166
abstract_inverted_index.resource	99, 154
abstract_inverted_index.rigorous	62, 114
abstract_inverted_index.10^{17}),	112
abstract_inverted_index.Moreover,	121
abstract_inverted_index.abilities	67
abstract_inverted_index.capturing	78
abstract_inverted_index.evaluated	33
abstract_inverted_index.graphical	53, 87
abstract_inverted_index.knowledge	6, 81
abstract_inverted_index.localized	179
abstract_inverted_index.prominent	185
abstract_inverted_index.proposing	161
abstract_inverted_index.questions	130
abstract_inverted_index.reasoning	1, 25, 66, 118, 139, 147, 176
abstract_inverted_index.recently,	122
abstract_inverted_index.structure	54, 88
abstract_inverted_index.typically	14
abstract_inverted_index.annotation	75
abstract_inverted_index.components	177
abstract_inverted_index.direction.	170
abstract_inverted_index.evaluation	63, 115
abstract_inverted_index.facilitate	165
abstract_inverted_index.maintained	50
abstract_inverted_index.mechanisms	163
abstract_inverted_index.real-world	70
abstract_inverted_index.remarkable	124
abstract_inverted_index.text-based	35
abstract_inverted_index.understood	10
abstract_inverted_index.Commonsense	0
abstract_inverted_index.activities.	71, 93
abstract_inverted_index.commonsense	24, 65, 109, 117, 193
abstract_inverted_index.performance	125
abstract_inverted_index.facilitating	113
abstract_inverted_index.interactions	17
abstract_inverted_index.understanding	27, 47
abstract_inverted_index.decision-making	188
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	4
citation_normalized_percentile