Negated Complementary Commonsense using Large Language Models Article Swipe
Navid Rezaei
,
Marek Reformat
·
YOU?
·
· 2023
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2307.06794
YOU?
·
· 2023
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2307.06794
Larger language models, such as GPT-3, have shown to be excellent in many tasks. However, we demonstrate that out-of-ordinary questions can throw the model off guard. This work focuses on finding answers to negated complementary questions in commonsense scenarios. We illustrate how such questions adversely affect the model responses. We propose a model-agnostic methodology to improve the performance in negated complementary scenarios. Our method outperforms few-shot generation from GPT-3 (by more than 11 points) and, more importantly, highlights the significance of studying the response of large language models in negated complementary questions. The code, data, and experiments are available under: https://github.com/navidre/negated_complementary_commonsense.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2307.06794
- https://arxiv.org/pdf/2307.06794
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4384388165
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4384388165Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2307.06794Digital Object Identifier
- Title
-
Negated Complementary Commonsense using Large Language ModelsWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2023Year of publication
- Publication date
-
2023-07-13Full publication date if available
- Authors
-
Navid Rezaei, Marek ReformatList of authors in order
- Landing page
-
https://arxiv.org/abs/2307.06794Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2307.06794Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2307.06794Direct OA link when available
- Concepts
-
Computer science, Language model, Commonsense reasoning, Guard (computer science), Commonsense knowledge, Natural language processing, Artificial intelligence, Code (set theory), Programming language, Knowledge-based systems, Set (abstract data type)Top concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4384388165 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2307.06794 |
| ids.doi | https://doi.org/10.48550/arxiv.2307.06794 |
| ids.openalex | https://openalex.org/W4384388165 |
| fwci | |
| type | preprint |
| title | Negated Complementary Commonsense using Large Language Models |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10028 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9990000128746033 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Topic Modeling |
| topics[1].id | https://openalex.org/T10181 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9911999702453613 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Natural Language Processing Techniques |
| topics[2].id | https://openalex.org/T12031 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9775000214576721 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1702 |
| topics[2].subfield.display_name | Artificial Intelligence |
| topics[2].display_name | Speech and dialogue systems |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C41008148 |
| concepts[0].level | 0 |
| concepts[0].score | 0.7034683227539062 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[0].display_name | Computer science |
| concepts[1].id | https://openalex.org/C137293760 |
| concepts[1].level | 2 |
| concepts[1].score | 0.6786508560180664 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q3621696 |
| concepts[1].display_name | Language model |
| concepts[2].id | https://openalex.org/C193221554 |
| concepts[2].level | 2 |
| concepts[2].score | 0.6743184328079224 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q5153664 |
| concepts[2].display_name | Commonsense reasoning |
| concepts[3].id | https://openalex.org/C141141315 |
| concepts[3].level | 2 |
| concepts[3].score | 0.5832701325416565 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q2379942 |
| concepts[3].display_name | Guard (computer science) |
| concepts[4].id | https://openalex.org/C30542707 |
| concepts[4].level | 3 |
| concepts[4].score | 0.4712653160095215 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q1603203 |
| concepts[4].display_name | Commonsense knowledge |
| concepts[5].id | https://openalex.org/C204321447 |
| concepts[5].level | 1 |
| concepts[5].score | 0.4508613348007202 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q30642 |
| concepts[5].display_name | Natural language processing |
| concepts[6].id | https://openalex.org/C154945302 |
| concepts[6].level | 1 |
| concepts[6].score | 0.45058366656303406 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[6].display_name | Artificial intelligence |
| concepts[7].id | https://openalex.org/C2776760102 |
| concepts[7].level | 3 |
| concepts[7].score | 0.4326813220977783 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q5139990 |
| concepts[7].display_name | Code (set theory) |
| concepts[8].id | https://openalex.org/C199360897 |
| concepts[8].level | 1 |
| concepts[8].score | 0.08951061964035034 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q9143 |
| concepts[8].display_name | Programming language |
| concepts[9].id | https://openalex.org/C115925183 |
| concepts[9].level | 2 |
| concepts[9].score | 0.0 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q1412694 |
| concepts[9].display_name | Knowledge-based systems |
| concepts[10].id | https://openalex.org/C177264268 |
| concepts[10].level | 2 |
| concepts[10].score | 0.0 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q1514741 |
| concepts[10].display_name | Set (abstract data type) |
| keywords[0].id | https://openalex.org/keywords/computer-science |
| keywords[0].score | 0.7034683227539062 |
| keywords[0].display_name | Computer science |
| keywords[1].id | https://openalex.org/keywords/language-model |
| keywords[1].score | 0.6786508560180664 |
| keywords[1].display_name | Language model |
| keywords[2].id | https://openalex.org/keywords/commonsense-reasoning |
| keywords[2].score | 0.6743184328079224 |
| keywords[2].display_name | Commonsense reasoning |
| keywords[3].id | https://openalex.org/keywords/guard |
| keywords[3].score | 0.5832701325416565 |
| keywords[3].display_name | Guard (computer science) |
| keywords[4].id | https://openalex.org/keywords/commonsense-knowledge |
| keywords[4].score | 0.4712653160095215 |
| keywords[4].display_name | Commonsense knowledge |
| keywords[5].id | https://openalex.org/keywords/natural-language-processing |
| keywords[5].score | 0.4508613348007202 |
| keywords[5].display_name | Natural language processing |
| keywords[6].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[6].score | 0.45058366656303406 |
| keywords[6].display_name | Artificial intelligence |
| keywords[7].id | https://openalex.org/keywords/code |
| keywords[7].score | 0.4326813220977783 |
| keywords[7].display_name | Code (set theory) |
| keywords[8].id | https://openalex.org/keywords/programming-language |
| keywords[8].score | 0.08951061964035034 |
| keywords[8].display_name | Programming language |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2307.06794 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2307.06794 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2307.06794 |
| locations[1].id | doi:10.48550/arxiv.2307.06794 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | cc-by |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | https://openalex.org/licenses/cc-by |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2307.06794 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5063022372 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-3945-9439 |
| authorships[0].author.display_name | Navid Rezaei |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Rezaei, Navid |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5024453737 |
| authorships[1].author.orcid | https://orcid.org/0000-0003-4783-0717 |
| authorships[1].author.display_name | Marek Reformat |
| authorships[1].author_position | last |
| authorships[1].raw_author_name | Reformat, Marek Z. |
| authorships[1].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2307.06794 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Negated Complementary Commonsense using Large Language Models |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10028 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9990000128746033 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Topic Modeling |
| related_works | https://openalex.org/W3035583586, https://openalex.org/W4320165839, https://openalex.org/W2151799802, https://openalex.org/W4385488510, https://openalex.org/W2196562041, https://openalex.org/W2073302931, https://openalex.org/W4378501473, https://openalex.org/W3206107299, https://openalex.org/W3082691151, https://openalex.org/W4287633646 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2307.06794 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2307.06794 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2307.06794 |
| primary_location.id | pmh:oai:arXiv.org:2307.06794 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2307.06794 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2307.06794 |
| publication_date | 2023-07-13 |
| publication_year | 2023 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 51 |
| abstract_inverted_index.11 | 72 |
| abstract_inverted_index.We | 39, 49 |
| abstract_inverted_index.as | 4 |
| abstract_inverted_index.be | 9 |
| abstract_inverted_index.in | 11, 36, 58, 88 |
| abstract_inverted_index.of | 80, 84 |
| abstract_inverted_index.on | 29 |
| abstract_inverted_index.to | 8, 32, 54 |
| abstract_inverted_index.we | 15 |
| abstract_inverted_index.(by | 69 |
| abstract_inverted_index.Our | 62 |
| abstract_inverted_index.The | 92 |
| abstract_inverted_index.and | 95 |
| abstract_inverted_index.are | 97 |
| abstract_inverted_index.can | 20 |
| abstract_inverted_index.how | 41 |
| abstract_inverted_index.off | 24 |
| abstract_inverted_index.the | 22, 46, 56, 78, 82 |
| abstract_inverted_index.This | 26 |
| abstract_inverted_index.and, | 74 |
| abstract_inverted_index.from | 67 |
| abstract_inverted_index.have | 6 |
| abstract_inverted_index.many | 12 |
| abstract_inverted_index.more | 70, 75 |
| abstract_inverted_index.such | 3, 42 |
| abstract_inverted_index.than | 71 |
| abstract_inverted_index.that | 17 |
| abstract_inverted_index.work | 27 |
| abstract_inverted_index.GPT-3 | 68 |
| abstract_inverted_index.code, | 93 |
| abstract_inverted_index.data, | 94 |
| abstract_inverted_index.large | 85 |
| abstract_inverted_index.model | 23, 47 |
| abstract_inverted_index.shown | 7 |
| abstract_inverted_index.throw | 21 |
| abstract_inverted_index.GPT-3, | 5 |
| abstract_inverted_index.Larger | 0 |
| abstract_inverted_index.affect | 45 |
| abstract_inverted_index.guard. | 25 |
| abstract_inverted_index.method | 63 |
| abstract_inverted_index.models | 87 |
| abstract_inverted_index.tasks. | 13 |
| abstract_inverted_index.under: | 99 |
| abstract_inverted_index.answers | 31 |
| abstract_inverted_index.finding | 30 |
| abstract_inverted_index.focuses | 28 |
| abstract_inverted_index.improve | 55 |
| abstract_inverted_index.models, | 2 |
| abstract_inverted_index.negated | 33, 59, 89 |
| abstract_inverted_index.points) | 73 |
| abstract_inverted_index.propose | 50 |
| abstract_inverted_index.However, | 14 |
| abstract_inverted_index.few-shot | 65 |
| abstract_inverted_index.language | 1, 86 |
| abstract_inverted_index.response | 83 |
| abstract_inverted_index.studying | 81 |
| abstract_inverted_index.adversely | 44 |
| abstract_inverted_index.available | 98 |
| abstract_inverted_index.excellent | 10 |
| abstract_inverted_index.questions | 19, 35, 43 |
| abstract_inverted_index.generation | 66 |
| abstract_inverted_index.highlights | 77 |
| abstract_inverted_index.illustrate | 40 |
| abstract_inverted_index.questions. | 91 |
| abstract_inverted_index.responses. | 48 |
| abstract_inverted_index.scenarios. | 38, 61 |
| abstract_inverted_index.commonsense | 37 |
| abstract_inverted_index.demonstrate | 16 |
| abstract_inverted_index.experiments | 96 |
| abstract_inverted_index.methodology | 53 |
| abstract_inverted_index.outperforms | 64 |
| abstract_inverted_index.performance | 57 |
| abstract_inverted_index.importantly, | 76 |
| abstract_inverted_index.significance | 79 |
| abstract_inverted_index.complementary | 34, 60, 90 |
| abstract_inverted_index.model-agnostic | 52 |
| abstract_inverted_index.out-of-ordinary | 18 |
| abstract_inverted_index.https://github.com/navidre/negated_complementary_commonsense. | 100 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 2 |
| citation_normalized_percentile |