BengaliFig: A Low-Resource Challenge for Figurative and Culturally Grounded Reasoning in Bengali Article Swipe
Large language models excel on broad multilingual benchmarks but remain to be evaluated extensively in figurative and culturally grounded reasoning, especially in low-resource contexts. We present BengaliFig, a compact yet richly annotated challenge set that targets this gap in Bengali, a widely spoken low-resourced language. The dataset contains 435 unique riddles drawn from Bengali oral and literary traditions. Each item is annotated along five orthogonal dimensions capturing reasoning type, trap type, cultural depth, answer category, and difficulty, and is automatically converted to multiple-choice format through a constraint-aware, AI-assisted pipeline. We evaluate eight frontier LLMs from major providers under zero-shot and few-shot chain-of-thought prompting, revealing consistent weaknesses in metaphorical and culturally specific reasoning. BengaliFig thus contributes both a diagnostic probe for evaluating LLM robustness in low-resource cultural contexts and a step toward inclusive and heritage-aware NLP evaluation.
Related Topics
Concepts
Bengali
Literal and figurative language
Linguistics
Cultural diversity
Natural language processing
Set (abstract data type)
Metaphor
Computational linguistics
Artificial intelligence
Computer science
Rubric
Psychology
Narrative
Pragmatics
Paraphrase
Robustness (evolution)
Corpus linguistics
Frontier
Cultural competence
Strengths and weaknesses
Grounded theory
Sociology
Interview
Culturally appropriate
Cultural background
Cultural knowledge
Metadata
- Type
- article
- Landing Page
- http://arxiv.org/abs/2511.20399
- https://arxiv.org/pdf/2511.20399
- OA Status
- green
- OpenAlex ID
- https://openalex.org/W7106861784
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W7106861784Canonical identifier for this work in OpenAlex
- Title
-
BengaliFig: A Low-Resource Challenge for Figurative and Culturally Grounded Reasoning in BengaliWork title
- Type
-
articleOpenAlex work type
- Publication year
-
2025Year of publication
- Publication date
-
2025-11-25Full publication date if available
- Authors
-
Sefat, Abdullah AlList of authors in order
- Landing page
-
https://arxiv.org/abs/2511.20399Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2511.20399Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2511.20399Direct OA link when available
- Concepts
-
Bengali, Literal and figurative language, Linguistics, Cultural diversity, Natural language processing, Set (abstract data type), Metaphor, Computational linguistics, Artificial intelligence, Computer science, Rubric, Psychology, Narrative, Pragmatics, Paraphrase, Robustness (evolution), Corpus linguistics, Frontier, Cultural competence, Strengths and weaknesses, Grounded theory, Sociology, Interview, Culturally appropriate, Cultural background, Cultural knowledgeTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
Full payload
| id | https://openalex.org/W7106861784 |
|---|---|
| doi | |
| ids.openalex | https://openalex.org/W7106861784 |
| fwci | 0.0 |
| type | article |
| title | BengaliFig: A Low-Resource Challenge for Figurative and Culturally Grounded Reasoning in Bengali |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10028 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.4439389407634735 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Topic Modeling |
| topics[1].id | https://openalex.org/T13910 |
| topics[1].field.id | https://openalex.org/fields/33 |
| topics[1].field.display_name | Social Sciences |
| topics[1].score | 0.07257042080163956 |
| topics[1].domain.id | https://openalex.org/domains/2 |
| topics[1].domain.display_name | Social Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/3300 |
| topics[1].subfield.display_name | General Social Sciences |
| topics[1].display_name | Computational and Text Analysis Methods |
| topics[2].id | https://openalex.org/T10181 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.04300680384039879 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1702 |
| topics[2].subfield.display_name | Artificial Intelligence |
| topics[2].display_name | Natural Language Processing Techniques |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C19235068 |
| concepts[0].level | 2 |
| concepts[0].score | 0.9031453728675842 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q9610 |
| concepts[0].display_name | Bengali |
| concepts[1].id | https://openalex.org/C46182478 |
| concepts[1].level | 2 |
| concepts[1].score | 0.7818306684494019 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q7363315 |
| concepts[1].display_name | Literal and figurative language |
| concepts[2].id | https://openalex.org/C41895202 |
| concepts[2].level | 1 |
| concepts[2].score | 0.5169742107391357 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q8162 |
| concepts[2].display_name | Linguistics |
| concepts[3].id | https://openalex.org/C125209646 |
| concepts[3].level | 2 |
| concepts[3].score | 0.510771632194519 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q1338878 |
| concepts[3].display_name | Cultural diversity |
| concepts[4].id | https://openalex.org/C204321447 |
| concepts[4].level | 1 |
| concepts[4].score | 0.48473966121673584 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q30642 |
| concepts[4].display_name | Natural language processing |
| concepts[5].id | https://openalex.org/C177264268 |
| concepts[5].level | 2 |
| concepts[5].score | 0.4597781300544739 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q1514741 |
| concepts[5].display_name | Set (abstract data type) |
| concepts[6].id | https://openalex.org/C2778311575 |
| concepts[6].level | 2 |
| concepts[6].score | 0.45414620637893677 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q18534 |
| concepts[6].display_name | Metaphor |
| concepts[7].id | https://openalex.org/C155092808 |
| concepts[7].level | 2 |
| concepts[7].score | 0.45281967520713806 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q182557 |
| concepts[7].display_name | Computational linguistics |
| concepts[8].id | https://openalex.org/C154945302 |
| concepts[8].level | 1 |
| concepts[8].score | 0.4427104592323303 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[8].display_name | Artificial intelligence |
| concepts[9].id | https://openalex.org/C41008148 |
| concepts[9].level | 0 |
| concepts[9].score | 0.43360766768455505 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[9].display_name | Computer science |
| concepts[10].id | https://openalex.org/C111640148 |
| concepts[10].level | 2 |
| concepts[10].score | 0.4238336980342865 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q847349 |
| concepts[10].display_name | Rubric |
| concepts[11].id | https://openalex.org/C15744967 |
| concepts[11].level | 0 |
| concepts[11].score | 0.38954177498817444 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q9418 |
| concepts[11].display_name | Psychology |
| concepts[12].id | https://openalex.org/C199033989 |
| concepts[12].level | 2 |
| concepts[12].score | 0.38151535391807556 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q1318295 |
| concepts[12].display_name | Narrative |
| concepts[13].id | https://openalex.org/C11693617 |
| concepts[13].level | 2 |
| concepts[13].score | 0.3595602214336395 |
| concepts[13].wikidata | https://www.wikidata.org/wiki/Q181839 |
| concepts[13].display_name | Pragmatics |
| concepts[14].id | https://openalex.org/C2780922921 |
| concepts[14].level | 2 |
| concepts[14].score | 0.3522202670574188 |
| concepts[14].wikidata | https://www.wikidata.org/wiki/Q255189 |
| concepts[14].display_name | Paraphrase |
| concepts[15].id | https://openalex.org/C63479239 |
| concepts[15].level | 3 |
| concepts[15].score | 0.3423788845539093 |
| concepts[15].wikidata | https://www.wikidata.org/wiki/Q7353546 |
| concepts[15].display_name | Robustness (evolution) |
| concepts[16].id | https://openalex.org/C532629269 |
| concepts[16].level | 2 |
| concepts[16].score | 0.3249259889125824 |
| concepts[16].wikidata | https://www.wikidata.org/wiki/Q865083 |
| concepts[16].display_name | Corpus linguistics |
| concepts[17].id | https://openalex.org/C2778571376 |
| concepts[17].level | 2 |
| concepts[17].score | 0.31019601225852966 |
| concepts[17].wikidata | https://www.wikidata.org/wiki/Q1355821 |
| concepts[17].display_name | Frontier |
| concepts[18].id | https://openalex.org/C169536714 |
| concepts[18].level | 2 |
| concepts[18].score | 0.30037838220596313 |
| concepts[18].wikidata | https://www.wikidata.org/wiki/Q1666159 |
| concepts[18].display_name | Cultural competence |
| concepts[19].id | https://openalex.org/C63882131 |
| concepts[19].level | 2 |
| concepts[19].score | 0.3003504276275635 |
| concepts[19].wikidata | https://www.wikidata.org/wiki/Q17122954 |
| concepts[19].display_name | Strengths and weaknesses |
| concepts[20].id | https://openalex.org/C156325361 |
| concepts[20].level | 3 |
| concepts[20].score | 0.297637939453125 |
| concepts[20].wikidata | https://www.wikidata.org/wiki/Q1152864 |
| concepts[20].display_name | Grounded theory |
| concepts[21].id | https://openalex.org/C144024400 |
| concepts[21].level | 0 |
| concepts[21].score | 0.2976316511631012 |
| concepts[21].wikidata | https://www.wikidata.org/wiki/Q21201 |
| concepts[21].display_name | Sociology |
| concepts[22].id | https://openalex.org/C24845683 |
| concepts[22].level | 2 |
| concepts[22].score | 0.288119375705719 |
| concepts[22].wikidata | https://www.wikidata.org/wiki/Q178651 |
| concepts[22].display_name | Interview |
| concepts[23].id | https://openalex.org/C3020486378 |
| concepts[23].level | 2 |
| concepts[23].score | 0.2771572768688202 |
| concepts[23].wikidata | https://www.wikidata.org/wiki/Q5188450 |
| concepts[23].display_name | Culturally appropriate |
| concepts[24].id | https://openalex.org/C2992975276 |
| concepts[24].level | 4 |
| concepts[24].score | 0.2715776264667511 |
| concepts[24].wikidata | https://www.wikidata.org/wiki/Q11042 |
| concepts[24].display_name | Cultural background |
| concepts[25].id | https://openalex.org/C2994055011 |
| concepts[25].level | 2 |
| concepts[25].score | 0.27116307616233826 |
| concepts[25].wikidata | https://www.wikidata.org/wiki/Q210272 |
| concepts[25].display_name | Cultural knowledge |
| keywords[0].id | https://openalex.org/keywords/bengali |
| keywords[0].score | 0.9031453728675842 |
| keywords[0].display_name | Bengali |
| keywords[1].id | https://openalex.org/keywords/literal-and-figurative-language |
| keywords[1].score | 0.7818306684494019 |
| keywords[1].display_name | Literal and figurative language |
| keywords[2].id | https://openalex.org/keywords/cultural-diversity |
| keywords[2].score | 0.510771632194519 |
| keywords[2].display_name | Cultural diversity |
| keywords[3].id | https://openalex.org/keywords/set |
| keywords[3].score | 0.4597781300544739 |
| keywords[3].display_name | Set (abstract data type) |
| keywords[4].id | https://openalex.org/keywords/metaphor |
| keywords[4].score | 0.45414620637893677 |
| keywords[4].display_name | Metaphor |
| keywords[5].id | https://openalex.org/keywords/computational-linguistics |
| keywords[5].score | 0.45281967520713806 |
| keywords[5].display_name | Computational linguistics |
| keywords[6].id | https://openalex.org/keywords/rubric |
| keywords[6].score | 0.4238336980342865 |
| keywords[6].display_name | Rubric |
| keywords[7].id | https://openalex.org/keywords/narrative |
| keywords[7].score | 0.38151535391807556 |
| keywords[7].display_name | Narrative |
| language | |
| locations[0].id | pmh:oai:arXiv.org:2511.20399 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2511.20399 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2511.20399 |
| indexed_in | arxiv |
| authorships[0].author.id | |
| authorships[0].author.orcid | |
| authorships[0].author.display_name | Sefat, Abdullah Al |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Sefat, Abdullah Al |
| authorships[0].is_corresponding | True |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2511.20399 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-11-28T00:00:00 |
| display_name | BengaliFig: A Low-Resource Challenge for Figurative and Culturally Grounded Reasoning in Bengali |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-28T02:12:24.556248 |
| primary_topic.id | https://openalex.org/T10028 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.4439389407634735 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Topic Modeling |
| cited_by_count | 0 |
| locations_count | 1 |
| best_oa_location.id | pmh:oai:arXiv.org:2511.20399 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2511.20399 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2511.20399 |
| primary_location.id | pmh:oai:arXiv.org:2511.20399 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2511.20399 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2511.20399 |
| publication_date | 2025-11-25 |
| publication_year | 2025 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 27, 40, 85, 116, 128 |
| abstract_inverted_index.We | 24, 89 |
| abstract_inverted_index.be | 11 |
| abstract_inverted_index.in | 14, 21, 38, 106, 123 |
| abstract_inverted_index.is | 60, 78 |
| abstract_inverted_index.on | 4 |
| abstract_inverted_index.to | 10, 81 |
| abstract_inverted_index.435 | 48 |
| abstract_inverted_index.LLM | 121 |
| abstract_inverted_index.NLP | 134 |
| abstract_inverted_index.The | 45 |
| abstract_inverted_index.and | 16, 55, 75, 77, 99, 108, 127, 132 |
| abstract_inverted_index.but | 8 |
| abstract_inverted_index.for | 119 |
| abstract_inverted_index.gap | 37 |
| abstract_inverted_index.set | 33 |
| abstract_inverted_index.yet | 29 |
| abstract_inverted_index.Each | 58 |
| abstract_inverted_index.LLMs | 93 |
| abstract_inverted_index.both | 115 |
| abstract_inverted_index.five | 63 |
| abstract_inverted_index.from | 52, 94 |
| abstract_inverted_index.item | 59 |
| abstract_inverted_index.oral | 54 |
| abstract_inverted_index.step | 129 |
| abstract_inverted_index.that | 34 |
| abstract_inverted_index.this | 36 |
| abstract_inverted_index.thus | 113 |
| abstract_inverted_index.trap | 69 |
| abstract_inverted_index.Large | 0 |
| abstract_inverted_index.along | 62 |
| abstract_inverted_index.broad | 5 |
| abstract_inverted_index.drawn | 51 |
| abstract_inverted_index.eight | 91 |
| abstract_inverted_index.excel | 3 |
| abstract_inverted_index.major | 95 |
| abstract_inverted_index.probe | 118 |
| abstract_inverted_index.type, | 68, 70 |
| abstract_inverted_index.under | 97 |
| abstract_inverted_index.answer | 73 |
| abstract_inverted_index.depth, | 72 |
| abstract_inverted_index.format | 83 |
| abstract_inverted_index.models | 2 |
| abstract_inverted_index.remain | 9 |
| abstract_inverted_index.richly | 30 |
| abstract_inverted_index.spoken | 42 |
| abstract_inverted_index.toward | 130 |
| abstract_inverted_index.unique | 49 |
| abstract_inverted_index.widely | 41 |
| abstract_inverted_index.Bengali | 53 |
| abstract_inverted_index.compact | 28 |
| abstract_inverted_index.dataset | 46 |
| abstract_inverted_index.present | 25 |
| abstract_inverted_index.riddles | 50 |
| abstract_inverted_index.targets | 35 |
| abstract_inverted_index.through | 84 |
| abstract_inverted_index.Bengali, | 39 |
| abstract_inverted_index.contains | 47 |
| abstract_inverted_index.contexts | 126 |
| abstract_inverted_index.cultural | 71, 125 |
| abstract_inverted_index.evaluate | 90 |
| abstract_inverted_index.few-shot | 100 |
| abstract_inverted_index.frontier | 92 |
| abstract_inverted_index.grounded | 18 |
| abstract_inverted_index.language | 1 |
| abstract_inverted_index.literary | 56 |
| abstract_inverted_index.specific | 110 |
| abstract_inverted_index.annotated | 31, 61 |
| abstract_inverted_index.capturing | 66 |
| abstract_inverted_index.category, | 74 |
| abstract_inverted_index.challenge | 32 |
| abstract_inverted_index.contexts. | 23 |
| abstract_inverted_index.converted | 80 |
| abstract_inverted_index.evaluated | 12 |
| abstract_inverted_index.inclusive | 131 |
| abstract_inverted_index.language. | 44 |
| abstract_inverted_index.pipeline. | 88 |
| abstract_inverted_index.providers | 96 |
| abstract_inverted_index.reasoning | 67 |
| abstract_inverted_index.revealing | 103 |
| abstract_inverted_index.zero-shot | 98 |
| abstract_inverted_index.BengaliFig | 112 |
| abstract_inverted_index.benchmarks | 7 |
| abstract_inverted_index.consistent | 104 |
| abstract_inverted_index.culturally | 17, 109 |
| abstract_inverted_index.diagnostic | 117 |
| abstract_inverted_index.dimensions | 65 |
| abstract_inverted_index.especially | 20 |
| abstract_inverted_index.evaluating | 120 |
| abstract_inverted_index.figurative | 15 |
| abstract_inverted_index.orthogonal | 64 |
| abstract_inverted_index.prompting, | 102 |
| abstract_inverted_index.reasoning, | 19 |
| abstract_inverted_index.reasoning. | 111 |
| abstract_inverted_index.robustness | 122 |
| abstract_inverted_index.weaknesses | 105 |
| abstract_inverted_index.AI-assisted | 87 |
| abstract_inverted_index.BengaliFig, | 26 |
| abstract_inverted_index.contributes | 114 |
| abstract_inverted_index.difficulty, | 76 |
| abstract_inverted_index.evaluation. | 135 |
| abstract_inverted_index.extensively | 13 |
| abstract_inverted_index.traditions. | 57 |
| abstract_inverted_index.low-resource | 22, 124 |
| abstract_inverted_index.metaphorical | 107 |
| abstract_inverted_index.multilingual | 6 |
| abstract_inverted_index.automatically | 79 |
| abstract_inverted_index.low-resourced | 43 |
| abstract_inverted_index.heritage-aware | 133 |
| abstract_inverted_index.multiple-choice | 82 |
| abstract_inverted_index.chain-of-thought | 101 |
| abstract_inverted_index.constraint-aware, | 86 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 1 |
| citation_normalized_percentile.value | 0.92015879 |
| citation_normalized_percentile.is_in_top_1_percent | False |
| citation_normalized_percentile.is_in_top_10_percent | True |