Revealing Multimodal Causality with Large Language Models Article Swipe
Jin Li
,
Shoujin Wang
,
Qi Zhang
,
Feng Liu
,
Tongliang Liu
,
Cao, Longbing
,
Shui Yu
,
Fang Chen
·
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2509.17784
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2509.17784
Uncovering cause-and-effect mechanisms from data is fundamental to scientific progress. While large language models (LLMs) show promise for enhancing causal discovery (CD) from unstructured data, their application to the increasingly prevalent multimodal setting remains a critical challenge. Even with the advent of multimodal LLMs (MLLMs), their efficacy in multimodal CD is hindered by two primary limitations: (1) difficulty in exploring intra- and inter-modal interactions for comprehensive causal variable identification; and (2) insufficiency to handle structural ambiguities with purely observational data. To address these challenges, we propose MLLM-CD, a novel framework for multimodal causal discovery from unstructured data. It consists of three key components: (1) a novel contrastive factor discovery module to identify genuine multimodal factors based on the interactions explored from contrastive sample pairs; (2) a statistical causal structure discovery module to infer causal relationships among discovered factors; and (3) an iterative multimodal counterfactual reasoning module to refine the discovery outcomes iteratively by incorporating the world knowledge and reasoning capabilities of MLLMs. Extensive experiments on both synthetic and real-world datasets demonstrate the effectiveness of the proposed MLLM-CD in revealing genuine factors and causal relationships among them from multimodal unstructured data.
Related Topics
Concepts
No concepts available.
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2509.17784
- https://arxiv.org/pdf/2509.17784
- OA Status
- green
- OpenAlex ID
- https://openalex.org/W4415255245
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4415255245Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2509.17784Digital Object Identifier
- Title
-
Revealing Multimodal Causality with Large Language ModelsWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2025Year of publication
- Publication date
-
2025-09-22Full publication date if available
- Authors
-
Jin Li, Shoujin Wang, Qi Zhang, Feng Liu, Tongliang Liu, Cao, Longbing, Shui Yu, Fang ChenList of authors in order
- Landing page
-
https://arxiv.org/abs/2509.17784Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2509.17784Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2509.17784Direct OA link when available
- Cited by
-
0Total citation count in OpenAlex
Full payload
| id | https://openalex.org/W4415255245 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2509.17784 |
| ids.doi | https://doi.org/10.48550/arxiv.2509.17784 |
| ids.openalex | https://openalex.org/W4415255245 |
| fwci | |
| type | preprint |
| title | Revealing Multimodal Causality with Large Language Models |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10028 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.942300021648407 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Topic Modeling |
| topics[1].id | https://openalex.org/T10181 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9375 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Natural Language Processing Techniques |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2509.17784 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2509.17784 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2509.17784 |
| locations[1].id | doi:10.48550/arxiv.2509.17784 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | cc-by |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | https://openalex.org/licenses/cc-by |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2509.17784 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5101980007 |
| authorships[0].author.orcid | https://orcid.org/0000-0003-3332-7790 |
| authorships[0].author.display_name | Jin Li |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Li, Jin |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5082317196 |
| authorships[1].author.orcid | https://orcid.org/0000-0003-1133-9379 |
| authorships[1].author.display_name | Shoujin Wang |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Wang, Shoujin |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5100360407 |
| authorships[2].author.orcid | https://orcid.org/0000-0003-0947-4942 |
| authorships[2].author.display_name | Qi Zhang |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Zhang, Qi |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5100609156 |
| authorships[3].author.orcid | https://orcid.org/0000-0003-2005-9117 |
| authorships[3].author.display_name | Feng Liu |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Liu, Feng |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5065250332 |
| authorships[4].author.orcid | https://orcid.org/0000-0002-9640-6472 |
| authorships[4].author.display_name | Tongliang Liu |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Liu, Tongliang |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | |
| authorships[5].author.orcid | https://orcid.org/0000-0003-1562-9429 |
| authorships[5].author.display_name | |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Cao, Longbing |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5005228053 |
| authorships[6].author.orcid | https://orcid.org/0000-0003-4485-6743 |
| authorships[6].author.display_name | Shui Yu |
| authorships[6].author_position | middle |
| authorships[6].raw_author_name | Yu, Shui |
| authorships[6].is_corresponding | False |
| authorships[7].author.id | https://openalex.org/A5100400036 |
| authorships[7].author.orcid | https://orcid.org/0000-0002-8008-1175 |
| authorships[7].author.display_name | Fang Chen |
| authorships[7].author_position | last |
| authorships[7].raw_author_name | Chen, Fang |
| authorships[7].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2509.17784 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-16T00:00:00 |
| display_name | Revealing Multimodal Causality with Large Language Models |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10028 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.942300021648407 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Topic Modeling |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2509.17784 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2509.17784 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2509.17784 |
| primary_location.id | pmh:oai:arXiv.org:2509.17784 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2509.17784 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2509.17784 |
| publication_date | 2025-09-22 |
| publication_year | 2025 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 34, 87, 104, 125 |
| abstract_inverted_index.CD | 49 |
| abstract_inverted_index.It | 97 |
| abstract_inverted_index.To | 80 |
| abstract_inverted_index.an | 140 |
| abstract_inverted_index.by | 52, 152 |
| abstract_inverted_index.in | 47, 58, 177 |
| abstract_inverted_index.is | 5, 50 |
| abstract_inverted_index.of | 41, 99, 160, 173 |
| abstract_inverted_index.on | 116, 164 |
| abstract_inverted_index.to | 7, 27, 72, 110, 131, 146 |
| abstract_inverted_index.we | 84 |
| abstract_inverted_index.(1) | 56, 103 |
| abstract_inverted_index.(2) | 70, 124 |
| abstract_inverted_index.(3) | 139 |
| abstract_inverted_index.and | 61, 69, 138, 157, 167, 181 |
| abstract_inverted_index.for | 17, 64, 90 |
| abstract_inverted_index.key | 101 |
| abstract_inverted_index.the | 28, 39, 117, 148, 154, 171, 174 |
| abstract_inverted_index.two | 53 |
| abstract_inverted_index.(CD) | 21 |
| abstract_inverted_index.Even | 37 |
| abstract_inverted_index.LLMs | 43 |
| abstract_inverted_index.both | 165 |
| abstract_inverted_index.data | 4 |
| abstract_inverted_index.from | 3, 22, 94, 120, 186 |
| abstract_inverted_index.show | 15 |
| abstract_inverted_index.them | 185 |
| abstract_inverted_index.with | 38, 76 |
| abstract_inverted_index.While | 10 |
| abstract_inverted_index.among | 135, 184 |
| abstract_inverted_index.based | 115 |
| abstract_inverted_index.data, | 24 |
| abstract_inverted_index.data. | 79, 96, 189 |
| abstract_inverted_index.infer | 132 |
| abstract_inverted_index.large | 11 |
| abstract_inverted_index.novel | 88, 105 |
| abstract_inverted_index.their | 25, 45 |
| abstract_inverted_index.these | 82 |
| abstract_inverted_index.three | 100 |
| abstract_inverted_index.world | 155 |
| abstract_inverted_index.(LLMs) | 14 |
| abstract_inverted_index.MLLMs. | 161 |
| abstract_inverted_index.advent | 40 |
| abstract_inverted_index.causal | 19, 66, 92, 127, 133, 182 |
| abstract_inverted_index.factor | 107 |
| abstract_inverted_index.handle | 73 |
| abstract_inverted_index.intra- | 60 |
| abstract_inverted_index.models | 13 |
| abstract_inverted_index.module | 109, 130, 145 |
| abstract_inverted_index.pairs; | 123 |
| abstract_inverted_index.purely | 77 |
| abstract_inverted_index.refine | 147 |
| abstract_inverted_index.sample | 122 |
| abstract_inverted_index.MLLM-CD | 176 |
| abstract_inverted_index.address | 81 |
| abstract_inverted_index.factors | 114, 180 |
| abstract_inverted_index.genuine | 112, 179 |
| abstract_inverted_index.primary | 54 |
| abstract_inverted_index.promise | 16 |
| abstract_inverted_index.propose | 85 |
| abstract_inverted_index.remains | 33 |
| abstract_inverted_index.setting | 32 |
| abstract_inverted_index.(MLLMs), | 44 |
| abstract_inverted_index.MLLM-CD, | 86 |
| abstract_inverted_index.consists | 98 |
| abstract_inverted_index.critical | 35 |
| abstract_inverted_index.datasets | 169 |
| abstract_inverted_index.efficacy | 46 |
| abstract_inverted_index.explored | 119 |
| abstract_inverted_index.factors; | 137 |
| abstract_inverted_index.hindered | 51 |
| abstract_inverted_index.identify | 111 |
| abstract_inverted_index.language | 12 |
| abstract_inverted_index.outcomes | 150 |
| abstract_inverted_index.proposed | 175 |
| abstract_inverted_index.variable | 67 |
| abstract_inverted_index.Extensive | 162 |
| abstract_inverted_index.discovery | 20, 93, 108, 129, 149 |
| abstract_inverted_index.enhancing | 18 |
| abstract_inverted_index.exploring | 59 |
| abstract_inverted_index.framework | 89 |
| abstract_inverted_index.iterative | 141 |
| abstract_inverted_index.knowledge | 156 |
| abstract_inverted_index.prevalent | 30 |
| abstract_inverted_index.progress. | 9 |
| abstract_inverted_index.reasoning | 144, 158 |
| abstract_inverted_index.revealing | 178 |
| abstract_inverted_index.structure | 128 |
| abstract_inverted_index.synthetic | 166 |
| abstract_inverted_index.Uncovering | 0 |
| abstract_inverted_index.challenge. | 36 |
| abstract_inverted_index.difficulty | 57 |
| abstract_inverted_index.discovered | 136 |
| abstract_inverted_index.mechanisms | 2 |
| abstract_inverted_index.multimodal | 31, 42, 48, 91, 113, 142, 187 |
| abstract_inverted_index.real-world | 168 |
| abstract_inverted_index.scientific | 8 |
| abstract_inverted_index.structural | 74 |
| abstract_inverted_index.ambiguities | 75 |
| abstract_inverted_index.application | 26 |
| abstract_inverted_index.challenges, | 83 |
| abstract_inverted_index.components: | 102 |
| abstract_inverted_index.contrastive | 106, 121 |
| abstract_inverted_index.demonstrate | 170 |
| abstract_inverted_index.experiments | 163 |
| abstract_inverted_index.fundamental | 6 |
| abstract_inverted_index.inter-modal | 62 |
| abstract_inverted_index.iteratively | 151 |
| abstract_inverted_index.statistical | 126 |
| abstract_inverted_index.capabilities | 159 |
| abstract_inverted_index.increasingly | 29 |
| abstract_inverted_index.interactions | 63, 118 |
| abstract_inverted_index.limitations: | 55 |
| abstract_inverted_index.unstructured | 23, 95, 188 |
| abstract_inverted_index.comprehensive | 65 |
| abstract_inverted_index.effectiveness | 172 |
| abstract_inverted_index.incorporating | 153 |
| abstract_inverted_index.insufficiency | 71 |
| abstract_inverted_index.observational | 78 |
| abstract_inverted_index.relationships | 134, 183 |
| abstract_inverted_index.counterfactual | 143 |
| abstract_inverted_index.identification; | 68 |
| abstract_inverted_index.cause-and-effect | 1 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 8 |
| citation_normalized_percentile |