UNIDOC-BENCH: A Unified Benchmark for Document-Centric Multimodal RAG Article Swipe
Can Qin
,
Zeyuan Chen
,
Ran Xu
·
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2510.03663
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2510.03663
Multimodal retrieval-augmented generation (MM-RAG) is a key approach for applying large language models (LLMs) and agents to real-world knowledge bases, yet current evaluations are fragmented, focusing on either text or images in isolation or on simplified multimodal setups that fail to capture document-centric multimodal use cases. In this paper, we introduce UniDoc-Bench, the first large-scale, realistic benchmark for MM-RAG built from 70k real-world PDF pages across eight domains. Our pipeline extracts and links evidence from text, tables, and figures, then generates 1,600 multimodal QA pairs spanning factual retrieval, comparison, summarization, and logical reasoning queries. To ensure reliability, 20% of QA pairs are validated by multiple annotators and expert adjudication. UniDoc-Bench supports apples-to-apples comparison across four paradigms: (1) text-only, (2) image-only, (3) multimodal text-image fusion, and (4) multimodal joint retrieval -- under a unified protocol with standardized candidate pools, prompts, and evaluation metrics. Our experiments show that multimodal text-image fusion RAG systems consistently outperform both unimodal and jointly multimodal embedding-based retrieval, indicating that neither text nor images alone are sufficient and that current multimodal embeddings remain inadequate. Beyond benchmarking, our analysis reveals when and how visual context complements textual evidence, uncovers systematic failure modes, and offers actionable guidance for developing more robust MM-RAG pipelines.
Related Topics
Concepts
No concepts available.
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2510.03663
- https://arxiv.org/pdf/2510.03663
- OA Status
- green
- OpenAlex ID
- https://openalex.org/W4416373075
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4416373075Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2510.03663Digital Object Identifier
- Title
-
UNIDOC-BENCH: A Unified Benchmark for Document-Centric Multimodal RAGWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2025Year of publication
- Publication date
-
2025-10-04Full publication date if available
- Authors
-
Can Qin, Zeyuan Chen, Ran XuList of authors in order
- Landing page
-
https://arxiv.org/abs/2510.03663Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2510.03663Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2510.03663Direct OA link when available
- Cited by
-
0Total citation count in OpenAlex
Full payload
| id | https://openalex.org/W4416373075 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2510.03663 |
| ids.doi | https://doi.org/10.48550/arxiv.2510.03663 |
| ids.openalex | https://openalex.org/W4416373075 |
| fwci | |
| type | preprint |
| title | UNIDOC-BENCH: A Unified Benchmark for Document-Centric Multimodal RAG |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2510.03663 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2510.03663 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2510.03663 |
| locations[1].id | doi:10.48550/arxiv.2510.03663 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2510.03663 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5021042598 |
| authorships[0].author.orcid | https://orcid.org/0000-0003-0712-5378 |
| authorships[0].author.display_name | Can Qin |
| authorships[0].author_position | middle |
| authorships[0].raw_author_name | Qin, Can |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5101631517 |
| authorships[1].author.orcid | https://orcid.org/0000-0002-0442-8501 |
| authorships[1].author.display_name | Zeyuan Chen |
| authorships[1].author_position | last |
| authorships[1].raw_author_name | Chen, Zeyuan |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5114357280 |
| authorships[2].author.orcid | https://orcid.org/0009-0003-8991-6153 |
| authorships[2].author.display_name | Ran Xu |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Xu, Ran |
| authorships[2].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2510.03663 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | UNIDOC-BENCH: A Unified Benchmark for Document-Centric Multimodal RAG |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-28T12:27:11.455440 |
| primary_topic | |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2510.03663 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2510.03663 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2510.03663 |
| primary_location.id | pmh:oai:arXiv.org:2510.03663 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2510.03663 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2510.03663 |
| publication_date | 2025-10-04 |
| publication_year | 2025 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 5, 131 |
| abstract_inverted_index.-- | 129 |
| abstract_inverted_index.In | 46 |
| abstract_inverted_index.QA | 83, 99 |
| abstract_inverted_index.To | 94 |
| abstract_inverted_index.by | 103 |
| abstract_inverted_index.in | 31 |
| abstract_inverted_index.is | 4 |
| abstract_inverted_index.of | 98 |
| abstract_inverted_index.on | 26, 34 |
| abstract_inverted_index.or | 29, 33 |
| abstract_inverted_index.to | 16, 40 |
| abstract_inverted_index.we | 49 |
| abstract_inverted_index.(1) | 116 |
| abstract_inverted_index.(2) | 118 |
| abstract_inverted_index.(3) | 120 |
| abstract_inverted_index.(4) | 125 |
| abstract_inverted_index.20% | 97 |
| abstract_inverted_index.70k | 61 |
| abstract_inverted_index.Our | 68, 142 |
| abstract_inverted_index.PDF | 63 |
| abstract_inverted_index.RAG | 149 |
| abstract_inverted_index.and | 14, 71, 77, 90, 106, 124, 139, 155, 169, 182, 193 |
| abstract_inverted_index.are | 23, 101, 167 |
| abstract_inverted_index.for | 8, 57, 197 |
| abstract_inverted_index.how | 183 |
| abstract_inverted_index.key | 6 |
| abstract_inverted_index.nor | 164 |
| abstract_inverted_index.our | 178 |
| abstract_inverted_index.the | 52 |
| abstract_inverted_index.use | 44 |
| abstract_inverted_index.yet | 20 |
| abstract_inverted_index.both | 153 |
| abstract_inverted_index.fail | 39 |
| abstract_inverted_index.four | 114 |
| abstract_inverted_index.from | 60, 74 |
| abstract_inverted_index.more | 199 |
| abstract_inverted_index.show | 144 |
| abstract_inverted_index.text | 28, 163 |
| abstract_inverted_index.that | 38, 145, 161, 170 |
| abstract_inverted_index.then | 79 |
| abstract_inverted_index.this | 47 |
| abstract_inverted_index.when | 181 |
| abstract_inverted_index.with | 134 |
| abstract_inverted_index.1,600 | 81 |
| abstract_inverted_index.alone | 166 |
| abstract_inverted_index.built | 59 |
| abstract_inverted_index.eight | 66 |
| abstract_inverted_index.first | 53 |
| abstract_inverted_index.joint | 127 |
| abstract_inverted_index.large | 10 |
| abstract_inverted_index.links | 72 |
| abstract_inverted_index.pages | 64 |
| abstract_inverted_index.pairs | 84, 100 |
| abstract_inverted_index.text, | 75 |
| abstract_inverted_index.under | 130 |
| abstract_inverted_index.(LLMs) | 13 |
| abstract_inverted_index.Beyond | 176 |
| abstract_inverted_index.MM-RAG | 58, 201 |
| abstract_inverted_index.across | 65, 113 |
| abstract_inverted_index.agents | 15 |
| abstract_inverted_index.bases, | 19 |
| abstract_inverted_index.cases. | 45 |
| abstract_inverted_index.either | 27 |
| abstract_inverted_index.ensure | 95 |
| abstract_inverted_index.expert | 107 |
| abstract_inverted_index.fusion | 148 |
| abstract_inverted_index.images | 30, 165 |
| abstract_inverted_index.models | 12 |
| abstract_inverted_index.modes, | 192 |
| abstract_inverted_index.offers | 194 |
| abstract_inverted_index.paper, | 48 |
| abstract_inverted_index.pools, | 137 |
| abstract_inverted_index.remain | 174 |
| abstract_inverted_index.robust | 200 |
| abstract_inverted_index.setups | 37 |
| abstract_inverted_index.visual | 184 |
| abstract_inverted_index.capture | 41 |
| abstract_inverted_index.context | 185 |
| abstract_inverted_index.current | 21, 171 |
| abstract_inverted_index.factual | 86 |
| abstract_inverted_index.failure | 191 |
| abstract_inverted_index.fusion, | 123 |
| abstract_inverted_index.jointly | 156 |
| abstract_inverted_index.logical | 91 |
| abstract_inverted_index.neither | 162 |
| abstract_inverted_index.reveals | 180 |
| abstract_inverted_index.systems | 150 |
| abstract_inverted_index.tables, | 76 |
| abstract_inverted_index.textual | 187 |
| abstract_inverted_index.unified | 132 |
| abstract_inverted_index.(MM-RAG) | 3 |
| abstract_inverted_index.analysis | 179 |
| abstract_inverted_index.applying | 9 |
| abstract_inverted_index.approach | 7 |
| abstract_inverted_index.domains. | 67 |
| abstract_inverted_index.evidence | 73 |
| abstract_inverted_index.extracts | 70 |
| abstract_inverted_index.figures, | 78 |
| abstract_inverted_index.focusing | 25 |
| abstract_inverted_index.guidance | 196 |
| abstract_inverted_index.language | 11 |
| abstract_inverted_index.metrics. | 141 |
| abstract_inverted_index.multiple | 104 |
| abstract_inverted_index.pipeline | 69 |
| abstract_inverted_index.prompts, | 138 |
| abstract_inverted_index.protocol | 133 |
| abstract_inverted_index.queries. | 93 |
| abstract_inverted_index.spanning | 85 |
| abstract_inverted_index.supports | 110 |
| abstract_inverted_index.uncovers | 189 |
| abstract_inverted_index.unimodal | 154 |
| abstract_inverted_index.benchmark | 56 |
| abstract_inverted_index.candidate | 136 |
| abstract_inverted_index.evidence, | 188 |
| abstract_inverted_index.generates | 80 |
| abstract_inverted_index.introduce | 50 |
| abstract_inverted_index.isolation | 32 |
| abstract_inverted_index.knowledge | 18 |
| abstract_inverted_index.realistic | 55 |
| abstract_inverted_index.reasoning | 92 |
| abstract_inverted_index.retrieval | 128 |
| abstract_inverted_index.validated | 102 |
| abstract_inverted_index.Multimodal | 0 |
| abstract_inverted_index.actionable | 195 |
| abstract_inverted_index.annotators | 105 |
| abstract_inverted_index.comparison | 112 |
| abstract_inverted_index.developing | 198 |
| abstract_inverted_index.embeddings | 173 |
| abstract_inverted_index.evaluation | 140 |
| abstract_inverted_index.generation | 2 |
| abstract_inverted_index.indicating | 160 |
| abstract_inverted_index.multimodal | 36, 43, 82, 121, 126, 146, 157, 172 |
| abstract_inverted_index.outperform | 152 |
| abstract_inverted_index.paradigms: | 115 |
| abstract_inverted_index.pipelines. | 202 |
| abstract_inverted_index.real-world | 17, 62 |
| abstract_inverted_index.retrieval, | 87, 159 |
| abstract_inverted_index.simplified | 35 |
| abstract_inverted_index.sufficient | 168 |
| abstract_inverted_index.systematic | 190 |
| abstract_inverted_index.text-image | 122, 147 |
| abstract_inverted_index.text-only, | 117 |
| abstract_inverted_index.comparison, | 88 |
| abstract_inverted_index.complements | 186 |
| abstract_inverted_index.evaluations | 22 |
| abstract_inverted_index.experiments | 143 |
| abstract_inverted_index.fragmented, | 24 |
| abstract_inverted_index.image-only, | 119 |
| abstract_inverted_index.inadequate. | 175 |
| abstract_inverted_index.UniDoc-Bench | 109 |
| abstract_inverted_index.consistently | 151 |
| abstract_inverted_index.large-scale, | 54 |
| abstract_inverted_index.reliability, | 96 |
| abstract_inverted_index.standardized | 135 |
| abstract_inverted_index.UniDoc-Bench, | 51 |
| abstract_inverted_index.adjudication. | 108 |
| abstract_inverted_index.benchmarking, | 177 |
| abstract_inverted_index.summarization, | 89 |
| abstract_inverted_index.embedding-based | 158 |
| abstract_inverted_index.apples-to-apples | 111 |
| abstract_inverted_index.document-centric | 42 |
| abstract_inverted_index.retrieval-augmented | 1 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 3 |
| citation_normalized_percentile |