CoT Referring: Improving Referring Expression Tasks with Grounded Reasoning Article Swipe
Qi Dong
,
Luis Figueroa
,
Handong Zhao
,
Kushal Kafle
,
Jason Kuen
,
Zhihong Ding
,
Scott Cohen
,
Yun Fu
·
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2510.06243
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2510.06243
Referring Expression Comprehension and Segmentation are critical tasks for assessing the integration of language understanding and image comprehension, serving as benchmarks for Multimodal Large Language Models (MLLMs) capabilities. To address these challenges, we propose a new strategy, CoT Referring, which enhances model reasoning across modalities through a structured, chain-of-thought training data structure. Our approach systematically parses textual structures to a sequential referring step, where in each step it identifies relationships and ensures consistent reference alignment, thereby improving accuracy in complex query scenarios. We restructure the training data to enforce a new output form, providing new annotations for existing datasets and compiling an evaluation benchmark from existing resources. This benchmark is designed explicitly for complex referring cases. We also integrate detection and segmentation capabilities into a unified MLLM framework, training it with a novel adaptive weighted loss to optimize performance. Experimental results on our curated benchmark and RefCOCO/+/g demonstrate the effectiveness of our approach, with a notable increase of 2.5%+ over baseline models.
Related Topics
Concepts
No concepts available.
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2510.06243
- https://arxiv.org/pdf/2510.06243
- OA Status
- green
- OpenAlex ID
- https://openalex.org/W4415314425
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4415314425Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2510.06243Digital Object Identifier
- Title
-
CoT Referring: Improving Referring Expression Tasks with Grounded ReasoningWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2025Year of publication
- Publication date
-
2025-10-03Full publication date if available
- Authors
-
Qi Dong, Luis Figueroa, Handong Zhao, Kushal Kafle, Jason Kuen, Zhihong Ding, Scott Cohen, Yun FuList of authors in order
- Landing page
-
https://arxiv.org/abs/2510.06243Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2510.06243Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2510.06243Direct OA link when available
- Cited by
-
0Total citation count in OpenAlex
Full payload
| id | https://openalex.org/W4415314425 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2510.06243 |
| ids.doi | https://doi.org/10.48550/arxiv.2510.06243 |
| ids.openalex | https://openalex.org/W4415314425 |
| fwci | |
| type | preprint |
| title | CoT Referring: Improving Referring Expression Tasks with Grounded Reasoning |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10028 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9794999957084656 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Topic Modeling |
| topics[1].id | https://openalex.org/T10181 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9782999753952026 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Natural Language Processing Techniques |
| topics[2].id | https://openalex.org/T12031 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9667999744415283 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1702 |
| topics[2].subfield.display_name | Artificial Intelligence |
| topics[2].display_name | Speech and dialogue systems |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2510.06243 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2510.06243 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2510.06243 |
| locations[1].id | doi:10.48550/arxiv.2510.06243 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2510.06243 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5100676473 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-7500-6541 |
| authorships[0].author.display_name | Qi Dong |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Dong, Qihua |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5026777424 |
| authorships[1].author.orcid | |
| authorships[1].author.display_name | Luis Figueroa |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Figueroa, Luis |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5024078415 |
| authorships[2].author.orcid | https://orcid.org/0000-0003-3775-2954 |
| authorships[2].author.display_name | Handong Zhao |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Zhao, Handong |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5047193437 |
| authorships[3].author.orcid | https://orcid.org/0000-0002-0847-7861 |
| authorships[3].author.display_name | Kushal Kafle |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Kafle, Kushal |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5076366439 |
| authorships[4].author.orcid | https://orcid.org/0000-0001-5099-8145 |
| authorships[4].author.display_name | Jason Kuen |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Kuen, Jason |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5079713811 |
| authorships[5].author.orcid | https://orcid.org/0009-0001-1088-3260 |
| authorships[5].author.display_name | Zhihong Ding |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Ding, Zhihong |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5055156362 |
| authorships[6].author.orcid | https://orcid.org/0000-0002-3459-6899 |
| authorships[6].author.display_name | Scott Cohen |
| authorships[6].author_position | middle |
| authorships[6].raw_author_name | Cohen, Scott |
| authorships[6].is_corresponding | False |
| authorships[7].author.id | https://openalex.org/A5035893627 |
| authorships[7].author.orcid | |
| authorships[7].author.display_name | Yun Fu |
| authorships[7].author_position | last |
| authorships[7].raw_author_name | Fu, Yun |
| authorships[7].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2510.06243 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-18T00:00:00 |
| display_name | CoT Referring: Improving Referring Expression Tasks with Grounded Reasoning |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10028 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9794999957084656 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Topic Modeling |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2510.06243 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2510.06243 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2510.06243 |
| primary_location.id | pmh:oai:arXiv.org:2510.06243 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2510.06243 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2510.06243 |
| publication_date | 2025-10-03 |
| publication_year | 2025 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 34, 46, 59, 89, 124, 131, 154 |
| abstract_inverted_index.To | 28 |
| abstract_inverted_index.We | 82, 116 |
| abstract_inverted_index.an | 101 |
| abstract_inverted_index.as | 19 |
| abstract_inverted_index.in | 64, 78 |
| abstract_inverted_index.is | 109 |
| abstract_inverted_index.it | 67, 129 |
| abstract_inverted_index.of | 12, 150, 157 |
| abstract_inverted_index.on | 141 |
| abstract_inverted_index.to | 58, 87, 136 |
| abstract_inverted_index.we | 32 |
| abstract_inverted_index.CoT | 37 |
| abstract_inverted_index.Our | 52 |
| abstract_inverted_index.and | 3, 15, 70, 99, 120, 145 |
| abstract_inverted_index.are | 5 |
| abstract_inverted_index.for | 8, 21, 96, 112 |
| abstract_inverted_index.new | 35, 90, 94 |
| abstract_inverted_index.our | 142, 151 |
| abstract_inverted_index.the | 10, 84, 148 |
| abstract_inverted_index.MLLM | 126 |
| abstract_inverted_index.This | 107 |
| abstract_inverted_index.also | 117 |
| abstract_inverted_index.data | 50, 86 |
| abstract_inverted_index.each | 65 |
| abstract_inverted_index.from | 104 |
| abstract_inverted_index.into | 123 |
| abstract_inverted_index.loss | 135 |
| abstract_inverted_index.over | 159 |
| abstract_inverted_index.step | 66 |
| abstract_inverted_index.with | 130, 153 |
| abstract_inverted_index.2.5%+ | 158 |
| abstract_inverted_index.Large | 23 |
| abstract_inverted_index.form, | 92 |
| abstract_inverted_index.image | 16 |
| abstract_inverted_index.model | 41 |
| abstract_inverted_index.novel | 132 |
| abstract_inverted_index.query | 80 |
| abstract_inverted_index.step, | 62 |
| abstract_inverted_index.tasks | 7 |
| abstract_inverted_index.these | 30 |
| abstract_inverted_index.where | 63 |
| abstract_inverted_index.which | 39 |
| abstract_inverted_index.Models | 25 |
| abstract_inverted_index.across | 43 |
| abstract_inverted_index.cases. | 115 |
| abstract_inverted_index.output | 91 |
| abstract_inverted_index.parses | 55 |
| abstract_inverted_index.(MLLMs) | 26 |
| abstract_inverted_index.address | 29 |
| abstract_inverted_index.complex | 79, 113 |
| abstract_inverted_index.curated | 143 |
| abstract_inverted_index.enforce | 88 |
| abstract_inverted_index.ensures | 71 |
| abstract_inverted_index.models. | 161 |
| abstract_inverted_index.notable | 155 |
| abstract_inverted_index.propose | 33 |
| abstract_inverted_index.results | 140 |
| abstract_inverted_index.serving | 18 |
| abstract_inverted_index.textual | 56 |
| abstract_inverted_index.thereby | 75 |
| abstract_inverted_index.through | 45 |
| abstract_inverted_index.unified | 125 |
| abstract_inverted_index.Language | 24 |
| abstract_inverted_index.accuracy | 77 |
| abstract_inverted_index.adaptive | 133 |
| abstract_inverted_index.approach | 53 |
| abstract_inverted_index.baseline | 160 |
| abstract_inverted_index.critical | 6 |
| abstract_inverted_index.datasets | 98 |
| abstract_inverted_index.designed | 110 |
| abstract_inverted_index.enhances | 40 |
| abstract_inverted_index.existing | 97, 105 |
| abstract_inverted_index.increase | 156 |
| abstract_inverted_index.language | 13 |
| abstract_inverted_index.optimize | 137 |
| abstract_inverted_index.training | 49, 85, 128 |
| abstract_inverted_index.weighted | 134 |
| abstract_inverted_index.Referring | 0 |
| abstract_inverted_index.approach, | 152 |
| abstract_inverted_index.assessing | 9 |
| abstract_inverted_index.benchmark | 103, 108, 144 |
| abstract_inverted_index.compiling | 100 |
| abstract_inverted_index.detection | 119 |
| abstract_inverted_index.improving | 76 |
| abstract_inverted_index.integrate | 118 |
| abstract_inverted_index.providing | 93 |
| abstract_inverted_index.reasoning | 42 |
| abstract_inverted_index.reference | 73 |
| abstract_inverted_index.referring | 61, 114 |
| abstract_inverted_index.strategy, | 36 |
| abstract_inverted_index.Expression | 1 |
| abstract_inverted_index.Multimodal | 22 |
| abstract_inverted_index.Referring, | 38 |
| abstract_inverted_index.alignment, | 74 |
| abstract_inverted_index.benchmarks | 20 |
| abstract_inverted_index.consistent | 72 |
| abstract_inverted_index.evaluation | 102 |
| abstract_inverted_index.explicitly | 111 |
| abstract_inverted_index.framework, | 127 |
| abstract_inverted_index.identifies | 68 |
| abstract_inverted_index.modalities | 44 |
| abstract_inverted_index.resources. | 106 |
| abstract_inverted_index.scenarios. | 81 |
| abstract_inverted_index.sequential | 60 |
| abstract_inverted_index.structure. | 51 |
| abstract_inverted_index.structures | 57 |
| abstract_inverted_index.RefCOCO/+/g | 146 |
| abstract_inverted_index.annotations | 95 |
| abstract_inverted_index.challenges, | 31 |
| abstract_inverted_index.demonstrate | 147 |
| abstract_inverted_index.integration | 11 |
| abstract_inverted_index.restructure | 83 |
| abstract_inverted_index.structured, | 47 |
| abstract_inverted_index.Experimental | 139 |
| abstract_inverted_index.Segmentation | 4 |
| abstract_inverted_index.capabilities | 122 |
| abstract_inverted_index.performance. | 138 |
| abstract_inverted_index.segmentation | 121 |
| abstract_inverted_index.Comprehension | 2 |
| abstract_inverted_index.capabilities. | 27 |
| abstract_inverted_index.effectiveness | 149 |
| abstract_inverted_index.relationships | 69 |
| abstract_inverted_index.understanding | 14 |
| abstract_inverted_index.comprehension, | 17 |
| abstract_inverted_index.systematically | 54 |
| abstract_inverted_index.chain-of-thought | 48 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 8 |
| citation_normalized_percentile |