Salience DETR: Enhancing Detection Transformer with Hierarchical Salience Filtering Refinement Article Swipe

PDF

Xiuquan Hou , Meiqin Liu , Senlin Zhang , Ping Wei , Badong Chen ·

YOU? · · 2024 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2403.16131

DETR-like methods have significantly increased detection performance in an end-to-end manner. The mainstream two-stage frameworks of them perform dense self-attention and select a fraction of queries for sparse cross-attention, which is proven effective for improving performance but also introduces a heavy computational burden and high dependence on stable query selection. This paper demonstrates that suboptimal two-stage selection strategies result in scale bias and redundancy due to the mismatch between selected queries and objects in two-stage initialization. To address these issues, we propose hierarchical salience filtering refinement, which performs transformer encoding only on filtered discriminative queries, for a better trade-off between computational efficiency and precision. The filtering process overcomes scale bias through a novel scale-independent salience supervision. To compensate for the semantic misalignment among queries, we introduce elaborate query refinement modules for stable two-stage initialization. Based on above improvements, the proposed Salience DETR achieves significant improvements of +4.0% AP, +0.2% AP, +4.4% AP on three challenging task-specific detection datasets, as well as 49.2% AP on COCO 2017 with less FLOPs. The code is available at https://github.com/xiuqhou/Salience-DETR.

Related Topics

Salience (Neuroscience)

Transformer

Computer Science

Artificial Intelligence

Engineering

Electrical Engineering

Voltage

Concepts

Salience (neuroscience) Transformer Computer science Psychology Artificial intelligence Engineering Electrical engineering Voltage

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2403.16131
PDF: https://arxiv.org/pdf/2403.16131
OA Status: green
Cited By: 2
Related Works: 10
OpenAlex ID: https://openalex.org/W4393213015

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4393213015

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2403.16131

Digital Object Identifier
Title: Salience DETR: Enhancing Detection Transformer with Hierarchical Salience Filtering Refinement

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2024

Year of publication
Publication date: 2024-03-24

Full publication date if available
Authors: Xiuquan Hou, Meiqin Liu, Senlin Zhang, Ping Wei, Badong Chen

List of authors in order
Landing page: https://arxiv.org/abs/2403.16131

Publisher landing page
PDF URL: https://arxiv.org/pdf/2403.16131

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2403.16131

Direct OA link when available
Concepts: Salience (neuroscience), Transformer, Computer science, Psychology, Artificial intelligence, Engineering, Electrical engineering, Voltage

Top concepts (fields/topics) attached by OpenAlex
Cited by: 2

Total citation count in OpenAlex
Citations by year (recent): 2025: 1, 2024: 1

Per-year citation counts (last 5 years)
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4393213015
doi	https://doi.org/10.48550/arxiv.2403.16131
ids.doi	https://doi.org/10.48550/arxiv.2403.16131
ids.openalex	https://openalex.org/W4393213015
fwci
type	preprint
title	Salience DETR: Enhancing Detection Transformer with Hierarchical Salience Filtering Refinement
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T11605
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	0.9621999859809875
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1707
topics[0].subfield.display_name	Computer Vision and Pattern Recognition
topics[0].display_name	Visual Attention and Saliency Detection
topics[1].id	https://openalex.org/T12389
topics[1].field.id	https://openalex.org/fields/22
topics[1].field.display_name	Engineering
topics[1].score	0.9050999879837036
topics[1].domain.id	https://openalex.org/domains/3
topics[1].domain.display_name	Physical Sciences
topics[1].subfield.id	https://openalex.org/subfields/2202
topics[1].subfield.display_name	Aerospace Engineering
topics[1].display_name	Infrared Target Detection Methodologies
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C108154423
concepts[0].level	2
concepts[0].score	0.8058507442474365
concepts[0].wikidata	https://www.wikidata.org/wiki/Q1469792
concepts[0].display_name	Salience (neuroscience)
concepts[1].id	https://openalex.org/C66322947
concepts[1].level	3
concepts[1].score	0.5432719588279724
concepts[1].wikidata	https://www.wikidata.org/wiki/Q11658
concepts[1].display_name	Transformer
concepts[2].id	https://openalex.org/C41008148
concepts[2].level	0
concepts[2].score	0.4134792387485504
concepts[2].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[2].display_name	Computer science
concepts[3].id	https://openalex.org/C15744967
concepts[3].level	0
concepts[3].score	0.3613174557685852
concepts[3].wikidata	https://www.wikidata.org/wiki/Q9418
concepts[3].display_name	Psychology
concepts[4].id	https://openalex.org/C154945302
concepts[4].level	1
concepts[4].score	0.3051599860191345
concepts[4].wikidata	https://www.wikidata.org/wiki/Q11660
concepts[4].display_name	Artificial intelligence
concepts[5].id	https://openalex.org/C127413603
concepts[5].level	0
concepts[5].score	0.11473685503005981
concepts[5].wikidata	https://www.wikidata.org/wiki/Q11023
concepts[5].display_name	Engineering
concepts[6].id	https://openalex.org/C119599485
concepts[6].level	1
concepts[6].score	0.0679081380367279
concepts[6].wikidata	https://www.wikidata.org/wiki/Q43035
concepts[6].display_name	Electrical engineering
concepts[7].id	https://openalex.org/C165801399
concepts[7].level	2
concepts[7].score	0.0
concepts[7].wikidata	https://www.wikidata.org/wiki/Q25428
concepts[7].display_name	Voltage
keywords[0].id	https://openalex.org/keywords/salience
keywords[0].score	0.8058507442474365
keywords[0].display_name	Salience (neuroscience)
keywords[1].id	https://openalex.org/keywords/transformer
keywords[1].score	0.5432719588279724
keywords[1].display_name	Transformer
keywords[2].id	https://openalex.org/keywords/computer-science
keywords[2].score	0.4134792387485504
keywords[2].display_name	Computer science
keywords[3].id	https://openalex.org/keywords/psychology
keywords[3].score	0.3613174557685852
keywords[3].display_name	Psychology
keywords[4].id	https://openalex.org/keywords/artificial-intelligence
keywords[4].score	0.3051599860191345
keywords[4].display_name	Artificial intelligence
keywords[5].id	https://openalex.org/keywords/engineering
keywords[5].score	0.11473685503005981
keywords[5].display_name	Engineering
keywords[6].id	https://openalex.org/keywords/electrical-engineering
keywords[6].score	0.0679081380367279
keywords[6].display_name	Electrical engineering
language	en
locations[0].id	pmh:oai:arXiv.org:2403.16131
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/2403.16131
locations[0].version	submittedVersion
locations[0].raw_type	text
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2403.16131
locations[1].id	doi:10.48550/arxiv.2403.16131
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2403.16131
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5069152360
authorships[0].author.orcid
authorships[0].author.display_name	Xiuquan Hou
authorships[0].author_position	first
authorships[0].raw_author_name	Hou, Xiuquan
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5033808402
authorships[1].author.orcid	https://orcid.org/0000-0003-0693-6574
authorships[1].author.display_name	Meiqin Liu
authorships[1].author_position	middle
authorships[1].raw_author_name	Liu, Meiqin
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5003643230
authorships[2].author.orcid	https://orcid.org/0000-0001-5117-3110
authorships[2].author.display_name	Senlin Zhang
authorships[2].author_position	middle
authorships[2].raw_author_name	Zhang, Senlin
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5101947241
authorships[3].author.orcid	https://orcid.org/0000-0002-8535-9527
authorships[3].author.display_name	Ping Wei
authorships[3].author_position	middle
authorships[3].raw_author_name	Wei, Ping
authorships[3].is_corresponding	False
authorships[4].author.id	https://openalex.org/A5077852542
authorships[4].author.orcid	https://orcid.org/0000-0003-1710-3818
authorships[4].author.display_name	Badong Chen
authorships[4].author_position	last
authorships[4].raw_author_name	Chen, Badong
authorships[4].is_corresponding	False
has_content.pdf	True
has_content.grobid_xml	True
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2403.16131
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	Salience DETR: Enhancing Detection Transformer with Hierarchical Salience Filtering Refinement
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T06:51:31.235846
primary_topic.id	https://openalex.org/T11605
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	0.9621999859809875
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1707
primary_topic.subfield.display_name	Computer Vision and Pattern Recognition
primary_topic.display_name	Visual Attention and Saliency Detection
related_works	https://openalex.org/W2748952813, https://openalex.org/W2390279801, https://openalex.org/W2358668433, https://openalex.org/W2376932109, https://openalex.org/W2001405890, https://openalex.org/W2382290278, https://openalex.org/W2117158897, https://openalex.org/W2478288626, https://openalex.org/W4391913857, https://openalex.org/W2350741829
cited_by_count	2
counts_by_year[0].year	2025
counts_by_year[0].cited_by_count	1
counts_by_year[1].year	2024
counts_by_year[1].cited_by_count	1
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2403.16131
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/2403.16131
best_oa_location.version	submittedVersion
best_oa_location.raw_type	text
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2403.16131
primary_location.id	pmh:oai:arXiv.org:2403.16131
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/2403.16131
primary_location.version	submittedVersion
primary_location.raw_type	text
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2403.16131
publication_date	2024-03-24
publication_year	2024
referenced_works_count	0
abstract_inverted_index.a	22, 39, 96, 111
abstract_inverted_index.AP	151, 162
abstract_inverted_index.To	76, 116
abstract_inverted_index.an	8
abstract_inverted_index.as	158, 160
abstract_inverted_index.at	173
abstract_inverted_index.in	7, 59, 73
abstract_inverted_index.is	30, 171
abstract_inverted_index.of	15, 24, 145
abstract_inverted_index.on	46, 91, 135, 152, 163
abstract_inverted_index.to	65
abstract_inverted_index.we	80, 124
abstract_inverted_index.AP,	147, 149
abstract_inverted_index.The	11, 104, 169
abstract_inverted_index.and	20, 43, 62, 71, 102
abstract_inverted_index.but	36
abstract_inverted_index.due	64
abstract_inverted_index.for	26, 33, 95, 118, 130
abstract_inverted_index.the	66, 119, 138
abstract_inverted_index.2017	165
abstract_inverted_index.COCO	164
abstract_inverted_index.DETR	141
abstract_inverted_index.This	50
abstract_inverted_index.also	37
abstract_inverted_index.bias	61, 109
abstract_inverted_index.code	170
abstract_inverted_index.have	2
abstract_inverted_index.high	44
abstract_inverted_index.less	167
abstract_inverted_index.only	90
abstract_inverted_index.that	53
abstract_inverted_index.them	16
abstract_inverted_index.well	159
abstract_inverted_index.with	166
abstract_inverted_index.+0.2%	148
abstract_inverted_index.+4.0%	146
abstract_inverted_index.+4.4%	150
abstract_inverted_index.49.2%	161
abstract_inverted_index.Based	134
abstract_inverted_index.above	136
abstract_inverted_index.among	122
abstract_inverted_index.dense	18
abstract_inverted_index.heavy	40
abstract_inverted_index.novel	112
abstract_inverted_index.paper	51
abstract_inverted_index.query	48, 127
abstract_inverted_index.scale	60, 108
abstract_inverted_index.these	78
abstract_inverted_index.three	153
abstract_inverted_index.which	29, 86
abstract_inverted_index.FLOPs.	168
abstract_inverted_index.better	97
abstract_inverted_index.burden	42
abstract_inverted_index.proven	31
abstract_inverted_index.result	58
abstract_inverted_index.select	21
abstract_inverted_index.sparse	27
abstract_inverted_index.stable	47, 131
abstract_inverted_index.address	77
abstract_inverted_index.between	68, 99
abstract_inverted_index.issues,	79
abstract_inverted_index.manner.	10
abstract_inverted_index.methods	1
abstract_inverted_index.modules	129
abstract_inverted_index.objects	72
abstract_inverted_index.perform	17
abstract_inverted_index.process	106
abstract_inverted_index.propose	81
abstract_inverted_index.queries	25, 70
abstract_inverted_index.through	110
abstract_inverted_index.Salience	140
abstract_inverted_index.achieves	142
abstract_inverted_index.encoding	89
abstract_inverted_index.filtered	92
abstract_inverted_index.fraction	23
abstract_inverted_index.mismatch	67
abstract_inverted_index.performs	87
abstract_inverted_index.proposed	139
abstract_inverted_index.queries,	94, 123
abstract_inverted_index.salience	83, 114
abstract_inverted_index.selected	69
abstract_inverted_index.semantic	120
abstract_inverted_index.DETR-like	0
abstract_inverted_index.available	172
abstract_inverted_index.datasets,	157
abstract_inverted_index.detection	5, 156
abstract_inverted_index.effective	32
abstract_inverted_index.elaborate	126
abstract_inverted_index.filtering	84, 105
abstract_inverted_index.improving	34
abstract_inverted_index.increased	4
abstract_inverted_index.introduce	125
abstract_inverted_index.overcomes	107
abstract_inverted_index.selection	56
abstract_inverted_index.trade-off	98
abstract_inverted_index.two-stage	13, 55, 74, 132
abstract_inverted_index.compensate	117
abstract_inverted_index.dependence	45
abstract_inverted_index.efficiency	101
abstract_inverted_index.end-to-end	9
abstract_inverted_index.frameworks	14
abstract_inverted_index.introduces	38
abstract_inverted_index.mainstream	12
abstract_inverted_index.precision.	103
abstract_inverted_index.redundancy	63
abstract_inverted_index.refinement	128
abstract_inverted_index.selection.	49
abstract_inverted_index.strategies	57
abstract_inverted_index.suboptimal	54
abstract_inverted_index.challenging	154
abstract_inverted_index.performance	6, 35
abstract_inverted_index.refinement,	85
abstract_inverted_index.significant	143
abstract_inverted_index.transformer	88
abstract_inverted_index.demonstrates	52
abstract_inverted_index.hierarchical	82
abstract_inverted_index.improvements	144
abstract_inverted_index.misalignment	121
abstract_inverted_index.supervision.	115
abstract_inverted_index.computational	41, 100
abstract_inverted_index.improvements,	137
abstract_inverted_index.significantly	3
abstract_inverted_index.task-specific	155
abstract_inverted_index.discriminative	93
abstract_inverted_index.self-attention	19
abstract_inverted_index.initialization.	75, 133
abstract_inverted_index.cross-attention,	28
abstract_inverted_index.scale-independent	113
abstract_inverted_index.https://github.com/xiuqhou/Salience-DETR.	174
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	5
citation_normalized_percentile