Learning Spatial-Frequency Transformer for Visual Object Tracking Article Swipe

MAKE IMPACT

Chuanming Tang , Xiao Wang , Yuanchao Bai , Zhe Wu , Jianlin Zhang , Yongmei Huang · YOU? · · 2022 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2208.08829

Recent trackers adopt the Transformer to combine or replace the widely used ResNet as their new backbone network. Although their trackers work well in regular scenarios, however, they simply flatten the 2D features into a sequence to better match the Transformer. We believe these operations ignore the spatial prior of the target object which may lead to sub-optimal results only. In addition, many works demonstrate that self-attention is actually a low-pass filter, which is independent of input features or key/queries. That is to say, it may suppress the high-frequency component of the input features and preserve or even amplify the low-frequency information. To handle these issues, in this paper, we propose a unified Spatial-Frequency Transformer that models the Gaussian spatial Prior and High-frequency emphasis Attention (GPHA) simultaneously. To be specific, Gaussian spatial prior is generated using dual Multi-Layer Perceptrons (MLPs) and injected into the similarity matrix produced by multiplying Query and Key features in self-attention. The output will be fed into a Softmax layer and then decomposed into two components, i.e., the direct signal and high-frequency signal. The low- and high-pass branches are rescaled and combined to achieve all-pass, therefore, the high-frequency features will be protected well in stacked self-attention layers. We further integrate the Spatial-Frequency Transformer into the Siamese tracking framework and propose a novel tracking algorithm, termed SFTransT. The cross-scale fusion based SwinTransformer is adopted as the backbone, and also a multi-head cross-attention module is used to boost the interaction between search and template features. The output will be fed into the tracking head for target localization. Extensive experiments on both short-term and long-term tracking benchmarks all demonstrate the effectiveness of our proposed framework.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2208.08829

PDF

https://arxiv.org/pdf/2208.08829

OA Status

green

Cited By

1

Related Works

10

OpenAlex ID

https://openalex.org/W4292436284

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4292436284

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2208.08829

Digital Object Identifier

Title

Learning Spatial-Frequency Transformer for Visual Object Tracking

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2022

Year of publication

Publication date

2022-08-18

Full publication date if available

Authors

Chuanming Tang, Xiao Wang, Yuanchao Bai, Zhe Wu, Jianlin Zhang, Yongmei Huang

List of authors in order

Landing page

https://arxiv.org/abs/2208.08829

Publisher landing page

PDF URL

https://arxiv.org/pdf/2208.08829

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2208.08829

Direct OA link when available

Concepts

Computer science, Transformer, Artificial intelligence, Softmax function, BitTorrent tracker, Pattern recognition (psychology), Computer vision, Eye tracking, Deep learning, Engineering, Voltage, Electrical engineering

Top concepts (fields/topics) attached by OpenAlex

Cited by

1

Total citation count in OpenAlex

Citations by year (recent)

2023: 1

Per-year citation counts (last 5 years)

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload