Re-examining Distillation For Continual Object Detection Article Swipe

MAKE IMPACT

Eli Verwimp , Kuo Yang , Sarah Parisot , Hong Lanqing , Steven McDonagh , Eduardo Pérez-Pellitero , Matthias De Lange , Tinne Tuytelaars · YOU? · · 2022 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2204.01407

Training models continually to detect and classify objects, from new classes and new domains, remains an open problem. In this work, we conduct a thorough analysis of why and how object detection models forget catastrophically. We focus on distillation-based approaches in two-stage networks; the most-common strategy employed in contemporary continual object detection work.Distillation aims to transfer the knowledge of a model trained on previous tasks -- the teacher -- to a new model -- the student -- while it learns the new task. We show that this works well for the region proposal network, but that wrong, yet overly confident teacher predictions prevent student models from effective learning of the classification head. Our analysis provides a foundation that allows us to propose improvements for existing techniques by detecting incorrect teacher predictions, based on current ground-truth labels, and by employing an adaptive Huber loss as opposed to the mean squared error for the distillation loss in the classification heads. We evidence that our strategy works not only in a class incremental setting, but also in domain incremental settings, which constitute a realistic context, likely to be the setting of representative real-world problems.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2204.01407

PDF

https://arxiv.org/pdf/2204.01407

OA Status

green

Cited By

5

Related Works

10

OpenAlex ID

https://openalex.org/W4226436870

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4226436870

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2204.01407

Digital Object Identifier

Title

Re-examining Distillation For Continual Object Detection

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2022

Year of publication

Publication date

2022-04-04

Full publication date if available

Authors

Eli Verwimp, Kuo Yang, Sarah Parisot, Hong Lanqing, Steven McDonagh, Eduardo Pérez-Pellitero, Matthias De Lange, Tinne Tuytelaars

List of authors in order

Landing page

https://arxiv.org/abs/2204.01407

Publisher landing page

PDF URL

https://arxiv.org/pdf/2204.01407

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2204.01407

Direct OA link when available

Concepts

Computer science, Artificial intelligence, Object (grammar), Machine learning, Context (archaeology), Focus (optics), Distillation, Class (philosophy), Object detection, Domain (mathematical analysis), Task (project management), Pattern recognition (psychology), Engineering, Mathematics, Optics, Paleontology, Organic chemistry, Chemistry, Systems engineering, Physics, Mathematical analysis, Biology

Top concepts (fields/topics) attached by OpenAlex

Cited by

5

Total citation count in OpenAlex

Citations by year (recent)

2023: 5

Per-year citation counts (last 5 years)

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload