Boosting with Tempered Exponential Measures Article Swipe

MAKE IMPACT

Richard Nock , Ehsan Amid , Manfred K. Warmuth · YOU? · · 2023 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2306.05487

One of the most popular ML algorithms, AdaBoost, can be derived from the dual of a relative entropy minimization problem subject to the fact that the positive weights on the examples sum to one. Essentially, harder examples receive higher probabilities. We generalize this setup to the recently introduced {\it tempered exponential measure}s (TEMs) where normalization is enforced on a specific power of the measure and not the measure itself. TEMs are indexed by a parameter $t$ and generalize exponential families ($t=1$). Our algorithm, $t$-AdaBoost, recovers AdaBoost~as a special case ($t=1$). We show that $t$-AdaBoost retains AdaBoost's celebrated exponential convergence rate when $t\in [0,1)$ while allowing a slight improvement of the rate's hidden constant compared to $t=1$. $t$-AdaBoost partially computes on a generalization of classical arithmetic over the reals and brings notable properties like guaranteed bounded leveraging coefficients for $t\in [0,1)$. From the loss that $t$-AdaBoost minimizes (a generalization of the exponential loss), we show how to derive a new family of {\it tempered} losses for the induction of domain-partitioning classifiers like decision trees. Crucially, strict properness is ensured for all while their boosting rates span the full known spectrum. Experiments using $t$-AdaBoost+trees display that significant leverage can be achieved by tuning $t$.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2306.05487

PDF

https://arxiv.org/pdf/2306.05487

OA Status

green

Related Works

10

OpenAlex ID

https://openalex.org/W4380352232

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4380352232

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2306.05487

Digital Object Identifier

Title

Boosting with Tempered Exponential Measures

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2023

Year of publication

Publication date

2023-06-08

Full publication date if available

Authors

Richard Nock, Ehsan Amid, Manfred K. Warmuth

List of authors in order

Landing page

https://arxiv.org/abs/2306.05487

Publisher landing page

PDF URL

https://arxiv.org/pdf/2306.05487

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2306.05487

Direct OA link when available

Concepts

AdaBoost, Boosting (machine learning), Mathematics, Exponential function, Word error rate, Normalization (sociology), Leverage (statistics), Algorithm, Artificial intelligence, Combinatorics, Applied mathematics, Computer science, Pattern recognition (psychology), Statistics, Mathematical analysis, Support vector machine, Sociology, Anthropology

Top concepts (fields/topics) attached by OpenAlex

Cited by

0

Total citation count in OpenAlex

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload