Robert C. Williamson
YOU?
Author Swipe
View article: Three Types of Calibration with Properties and their Semantic and Formal Relationships
Three Types of Calibration with Properties and their Semantic and Formal Relationships Open
Fueled by discussions around "trustworthiness" and algorithmic fairness, calibration of predictive systems has regained scholars attention. The vanilla definition and understanding of calibration is, simply put, on all days on which the ra…
View article: Decision making, symmetry and structure: Justifying causal interventions
Decision making, symmetry and structure: Justifying causal interventions Open
We can use structural causal models (SCMs) to help us evaluate the consequences of actions given data. SCMs identify actions with structural interventions. A careful decision maker may wonder whether this identification is justified. We se…
View article: Fairness and Randomness in Machine Learning: Statistical Independence and Relativization
Fairness and Randomness in Machine Learning: Statistical Independence and Relativization Open
Fair Machine Learning endeavors to prevent unfairness arising in the context of machine learning applications embedded in society. To this end, several mathematical fairness notions have been proposed. The most known and used notions turn …
View article: Scoring Rules and Calibration for Imprecise Probabilities
Scoring Rules and Calibration for Imprecise Probabilities Open
What does it mean to say that, for example, the probability for rain tomorrow is between 20% and 30%? The theory for the evaluation of precise probabilistic forecasts is well-developed and is grounded in the key concepts of proper scoring …
View article: Formalising causal inference as prediction on a target population
Formalising causal inference as prediction on a target population Open
The standard approach to causal modelling especially in social and health sciences is the potential outcomes framework due to Neyman and Rubin. In this framework, observations are thought to be drawn from a distribution over variables of i…
View article: We should avoid the assumption of data-generating probability distributions in social settings
We should avoid the assumption of data-generating probability distributions in social settings Open
Machine Learning research, including work promoting fair or equitable algorithms, heavily relies on the concept of a data-generating probability distribution. The standard presumption is that since data points are 'sampled from' such a dis…
View article: Limits to Predicting Online Speech Using Large Language Models
Limits to Predicting Online Speech Using Large Language Models Open
We study the predictability of online speech on social media, and whether predictability improves with information outside a user's own posts. Recent theoretical results suggest that posts from a user's social circle are as predictive of t…
View article: An Axiomatic Approach to Loss Aggregation and an Adapted Aggregating Algorithm
An Axiomatic Approach to Loss Aggregation and an Adapted Aggregating Algorithm Open
Supervised learning has gone beyond the expected risk minimization framework. Central to most of these developments is the introduction of more general aggregation functions for losses incurred by the learner. In this paper, we turn toward…
View article: Insights From Insurance for Fair Machine Learning
Insights From Insurance for Fair Machine Learning Open
We argue that insurance can act as an analogon for the social situatedness of machine learning systems, hence allowing machine learning scholars to take insights from the rich and interdisciplinary insurance literature. Tracing the interac…
View article: Data Models With Two Manifestations of Imprecision
Data Models With Two Manifestations of Imprecision Open
Motivated by recently emerging problems in machine learning and statistics, we propose data models which relax the familiar i.i.d. assumption. In essence, we seek to understand what it means for data to come from a set of probability measu…
View article: Geometry and Stability of Supervised Learning Problems
Geometry and Stability of Supervised Learning Problems Open
We introduce a notion of distance between supervised learning problems, which we call the Risk distance. This distance, inspired by optimal transport, facilitates stability results; one can quantify how seriously issues like sampling bias,…
View article: Forecast Evaluation and the Relationship of Regret and Calibration
Forecast Evaluation and the Relationship of Regret and Calibration Open
Machine learning is about forecasting. When the forecasts come with an evaluation metric the forecasts become useful. What are reasonable evaluation metrics? How do existing evaluation metrics relate? In this work, we provide a general str…
View article: Systems of Precision: Coherent Probabilities on Pre-Dynkin Systems and Coherent Previsions on Linear Subspaces
Systems of Precision: Coherent Probabilities on Pre-Dynkin Systems and Coherent Previsions on Linear Subspaces Open
In the literature on imprecise probability, little attention is paid to the fact that imprecise probabilities are precise on a set of events. We call these sets systems of precision. We show that, under mild assumptions, the system of prec…
View article: Corruptions of Supervised Learning Problems: Typology and Mitigations
Corruptions of Supervised Learning Problems: Typology and Mitigations Open
Corruption is notoriously widespread in data collection. Despite extensive research, the existing literature predominantly focuses on specific settings and learning scenarios, lacking a unified view of corruption modelization and mitigatio…
View article: Insights From Insurance for Fair Machine Learning
Insights From Insurance for Fair Machine Learning Open
We argue that insurance can act as an analogon for the social situatedness of machine learning systems, hence allowing machine learning scholars to take insights from the rich and interdisciplinary insurance literature. Tracing the interac…
View article: The Geometry of Mixability
The Geometry of Mixability Open
Mixable loss functions are of fundamental importance in the context of prediction with expert advice in the online setting since they characterize fast learning rates. By re-interpreting properness from the point of view of differential ge…
View article: On the Richness of Calibration
On the Richness of Calibration Open
Probabilistic predictions can be evaluated through comparisons with observed label frequencies, that is, through the lens of calibration. Recent scholarship on algorithmic fairness has started to look at a growing variety of calibration-ba…
View article: Strictly Frequentist Imprecise Probability
Strictly Frequentist Imprecise Probability Open
Strict frequentism defines probability as the limiting relative frequency in an infinite sequence. What if the limit does not exist? We present a broader theory, which is applicable also to random phenomena that exhibit diverging relative …
View article: Systems of Precision: Coherent Probabilities on Pre-Dynkin-Systems and Coherent Previsions on Linear Subspaces
Systems of Precision: Coherent Probabilities on Pre-Dynkin-Systems and Coherent Previsions on Linear Subspaces Open
In literature on imprecise probability little attention is paid to the fact that imprecise probabilities are precise on a set of events. We call these sets systems of precision. We show that, under mild assumptions, the system of precision…
View article: The Geometry and Calculus of Losses
The Geometry and Calculus of Losses Open
Statistical decision problems lie at the heart of statistical machine learning. The simplest problems are binary and multiclass classification and class probability estimation. Central to their definition is the choice of loss function, wh…
View article: Tailoring to the Tails: Risk Measures for Fine-Grained Tail Sensitivity
Tailoring to the Tails: Risk Measures for Fine-Grained Tail Sensitivity Open
Expected risk minimization (ERM) is at the core of many machine learning systems. This means that the risk inherent in a loss distribution is summarized using a single number - its average. In this paper, we propose a general approach to c…
View article: Fairness and Randomness in Machine Learning: Statistical Independence and Relativization
Fairness and Randomness in Machine Learning: Statistical Independence and Relativization Open
Fair Machine Learning endeavors to prevent unfairness arising in the context of machine learning applications embedded in society. Despite the variety of definitions of fairness and proposed "fair algorithms", there remain unresolved conce…
View article: Information Processing Equalities and the Information-Risk Bridge
Information Processing Equalities and the Information-Risk Bridge Open
We introduce two new classes of measures of information for statistical experiments which generalise and subsume $ϕ$-divergences, integral probability metrics, $\mathfrak{N}$-distances (MMD), and $(f,Γ)$ divergences between two or more dis…
View article: Risk Measures and Upper Probabilities: Coherence and Stratification
Risk Measures and Upper Probabilities: Coherence and Stratification Open
Machine learning typically presupposes classical probability theory which implies that aggregation is built upon expectation. There are now multiple reasons to motivate looking at richer alternatives to classical probability theory as a ma…
View article: What killed the Convex Booster ?
What killed the Convex Booster ? Open
A landmark negative result of Long and Servedio established a worst-case spectacular failure of a supervised learning trio (loss, algorithm, model) otherwise praised for its high precision machinery. Hundreds of papers followed up on the t…
View article: Assessing AI Fairness in Finance
Assessing AI Fairness in Finance Open
If society demands that a bank's use of artificial intelligence systems is "fair," what is the bank to actually do? This article outlines a pragmatic and defensible answer.