Frederic Koehler
YOU?
Author Swipe
View article: Efficiently learning and sampling multimodal distributions with data-based initialization
Efficiently learning and sampling multimodal distributions with data-based initialization Open
We consider the problem of sampling a multimodal distribution with a Markov chain given a small number of samples from the stationary measure. Although mixing can be arbitrarily slow, we show that if the Markov chain has a $k$th order spec…
View article: Trickle-Down in Localization Schemes and Applications
Trickle-Down in Localization Schemes and Applications Open
Trickle-down is a phenomenon in high-dimensional expanders with many important applications — for example, it is a key ingredient in various constructions of high-dimensional expanders or the proof of rapid mixing for the basis exchange wa…
View article: Influences in Mixing Measures
Influences in Mixing Measures Open
The theory of influences in product measures has profound applications in theoretical computer science, combinatorics, and discrete probability. This deep theory is intimately connected to functional inequalities and to the Fourier analysi…
View article: Inferring Dynamic Networks from Marginals with Iterative Proportional Fitting
Inferring Dynamic Networks from Marginals with Iterative Proportional Fitting Open
A common network inference problem, arising from real-world data constraints, is how to infer a dynamic network from its time-aggregated adjacency matrix and time-varying marginals (i.e., row and column sums). Prior approaches to this prob…
View article: Lasso with Latents: Efficient Estimation, Covariate Rescaling, and Computational-Statistical Gaps
Lasso with Latents: Efficient Estimation, Covariate Rescaling, and Computational-Statistical Gaps Open
It is well-known that the statistical performance of Lasso can suffer significantly when the covariates of interest have strong correlations. In particular, the prediction error of Lasso becomes much worse than computationally inefficient …
View article: Optimistic Rates: A Unifying Theory for Interpolation Learning and Regularization in Linear Regression
Optimistic Rates: A Unifying Theory for Interpolation Learning and Regularization in Linear Regression Open
We study a localized notion of uniform convergence known as an “optimistic rate” [ 34 , 39 ] for linear regression with Gaussian data. Our refined analysis avoids the hidden constant and logarithmic factor in existing results, which are kn…
View article: Sampling Multimodal Distributions with the Vanilla Score: Benefits of Data-Based Initialization
Sampling Multimodal Distributions with the Vanilla Score: Benefits of Data-Based Initialization Open
There is a long history, as well as a recent explosion of interest, in statistical and generative modeling approaches based on score functions -- derivatives of the log-likelihood of a distribution. In seminal works, Hyvärinen proposed van…
View article: Universality of Spectral Independence with Applications to Fast Mixing in Spin Glasses
Universality of Spectral Independence with Applications to Fast Mixing in Spin Glasses Open
We study Glauber dynamics for sampling from discrete distributions $μ$ on the hypercube $\{\pm 1\}^n$. Recently, techniques based on spectral independence have successfully yielded optimal $O(n)$ relaxation times for a host of different di…
View article: Influences in Mixing Measures
Influences in Mixing Measures Open
The theory of influences in product measures has profound applications in theoretical computer science, combinatorics, and discrete probability. This deep theory is intimately connected to functional inequalities and to the Fourier analysi…
View article: Uniform Convergence with Square-Root Lipschitz Loss
Uniform Convergence with Square-Root Lipschitz Loss Open
We establish generic uniform convergence guarantees for Gaussian data in terms of the Rademacher complexity of the hypothesis class and the Lipschitz constant of the square root of the scalar loss function. We show how these guarantees sub…
View article: Feature Adaptation for Sparse Linear Regression
Feature Adaptation for Sparse Linear Regression Open
Sparse linear regression is a central problem in high-dimensional statistics. We study the correlated random design setting, where the covariates are drawn from a multivariate Gaussian $N(0,Σ)$, and we seek an estimator with small excess r…
View article: A Non-Asymptotic Moreau Envelope Theory for High-Dimensional Generalized Linear Models
A Non-Asymptotic Moreau Envelope Theory for High-Dimensional Generalized Linear Models Open
We prove a new generalization bound that shows for any class of linear predictors in Gaussian space, the Rademacher complexity of the class and the training error under any continuous loss $\ell$ can control the test error under all Moreau…
View article: Statistical Efficiency of Score Matching: The View from Isoperimetry
Statistical Efficiency of Score Matching: The View from Isoperimetry Open
Deep generative models parametrized up to a normalizing constant (e.g. energy-based models) are difficult to train by maximizing the likelihood of the data because the likelihood and/or gradients thereof cannot be explicitly or efficiently…
View article: Kalman filtering with adversarial corruptions
Kalman filtering with adversarial corruptions Open
Here we revisit the classic problem of linear quadratic estimation, i.e. estimating the trajectory of a linear dynamical system from noisy measurements. The celebrated Kalman filter gives an optimal estimator when the measurement noise is …
View article: Distributional Hardness Against Preconditioned Lasso via Erasure-Robust Designs
Distributional Hardness Against Preconditioned Lasso via Erasure-Robust Designs Open
Sparse linear regression with ill-conditioned Gaussian random designs is widely believed to exhibit a statistical/computational gap, but there is surprisingly little formal evidence for this belief, even in the form of examples that are ha…
View article: Sampling Approximately Low-Rank Ising Models: MCMC meets Variational Methods
Sampling Approximately Low-Rank Ising Models: MCMC meets Variational Methods Open
We consider Ising models on the hypercube with a general interaction matrix $J$, and give a polynomial time sampling algorithm when all but $O(1)$ eigenvalues of $J$ lie in an interval of length one, a situation which occurs in many models…
View article: Double Balanced Sets in High Dimensional Expanders
Double Balanced Sets in High Dimensional Expanders Open
Recent works have shown that expansion of pseudorandom sets is of great importance. However, all current works on pseudorandom sets are limited only to product (or approximate product) spaces, where Fourier Analysis methods could be applie…
View article: Variational autoencoders in the presence of low-dimensional data: landscape and implicit bias
Variational autoencoders in the presence of low-dimensional data: landscape and implicit bias Open
Variational Autoencoders are one of the most commonly used generative models, particularly for image data. A prominent difficulty in training VAEs is data that is supported on a lower-dimensional manifold. Recent work by Dai and Wipf (2020…
View article: Optimistic Rates: A Unifying Theory for Interpolation Learning and Regularization in Linear Regression
Optimistic Rates: A Unifying Theory for Interpolation Learning and Regularization in Linear Regression Open
We study a localized notion of uniform convergence known as an "optimistic rate" (Panchenko 2002; Srebro et al. 2010) for linear regression with Gaussian data. Our refined analysis avoids the hidden constant and logarithmic factor in exist…
View article: Kalman Filtering with Adversarial Corruptions
Kalman Filtering with Adversarial Corruptions Open
Here we revisit the classic problem of linear quadratic estimation, i.e. estimating the trajectory of a linear dynamical system from noisy measurements. The celebrated Kalman filter gives an optimal estimator when the measurement noise is …
View article: Entropic Independence II: Optimal Sampling and Concentration via Restricted Modified Log-Sobolev Inequalities
Entropic Independence II: Optimal Sampling and Concentration via Restricted Modified Log-Sobolev Inequalities Open
We introduce a framework for obtaining tight mixing times for Markov chains based on what we call restricted modified log-Sobolev inequalities. Modified log-Sobolev inequalities (MLSI) quantify the rate of relative entropy contraction for …
View article: Multidimensional Scaling: Approximation and Complexity
Multidimensional Scaling: Approximation and Complexity Open
Metric Multidimensional scaling (MDS) is a classical method for generating meaningful (non-linear) low-dimensional embeddings of high-dimensional data. MDS has a long history in the statistics, machine learning, and graph drawing communiti…
View article: Reconstruction on Trees and Low-Degree Polynomials
Reconstruction on Trees and Low-Degree Polynomials Open
The study of Markov processes and broadcasting on trees has deep connections to a variety of areas including statistical physics, graphical models, phylogenetic reconstruction, Markov Chain Monte Carlo, and community detection in random gr…
View article: On the Power of Preconditioning in Sparse Linear Regression
On the Power of Preconditioning in Sparse Linear Regression Open
Sparse linear regression is a fundamental problem in high-dimensional statistics, but strikingly little is known about how to efficiently solve it without restrictive conditions on the design matrix. We consider the (correlated) random des…
View article: Uniform Convergence of Interpolators: Gaussian Width, Norm Bounds, and Benign Overfitting
Uniform Convergence of Interpolators: Gaussian Width, Norm Bounds, and Benign Overfitting Open
We consider interpolation learning in high-dimensional linear regression with Gaussian data, and prove a generic uniform convergence guarantee on the generalization error of interpolators in an arbitrary hypothesis class in terms of the cl…
View article: Entropic Independence I: Modified Log-Sobolev Inequalities for Fractionally Log-Concave Distributions and High-Temperature Ising Models
Entropic Independence I: Modified Log-Sobolev Inequalities for Fractionally Log-Concave Distributions and High-Temperature Ising Models Open
We introduce a notion called entropic independence that is an entropic analog of spectral notions of high-dimensional expansion. Informally, entropic independence of a background distribution $μ$ on $k$-sized subsets of a ground set of ele…
View article: Entropic Independence in High-Dimensional Expanders: Modified Log-Sobolev Inequalities for Fractionally Log-Concave Polynomials and the Ising Model.
Entropic Independence in High-Dimensional Expanders: Modified Log-Sobolev Inequalities for Fractionally Log-Concave Polynomials and the Ising Model. Open
We introduce a notion called entropic independence for distributions $\mu$ defined on pure simplicial complexes, i.e., subsets of size $k$ of a ground set of elements. Informally, we call a background measure $\mu$ entropically independent…
View article: Chow-Liu++: Optimal Prediction-Centric Learning of Tree Ising Models
Chow-Liu++: Optimal Prediction-Centric Learning of Tree Ising Models Open
We consider the problem of learning a tree-structured Ising model from data, such that subsequent predictions computed using the model are accurate. Concretely, we aim to learn a model such that posteriors $P(X_i|X_S)$ for small sets of va…
View article: Online and Distribution-Free Robustness: Regression and Contextual Bandits with Huber Contamination
Online and Distribution-Free Robustness: Regression and Contextual Bandits with Huber Contamination Open
In this work we revisit two classic high-dimensional online learning problems, namely linear regression and contextual bandits, from the perspective of adversarial robustness. Existing works in algorithmic robust statistics make strong dis…