Guy Kornowski
YOU?
Author Swipe
Beyond Benign Overfitting in Nadaraya-Watson Interpolators Open
In recent years, there has been much interest in understanding the generalization behavior of interpolating predictors, which overfit on noisy training data. Whereas standard analyses are concerned with whether a method is consistent or no…
The Oracle Complexity of Simplex-based Matrix Games: Linear Separability and Nash Equilibria Open
We study the problem of solving matrix games of the form $\max_{\mathbf{w}\in\mathcal{W}}\min_{\mathbf{p}\inΔ}\mathbf{p}^{\top}A\mathbf{w}$, where $A$ is some matrix and $Δ$ is the probability simplex. This problem encapsulates canonical t…
Improved Sample Complexity for Private Nonsmooth Nonconvex Optimization Open
We study differentially private (DP) optimization algorithms for stochastic and empirical objectives which are neither smooth nor convex, and propose methods that return a Goldstein-stationary point with sample complexity bounds that impro…
Differentially Private Bilevel Optimization Open
We present differentially private (DP) algorithms for bilevel optimization, a problem class that received significant attention lately in various machine learning applications. These are the first algorithms for such problems under standar…
On the Hardness of Meaningful Local Guarantees in Nonsmooth Nonconvex Optimization Open
We study the oracle complexity of nonsmooth nonconvex optimization, with the algorithm assumed to have access only to local function information. It has been shown by Davis, Drusvyatskiy, and Jiang (2023) that for nonsmooth Lipschitz funct…
Open Problem: Anytime Convergence Rate of Gradient Descent Open
Recent results show that vanilla gradient descent can be accelerated for smooth convex objectives, merely by changing the stepsize sequence. We show that this can lead to surprisingly large errors indefinitely, and therefore ask: Is there …
First-Order Methods for Linearly Constrained Bilevel Optimization Open
Algorithms for bilevel optimization often encounter Hessian computations, which are prohibitive in high dimensions. While recent works offer first-order methods for unconstrained bilevel problems, the constrained setting remains relatively…
Efficient Agnostic Learning with Average Smoothness Open
We study distribution-free nonparametric regression following a notion of average smoothness initiated by Ashlagi et al. (2021), which measures the "effective" smoothness of a function with respect to an arbitrary unknown underlying distri…
An Algorithm with Optimal Dimension-Dependence for Zero-Order Nonsmooth Nonconvex Stochastic Optimization Open
We study the complexity of producing $(δ,ε)$-stationary points of Lipschitz objectives which are possibly neither smooth nor convex, using only noisy function evaluations. Recent works proposed several stochastic zero-order algorithms that…
From Tempered to Benign Overfitting in ReLU Neural Networks Open
Overparameterized neural networks (NNs) are observed to generalize well even when trained to perfectly fit noisy data. This phenomenon motivated a large body of work on "benign overfitting", where interpolating predictors achieve near-opti…
Deterministic Nonsmooth Nonconvex Optimization Open
We study the complexity of optimizing nonsmooth nonconvex Lipschitz functions by producing $(δ,ε)$-stationary points. Several recent works have presented randomized algorithms that produce such points using $\tilde O(δ^{-1}ε^{-3})$ first-o…
Near-optimal learning with average Hölder smoothness Open
We generalize the notion of average Lipschitz smoothness proposed by Ashlagi et al. (COLT 2021) by extending it to Hölder smoothness. This measure of the "effective smoothness" of a function is sensitive to the underlying distribution and …
On the Complexity of Finding Small Subgradients in Nonsmooth Optimization Open
We study the oracle complexity of producing $(δ,ε)$-stationary points of Lipschitz functions, in the sense proposed by Zhang et al. [2020]. While there exist dimension-free randomized algorithms for producing such points within $\widetilde…
Oracle Complexity in Nonsmooth Nonconvex Optimization Open
It is well-known that given a smooth, bounded-from-below, and possibly nonconvex function, standard gradient-based methods can find $ε$-stationary points (with gradient norm less than $ε$) in $\mathcal{O}(1/ε^2)$ iterations. However, many …
High-Order Oracle Complexity of Smooth and Strongly Convex Optimization Open
In this note, we consider the complexity of optimizing a highly smooth (Lipschitz $k$-th order derivative) and strongly convex function, via calls to a $k$-th order oracle which returns the value and first $k$ derivatives of the function a…