Sublinear function
View article
Training Deep Nets with Sublinear Memory Cost Open
We propose a systematic approach to reduce the memory consumption of deep neural network training. Specifically, we design an algorithm that costs O(sqrt(n)) memory to train a n layer network, with only the computational cost of an extra f…
View article
Hamiltonian Simulation with Nearly Optimal Dependence on all Parameters Open
We present an algorithm for sparse Hamiltonian simulation whose complexity is\noptimal (up to log factors) as a function of all parameters of interest.\nPrevious algorithms had optimal or near-optimal scaling in some parameters at\nthe cos…
View article
Personalized Federated Learning with Moreau Envelopes Open
Federated learning (FL) is a decentralized and privacy-preserving machine learning technique in which a group of clients collaborate with a server to learn a global model without sharing clients' data. One challenge associated with FL is s…
View article
Adafactor: Adaptive Learning Rates with Sublinear Memory Cost Open
In several recently proposed stochastic optimization methods (e.g. RMSProp, Adam, Adadelta), parameter updates are scaled by the inverse square roots of exponential moving averages of squared past gradients. Maintaining these per-parameter…
View article
Contribution of sublinear and supralinear dendritic integration to neuronal computations Open
Nonlinear dendritic integration is thought to increase the computational ability of neurons. Most studies focus on how supralinear summation of excitatory synaptic responses arising from clustered inputs within single dendrites result in t…
View article
On the Global Linear Convergence of Frank-Wolfe Optimization Variants Open
The Frank-Wolfe (FW) optimization algorithm has lately re-gained popularity thanks in particular to its ability to nicely handle the structured constraints appearing in machine learning applications. However, its convergence rate is known …
View article
Entanglement growth and correlation spreading with variable-range interactions in spin and fermionic tunneling models Open
We investigate the dynamics following a global parameter quench for two one-dimensional models with variable-range power-law interactions: a long-range transverse Ising model, which has recently been realized in chains of trapped ions, and…
View article
SpotLight Open
How do we spot interesting events from e-mail or transportation logs? How can we detect port scan or denial of service attacks from IP-IP communication data? In general, given a sequence of weighted, directed or bipartite graphs, each summ…
View article
I-LAMM for sparse learning: Simultaneous control of algorithmic complexity and statistical error Open
We propose a computational framework named iterative local adaptive majorize-minimization (I-LAMM) to simultaneously control algorithmic complexity and statistical error when fitting high dimensional models. I-LAMM is a two-stage algorithm…
View article
Approximate K-Means++ in Sublinear Time Open
The quality of K-Means clustering is extremely sensitive to proper initialization. The classic remedy is to apply k-means++ to obtain an initial set of centers that is provably competitive with the optimal solution. Unfortunately, k-means+…
View article
Infinitely many solutions for the stationary Kirchhoff problems involving the fractional<i>p</i>-Laplacian Open
The aim of this paper is to establish the multiplicity of weak solutions for a Kirchhoff-type problem driven by a fractional p-Laplacian operator with homogeneous Dirichlet boundary conditions.
View article
A note on the boundedness of sublinear operators on grand variable Herz spaces Open
In this paper, we introduce grand variable Herz type spaces using discrete grand spaces and prove the boundedness of sublinear operators on these spaces.
View article
Factorization Bandits for Interactive Recommendation Open
We perform online interactive recommendation via a factorization-based bandit algorithm. Low-rank matrix completion is performed over an incrementally constructed user-item preference matrix, where an upper confidence bound based item sele…
View article
Neural Policy Gradient Methods: Global Optimality and Rates of Convergence Open
Policy gradient methods with actor-critic schemes demonstrate tremendous empirical successes, especially when the actors and critics are parameterized by neural networks. However, it remains less clear whether such "neural" policy gradient…
View article
An Improved Convergence Analysis for Decentralized Online Stochastic Non-Convex Optimization Open
In this paper, we study decentralized online stochastic non-convex\noptimization over a network of nodes. Integrating a technique called gradient\ntracking in decentralized stochastic gradient descent, we show that the\nresulting algorithm…
View article
Solving the Diamond-Mortensen-Pissarides model accurately Open
An accurate global projection algorithm is critical for quantifying the basic moments of the Diamond-Mortensen-Pissarides model. Log linearization under- states the mean and volatility of unemployment, but overstates the volatility of labo…
View article
Convex Optimization: Algorithms and Complexity Open
This monograph presents the main complexity theorems in convex optimization and their corresponding algorithms. Starting from the fundamental theory of black-box optimization, the material progresses towards recent advances in structural o…
View article
QSGD: Randomized Quantization for Communication-Optimal Stochastic Gradient Descent Open
Parallel implementations of stochastic gradient descent (SGD) have received
significant research attention, thanks to excellent scalability properties of
this algorithm, and to its efficiency in the context of training deep neural
networks…
View article
Stochastic Recursive Gradient Algorithm for Nonconvex Optimization Open
In this paper, we study and analyze the mini-batch version of StochAstic Recursive grAdient algoritHm (SARAH), a method employing the stochastic recursive gradient, for solving empirical loss minimization for the case of nonconvex losses. …
View article
Law of large numbers and central limit theorem under nonlinear expectations Open
The main achievement of this paper is the finding and proof of Central Limit Theorem (CLT, see Theorem 12) under the framework of sublinear expectation. Roughly speaking under some reasonable assumption, the random sequence {1/n(X1+⋯+Xn)}i…
View article
Quantum chaos dynamics in long-range power law interaction systems Open
We use out-of-time-order commutator (OTOC) to diagnose the propagation of chaos in one dimensional long-range power law interaction system. We map the evolution of OTOC to a classical stochastic dynamics problem and use a Brownian quantum …
View article
Ligero++: A New Optimized Sublinear IOP Open
This paper follows the line of works that design concretely efficient transparent sublinear zero-knowledge Interactive Oracle Proofs (IOP). Arguments obtained via this paradigm have the advantages of not relying on public-key cryptography,…
View article
Optimal hashing-based time-space trade-offs for approximate near neighbors Open
We show tight upper and lower bounds for time-space trade-offs for the c-approximate Near Neighbor Search problem. For the d-dimensional Euclidean space and n-point datasets, we develop a data structure with space n1+ρu+o(1) + O(dn) and qu…
View article
On Thompson Sampling and Asymptotic Optimality Open
We discuss some recent results on Thompson sampling for nonparametric reinforcement learning in countable classes of general stochastic environments. These environments can be non-Markovian, non-ergodic, and partially observable. We show t…
View article
Nonlocal Schrödinger-Kirchhoff equations with external magnetic field Open
The paper deals with the existence and multiplicity of solutions of the fractional Schrödinger-Kirchhoff equation involving an external magnetic potential. As a consequence, the results can be applied to the special case $\begin{equation*}…
View article
Disorder, entropy and harmonic functions Open
We study harmonic functions on random environments with particular emphasis on the case of the infinite cluster of supercritical percolation on $\\mathbb{Z}^{d}$. We prove that the vector space of harmonic functions growing at most linearl…
View article
Estimating the Unseen Open
We show that a class of statistical properties of distributions, which includes such practically relevant properties as entropy, the number of distinct elements, and distance metrics between pairs of distributions, can be estimated given a…
View article
A Coordinate-Descent Primal-Dual Algorithm with Large Step Size and Possibly Nonseparable Functions Open
This paper introduces a coordinate descent version of the V\\~u-Condat\nalgorithm. By coordinate descent, we mean that only a subset of the coordinates\nof the primal and dual iterates is updated at each iteration, the other\ncoordinates b…
View article
Sketching and Sublinear Data Structures in Genomics Open
Large-scale genomics demands computational methods that scale sublinearly with the growth of data. We review several data structures and sketching techniques that have been used in genomic analysis methods. Specifically, we focus on four k…
View article
Model-Free Linear Quadratic Control via Reduction to Expert Prediction Open
Model-free approaches for reinforcement learning (RL) and continuous control find policies based only on past states and rewards, without fitting a model of the system dynamics. They are appealing as they are general purpose and easy to im…