Convex function ≈ Convex function
View article
Gradient Descent Provably Optimizes Over-parameterized Neural Networks Open
One of the mysteries in the success of neural networks is randomly initialized first order methods like gradient descent can achieve zero training loss even though the objective function is non-convex and non-smooth. This paper demystifies…
View article
Fair Resource Allocation in Federated Learning Open
Federated learning involves training statistical models in massive, heterogeneous networks. Naively minimizing an aggregate loss function in such a network may disproportionately advantage or disadvantage some of the devices. In this work,…
View article
Parallel and Distributed Methods for Constrained Nonconvex Optimization—Part I: Theory Open
In Part I of this paper, we proposed and analyzed a novel algorithmic\nframework for the minimization of a nonconvex (smooth) objective function,\nsubject to nonconvex constraints, based on inner convex approximations. This\nPart II is dev…
View article
Relatively Smooth Convex Optimization by First-Order Methods, and Applications Open
The usual approach to developing and analyzing first-order methods for smooth convex optimization assumes that the gradient of the objective function is uniformly smooth with some Lipschitz constant $L$. However, in many settings the diffe…
View article
SARAH: A Novel Method for Machine Learning Problems Using Stochastic\n Recursive Gradient Open
In this paper, we propose a StochAstic Recursive grAdient algoritHm (SARAH),\nas well as its practical variant SARAH+, as a novel approach to the finite-sum\nminimization problems. Different from the vanilla SGD and other modern\nstochasti…
View article
A Decentralized Proximal-Gradient Method With Network Independent Step-Sizes and Separated Convergence Rates Open
This paper proposes a novel proximal-gradient algorithm for a decentralized\noptimization problem with a composite objective containing smooth and\nnon-smooth terms. Specifically, the smooth and nonsmooth terms are dealt with\nby gradient …
View article
q -Hermite Hadamard inequalities and quantum estimates for midpoint type inequalities via convex and quasi-convex functions Open
In this paper, we prove the correct q-Hermite–Hadamard inequality, some new q-Hermite–Hadamard inequalities, and generalized q-Hermite–Hadamard inequality. By using the left hand part of the correct q-Hermite–Hadamard inequality, we have a…
View article
BAS-ADAM: an ADAM based approach to improve the performance of beetle antennae search optimizer Open
In this paper, we propose enhancements to Beetle Antennae search ( BAS ) algorithm, called BAS-ADAM, to smoothen the convergence behavior and avoid trapping in local-minima for a highly non-convex objective function. We achieve this by ada…
View article
Asynchronous Distributed ADMM for Large-Scale Optimization—Part I: Algorithm andConvergence Analysis Open
Aiming at solving large-scale learning problems, this paper studies\ndistributed optimization methods based on the alternating direction method of\nmultipliers (ADMM). By formulating the learning problem as a consensus problem,\nthe ADMM c…
View article
On the Properties of the Softmax Function with Application in Game Theory and Reinforcement Learning Open
In this paper, we utilize results from convex analysis and monotone operator theory to derive additional properties of the softmax function that have not yet been covered in the existing literature. In particular, we show that the softmax …
View article
Convergence Rate of Distributed ADMM over Networks Open
We propose a new distributed algorithm based on alternating direction method of multipliers (ADMM) to minimize sum of locally known convex functions using communication over a network. This optimization problem emerges in many applications…
View article
Dynamic Control of Agents Playing Aggregative Games With Coupling Constraints Open
We address the problem to control a population of noncooperative heterogeneous agents, each with convex cost function depending on the average population state, and all sharing a convex constraint, toward an aggregative equilibrium. We ass…
View article
LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed\n Learning Open
This paper presents a new class of gradient methods for distributed machine\nlearning that adaptively skip the gradient calculations to learn with reduced\ncommunication and computation. Simple rules are designed to detect\nslowly-varying …
View article
Optimal algorithms for smooth and strongly convex distributed optimization in networks Open
In this paper, we determine the optimal convergence rates for strongly convex and smooth distributed optimization in two settings: centralized and decentralized communications over a network. For centralized (i.e. master/slave) algorithms,…
View article
On the Global Convergence of Gradient Descent for Over-parameterized\n Models using Optimal Transport Open
Many tasks in machine learning and signal processing can be solved by\nminimizing a convex function of a measure. This includes sparse spikes\ndeconvolution or training a neural network with a single hidden layer. For\nthese problems, we s…
View article
Matrix Completion has No Spurious Local Minimum Open
Matrix completion is a basic machine learning problem that has wide applications, especially in collaborative filtering and recommender systems. Simple non-convex optimization algorithms are popular and effective in practice. Despite recen…
View article
Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communication Open
We consider decentralized stochastic optimization with the objective function (e.g. data samples for machine learning task) being distributed over $n$ machines that can only communicate to their neighbors on a fixed communication graph. To…
View article
THE SHARP BOUND FOR THE HANKEL DETERMINANT OF THE THIRD KIND FOR CONVEX FUNCTIONS Open
We prove the sharp inequality $|H_{3,1}(f)|\leq 4/135$ for convex functions, that is, for analytic functions $f$ with $a_{n}:=f^{(n)}(0)/n!,~n\in \mathbb{N}$ , such that $$\begin{eqnarray}Re\bigg\{1+\frac{zf^{\prime \prime }(z)}{f^{\prime …
View article
An Effective Optimization Method for Machine Learning Based on ADAM Open
A machine is taught by finding the minimum value of the cost function which is induced by learning data. Unfortunately, as the amount of learning increases, the non-liner activation function in the artificial neural network (ANN), the comp…
View article
On Analog Gradient Descent Learning Over Multiple Access Fading Channels Open
We consider a distributed learning problem over multiple access channel (MAC)\nusing a large wireless network. The computation is made by the network edge and\nis based on received data from a large number of distributed nodes which\ntrans…
View article
New Jensen and Hermite–Hadamard type inequalities for h-convex interval-valued functions Open
In this paper, we introduce the h-convex concept for interval-valued functions. By using the h-convex concept, we present new Jensen and Hermite–Hadamard type inequalities for interval-valued functions. Our inequalities generalize some kno…
View article
A Class of Prediction-Correction Methods for Time-Varying Convex Optimization Open
This paper considers unconstrained convex optimization problems with time-varying objective functions. We propose algorithms with a discrete time-sampling scheme to find and track the solution trajectory based on prediction and correction …
View article
Stochastic Successive Convex Approximation for Non-Convex Constrained Stochastic Optimization Open
This paper proposes a constrained stochastic successive convex approximation (CSSCA) algorithm to find a stationary point for a general non-convex stochastic optimization problem, whose objective and constraint functions are non-convex and…
View article
Coefficient estimates of new classes of q-starlike and q-convex functions of complex order Open
We introduce new classes of q -starlike and q -convex functions of complex order involving the q -derivative operator defined in the open unit disc.Furthermore, we find estimates on the coefficients for second and third coefficients of the…
View article
Some new inequalities of Hermite-Hadamard type for s-convex functions with applications Open
In this paper, we present several new and generalized Hermite-Hadamard type inequalities for s-convex as well as s-concave functions via classical and Riemann-Liouville fractional integrals. As applications, we provide new error estimation…
View article
Differentially Private Empirical Risk Minimization Revisited: Faster and\n More General Open
In this paper we study the differentially private Empirical Risk Minimization\n(ERM) problem in different settings. For smooth (strongly) convex loss function\nwith or without (non)-smooth regularization, we give algorithms that achieve\ne…
View article
Robust Federated Learning With Noisy Communication Open
Federated learning is a communication-efficient training process that alternate between local training at the edge devices and averaging of the updated local model at the center server. Nevertheless, it is impractical to achieve perfect ac…
View article
Third Hankel Determinants for Subclasses of Univalent Functions Open
The main aim of this paper is to discuss the third Hankel determinants for three classes: $$S^*$$ of starlike functions, $$\mathcal {K}$$ of convex functions and $$\mathcal {R}$$ of functions whose derivative has a positive real part. More…
View article
Decentralized Stochastic Optimization and Gossip Algorithms with\n Compressed Communication Open
We consider decentralized stochastic optimization with the objective function\n(e.g. data samples for machine learning task) being distributed over $n$\nmachines that can only communicate to their neighbors on a fixed communication\ngraph.…
View article
Variance Reduction for Faster Non-Convex Optimization Open
We consider the fundamental problem in non-convex optimization of efficiently reaching a stationary point. In contrast to the convex case, in the long history of this basic problem, the only known theoretical results on first-order non-con…