Maxima and minima
View article
GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium Open
Generative Adversarial Networks (GANs) excel at creating realistic images with complex models for which maximum likelihood is infeasible. However, the convergence of GAN training has still not been proved. We propose a two time-scale updat…
View article
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima Open
The stochastic gradient descent (SGD) method and its variants are algorithms of choice for many Deep Learning tasks. These methods operate in a small-batch regime wherein a fraction of the training data, say $32$-$512$ data points, is samp…
View article
A mean field view of the landscape of two-layer neural networks Open
Significance Multilayer neural networks have proven extremely successful in a variety of tasks, from image classification to robotics. However, the reasons for this practical success and its precise domain of applicability are unknown. Lea…
View article
Quantum optimization of maximum independent set using Rydberg atom arrays Open
Realizing quantum speedup for practically relevant, computationally hard problems is a central challenge in quantum information science. Using Rydberg atom arrays with up to 289 qubits in two spatial dimensions, we experimentally investiga…
View article
GANs Trained by a Two Time-Scale Update Rule Converge to a Nash Equilibrium Open
Generative Adversarial Networks (GANs) excel at creating realistic images with complex models for which maximum likelihood is infeasible. However, the convergence of GAN training has still not been proved. We propose a two time-scale updat…
View article
Training Variational Quantum Algorithms Is NP-Hard Open
Variational quantum algorithms are proposed to solve relevant computational problems on near term quantum devices. Popular versions are variational quantum eigensolvers and quantum approximate optimization algorithms that solve ground stat…
View article
Dynamic Coattention Networks For Question Answering Open
Several deep learning models have been proposed for question answering. However, due to their single-pass nature, they have no way to recover from local maxima corresponding to incorrect answers. To address this problem, we introduce the D…
View article
Multivariate <span>C</span>opula <span>A</span>nalysis <span>T</span>oolbox (MvCAT): Describing dependence and underlying uncertainty using a <span>B</span>ayesian framework Open
We present a newly developed Multivariate Copula Analysis Toolbox (MvCAT) which includes a wide range of copula families with different levels of complexity. MvCAT employs a Bayesian framework with a residual‐based Gaussian likelihood func…
View article
Dynamic Coattention Networks For Question Answering Open
Several deep learning models have been proposed for question answering. However, due to their single-pass nature, they have no way to recover from local maxima corresponding to incorrect answers. To address this problem, we introduce the D…
View article
Generative Deep Neural Networks for Inverse Materials Design Using Backpropagation and Active Learning Open
In recent years, machine learning (ML) techniques are seen to be promising tools to discover and design novel materials. However, the lack of robust inverse design approaches to identify promising candidate materials without exploring the …
View article
Optimal Tuning of Fractional Order PID Controller for DC Motor Speed Control via Chaotic Atom Search Optimization Algorithm Open
In this paper, atom search optimization (ASO) algorithm and a novel chaotic version of it [chaotic ASO (ChASO)] are proposed to determine the optimal parameters of the fractional-order proportional+integral+derivative (FOPID) controller fo…
View article
An Efficient Sampling-Based Method for Online Informative Path Planning in Unknown Environments Open
The ability to plan informative paths online is essential to robot autonomy. In particular, sampling-based approaches are often used as they are capable of using arbitrary information gain formulations. However, they are prone to local min…
View article
Non-autonomous functionals, borderline cases and related function classes Open
The class of non-autonomous functionals under study is characterized by the fact that the energy density changes its ellipticity and growth properties according to the point; some regularity results are proved for related minimizers. These…
View article
Building a More Predictive Protein Force Field: A Systematic and Reproducible Route to AMBER-FB15 Open
The increasing availability of high-quality experimental data and first-principles calculations creates opportunities for developing more accurate empirical force fields for simulation of proteins. We developed the AMBER-FB15 protein force…
View article
Evaluation of Denoising Strategies to Address Motion-Correlated Artifacts in Resting-State Functional Magnetic Resonance Imaging Data from the Human Connectome Project Open
Like all resting-state functional connectivity data, the data from the Human Connectome Project (HCP) are adversely affected by structured noise artifacts arising from head motion and physiological processes. Functional connectivity estima…
View article
Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data Open
One of the defining properties of deep learning is that models are chosen to have many more parameters than available training data. In light of this capacity for overfitting, it is remarkable that simple algorithms like SGD reliably retur…
View article
Three Factors Influencing Minima in SGD Open
We investigate the dynamical and convergent properties of stochastic gradient descent (SGD) applied to Deep Neural Networks (DNNs). Characterizing the relation between learning rate, batch size and the properties of the final minima, such …
View article
How to Escape Saddle Points Efficiently Open
This paper shows that a perturbed form of gradient descent converges to a second-order stationary point in a number iterations which depends only poly-logarithmically on dimension (i.e., it is almost "dimension-free"). The convergence rate…
View article
BAS-ADAM: an ADAM based approach to improve the performance of beetle antennae search optimizer Open
In this paper, we propose enhancements to Beetle Antennae search ( BAS ) algorithm, called BAS-ADAM, to smoothen the convergence behavior and avoid trapping in local-minima for a highly non-convex objective function. We achieve this by ada…
View article
No Spurious Local Minima in Nonconvex Low Rank Problems: A Unified Geometric Analysis Open
In this paper we develop a new framework that captures the common landscape underlying the common non-convex low-rank matrix problems including matrix sensing, matrix completion and robust PCA. In particular, we show for all above problems…
View article
Improved Artificial Potential Field Method Applied for AUV Path Planning Open
With the topics related to the intelligent AUV, control and navigation have become one of the key researching fields. This paper presents a concise and reliable path planning method for AUV based on the improved APF method. AUV can make th…
View article
Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks Open
We study the problem of training deep neural networks with Rectified Linear Unit (ReLU) activation function using gradient descent and stochastic gradient descent. In particular, we study the binary classification problem and show that for…
View article
Gradient Descent Finds Global Minima of Deep Neural Networks Open
Gradient descent finds a global minimum in training deep neural networks despite the objective function being non-convex. The current paper proves gradient descent achieves zero training loss in polynomial time for a deep over-parameterize…
View article
Path planning algorithms in the autonomous driving system: A comprehensive review Open
This comprehensive review focuses on the Autonomous Driving System (ADS), which aims to reduce human errors that are the reason for about 95% of car accidents. The ADS consists of six stages: sensors, perception, localization, assessment, …
View article
Matrix Completion has No Spurious Local Minimum Open
Matrix completion is a basic machine learning problem that has wide applications, especially in collaborative filtering and recommender systems. Simple non-convex optimization algorithms are popular and effective in practice. Despite recen…
View article
Region-of-interest analyses of one-dimensional biomechanical trajectories: bridging 0D and 1D theory, augmenting statistical power Open
One-dimensional (1D) kinematic, force, and EMG trajectories are often analyzed using zero-dimensional (0D) metrics like local extrema. Recently whole-trajectory 1D methods have emerged in the literature as alternatives. Since 0D and 1D met…
View article
Valley-locked waveguide transport in acoustic heterostructures Open
Valley pseudospin, labeling the pair of energy extrema in momentum space, has been attracting attention because of its potential as a new degree of freedom in manipulating electrons or classical waves. Recently, topological valley edge tra…
View article
Modified artificial potential field method for online path planning applications Open
IEEE Intelligent Vehicle symposium 2017, Redondo Beach, ETATS-UNIS, 11-/06/2017 - 14/06/2017
View article
Random vector functional link network: Recent developments, applications, and future directions Open
Neural networks have been successfully employed in various domains such as classification, regression and clustering, etc. Generally, the back propagation (BP) based iterative approaches are used to train the neural networks, however, it r…
View article
Primordial black holes from polynomial potentials in single field inflation Open
Within canonical single field inflation models, we provide a method to\nreverse engineer and reconstruct the inflaton potential from a given power\nspectrum. This is not only a useful tool to find a potential from observational\nconstraint…