Broyden–Fletcher–Goldfarb–Shanno algorithm ≈ Broyden–Fletcher–Goldfarb–Shanno algorithm
View article
Faster Independent Component Analysis by Preconditioning With Hessian Approximations Open
Independent Component Analysis (ICA) is a technique for unsupervised exploration of multi-channel data that is widely used in observational sciences. In its classic form, ICA relies on modeling the data as linear mixtures of non-Gaussian i…
View article
Decentralized Quasi-Newton Methods Open
We introduce the decentralized Broyden-Fletcher-Goldfarb-Shanno (D-BFGS)\nmethod as a variation of the BFGS quasi-Newton method for solving decentralized\noptimization problems. The D-BFGS method is of interest in problems that are\nnot we…
View article
New Noise-Tolerant Neural Algorithms for Future Dynamic Nonlinear Optimization With Estimation on Hessian Matrix Inversion Open
Nonlinear optimization problems with dynamical parameters are widely arising in many practical scientific and engineering applications, and various computational models are presented for solving them under the hypothesis of short-time inva…
View article
Descent methods for elastic body simulation on the GPU Open
We show that many existing elastic body simulation approaches can be interpreted as descent methods, under a nonlinear optimization framework derived from implicit time integration. The key question is how to find an effective descent dire…
View article
Limited memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) method for the parameter estimation on geographically weighted ordinal logistic regression model (GWOLR) Open
In general, the parameter estimation of GWOLR model uses maximum likelihood method, but it constructs a system of nonlinear equations, making it difficult to find the solution. Therefore, an approximate solution is needed. There are two po…
View article
A BFGS-SQP method for nonsmooth, nonconvex, constrained optimization and its evaluation using relative minimization profiles Open
We propose an algorithm for solving nonsmooth, nonconvex, constrained optimization problems as well as a new set of visualization tools for comparing the performance of optimization algorithms. Our algorithm is a sequential quadratic optim…
View article
Seismic waveform inversion best practices: regional, global and exploration test cases Open
Reaching the global minimum of a waveform misfit function requires careful choices about the nonlinear optimization, preconditioning and regularization methods underlying an inversion. Because waveform inversion problems are susceptible to…
View article
Three-dimensional full waveform inversion of short-period teleseismic wavefields based upon the SEM–DSM hybrid method Open
International audience
View article
Comparative analysis of kernel-based versus ANN and deep learning methods in monthly reference evapotranspiration estimation Open
Timely and accurate estimation of reference evapotranspiration (ET0) is indispensable for agricultural water management for efficient water use. This study aims to estimate the amount of ET0 with machine learning approaches by using minimu…
View article
A Linearly-Convergent Stochastic L-BFGS Algorithm Open
We propose a new stochastic L-BFGS algorithm and prove a linear convergence rate for strongly convex and smooth functions. Our algorithm draws heavily from a recent stochastic variant of L-BFGS proposed in Byrd et al. (2014) as well as a r…
View article
Analysis and Prediction of COVID-19 Using SIR, SEIQR, and Machine Learning Models: Australia, Italy, and UK Cases Open
The novel coronavirus disease, also known as COVID-19, is a disease outbreak that was first identified in Wuhan, a Central Chinese city. In this report, a short analysis focusing on Australia, Italy, and UK is conducted. The analysis inclu…
View article
Quasi-Newton Methods for Real-Time Simulation of Hyperelastic Materials Open
We present a new method for real-time physics-based simulation supporting many different types of hyperelastic materials. Previous methods such as Position-Based or Projective Dynamics are fast but support only a limited selection of mater…
View article
Straggler Mitigation in Distributed Optimization Through Data Encoding Open
Slow running or straggler tasks can significantly reduce computation speed in distributed computation. Recently, coding-theory-inspired approaches have been applied to mitigate the effect of straggling, through embedding redundancy in cert…
View article
optimParallel: An R Package Providing a Parallel Version of the L-BFGS-B Optimization Method Open
The R package optimParallel provides a parallel version of the L-BFGS-B optimization method of optim(). The main function of the package is optimParallel(), which has the same usage and output as optim(). Using optimParallel() can signific…
View article
Randomized Quasi-Newton Updates Are Linearly Convergent Matrix Inversion Algorithms Open
We develop and analyze a broad family of stochastic/randomized algorithms for calculating an approximate inverse matrix. We also develop specialized variants maintaining symmetry or positive definiteness of the iterates. All methods in the…
View article
Neural Network Pattern Classification and Weather Dependent Fuzzy Logic Model for Irrigation Control in WSN Based Precision Agriculture Open
Watering system in agricultural lands plays a major activity in water and soil conservations. The future expectation of soil moisture content (MC) utilizing online soil and ecological parameters may give an effective stage to agriculture l…
View article
A Progressive Batching L-BFGS Method for Machine Learning Open
The standard L-BFGS method relies on gradient approximations that are not dominated by noise, so that search directions are descent directions, the line search is reliable, and quasi-Newton updating yields useful quadratic models of the ob…
View article
An Evaluation of ANN Algorithm Performance for MPPT Energy Harvesting in Solar PV Systems Open
In this paper, the Levenberg–Marquardt (LM), Bayesian regularization (BR), resilient backpropagation (RP), gradient descent momentum (GDM), Broyden–Fletcher–Goldfarb–Shanno (BFGS), and scaled conjugate gradient (SCG) algorithms constructed…
View article
On Quasi‐Newton methods in fast Fourier transform‐based micromechanics Open
SUMMARY This work is devoted to investigating the computational power of Quasi‐Newton methods in the context of fast Fourier transform (FFT)‐based computational micromechanics. We revisit FFT‐based Newton‐Krylov solvers as well as modern Q…
View article
Parameter Estimation of a Gaussian Mixture Model for Wind Power Forecast Error by Riemann L-BFGS Optimization Open
With the increasing penetration of wind power into the electricity grid, wind power forecast error analysis plays an important role in operations scheduling. To better describe the characteristics of power forecast error, a probability den…
View article
Seismic Full-Waveform Inversion Using Deep Learning Tools and Techniques Open
I demonstrate that the conventional seismic full-waveform inversion algorithm can be constructed as a recurrent neural network and so implemented using deep learning software such as TensorFlow. Applying another deep learning concept, the …
View article
Large-scale distributed L-BFGS Open
With the increasing demand for examining and extracting patterns from massive amounts of data, it is critical to be able to train large models to fulfill the needs that recent advances in the machine learning area create. L-BFGS (Limited-m…
View article
Improving the Classification Efficiency of an ANN Utilizing a New Training Methodology Open
In this work, a new approach for training artificial neural networks is presented which utilises techniques for solving the constraint optimisation problem. More specifically, this study converts the training of a neural network into a con…
View article
A robust multi-batch L-BFGS method for machine learning<sup>*</sup> Open
This paper describes an implementation of the L-BFGS method designed to deal with two adversarial situations. The first occurs in distributed computing environments where some of the computational nodes devoted to the evaluation of the fun…
View article
Performance Evaluation of Training Algorithms in Backpropagation Neural Network Approach to Blast-Induced Ground Vibration Prediction Open
Backpropagation Neural Network (BPNN) is an artificial intelligence technique that has seen several applications in many fields of science and engineering. It is well-known that, the critical task in developing an effective and accurate BP…
View article
Redundancy Techniques for Straggler Mitigation in Distributed Optimization and Learning Open
Performance of distributed optimization and learning systems is bottlenecked by "straggler" nodes and slow communication links, which significantly delay computation. We propose a distributed optimization framework where the dataset is "en…
View article
An Improved DDPG and Its Application Based on the Double-Layer BP Neural Network Open
This paper focused on three application problems of the traditional Deep Deterministic Policy Gradient(DDPG) algorithm. That is, the agent exploration is insufficient, the neural network performance is unsatisfied, the agent output fluctua…
View article
Practical Quasi-Newton Methods for Training Deep Neural Networks Open
We consider the development of practical stochastic quasi-Newton, and in particular Kronecker-factored block-diagonal BFGS and L-BFGS methods, for training deep neural networks (DNNs). In DNN training, the number of variables and component…
View article
Investigating the effect of textural properties on CO2 adsorption in porous carbons via deep neural networks using various training algorithms Open
The adsorption of carbon dioxide (CO 2 ) on porous carbon materials offers a promising avenue for cost-effective CO 2 emissions mitigation. This study investigates the impact of textural properties, particularly micropores, on CO 2 adsorpt…
View article
Solving forward and inverse problems of contact mechanics using physics-informed neural networks Open
This paper explores the ability of physics-informed neural networks (PINNs) to solve forward and inverse problems of contact mechanics for small deformation elasticity. We deploy PINNs in a mixed-variable formulation enhanced by output tra…