Sean Meyn
YOU?
Author Swipe
View article: Functional role of synchronization: A mean-field control perspective
Functional role of synchronization: A mean-field control perspective Open
The broad goal of the research surveyed in this article is to develop methods for understanding the aggregate behavior of interconnected dynamical systems, as found in mathematical physics, neuroscience, economics, power systems and neural…
View article: Control Engineer Roles in the Next Power Market Transition
Control Engineer Roles in the Next Power Market Transition Open
This survey of power operations and power markets is a collaboration between members of academia and industry. It describes the thinking behind rules in organized electricity markets, which are rooted in the theory of efficient markets, an…
View article: Coherency-Constrained Spectral Clustering for Power Network Reduction
Coherency-Constrained Spectral Clustering for Power Network Reduction Open
This paper presents a methodology for reducing the complexity of large-scale power network models using spectral clustering, aggregation of electrical components, and cost function approximation. Two approaches are explored using unconstra…
View article: Proactive Frequency Stability Scheme Based on Bayesian Filters and Spectral Clustering
Proactive Frequency Stability Scheme Based on Bayesian Filters and Spectral Clustering Open
This work presents a proactive distributed model for power system frequency stability. High-level penetration of renewable energy sources into the grid have introduced unforeseen and unmodeled system dynamics. Underfrequency load shedding …
View article: Markovian Foundations for Quasi-Stochastic Approximation in Two Timescales: Extended Version
Markovian Foundations for Quasi-Stochastic Approximation in Two Timescales: Extended Version Open
Many machine learning and optimization algorithms can be cast as instances of stochastic approximation (SA). The convergence rate of these algorithms is known to be slow, with the optimal mean squared error (MSE) of order $O(n^{-1})$. In p…
View article: Quickest Change Detection Using Mismatched CUSUM
Quickest Change Detection Using Mismatched CUSUM Open
The field of quickest change detection (QCD) concerns design and analysis of algorithms to estimate in real time the time at which an important event takes place and identify properties of the post-change behavior. The goal is to devise a …
View article: Interacting Particle Systems for Fast Linear Quadratic RL
Interacting Particle Systems for Fast Linear Quadratic RL Open
This paper is concerned with the design of algorithms based on systems of interacting particles to represent, approximate, and learn the optimal control law for reinforcement learning (RL). The primary contribution is that convergence rate…
View article: Revisiting Step-Size Assumptions in Stochastic Approximation
Revisiting Step-Size Assumptions in Stochastic Approximation Open
Many machine learning and optimization algorithms are built upon the framework of stochastic approximation (SA), for which the selection of step-size (or learning rate) $\{α_n\}$ is crucial for success. An essential condition for convergen…
View article: Dual Ensemble Kalman Filter for Stochastic Optimal Control
Dual Ensemble Kalman Filter for Stochastic Optimal Control Open
In this paper, stochastic optimal control problems in continuous time and space are considered. In recent years, such problems have received renewed attention from the lens of reinforcement learning (RL) which is also one of our motivation…
View article: Reinforcement Learning Design for Quickest Change Detection
Reinforcement Learning Design for Quickest Change Detection Open
The field of quickest change detection (QCD) concerns design and analysis of algorithms to estimate in real time the time at which an important event takes place, and identify properties of the post-change behavior. It is shown in this pap…
View article: Learning Optimal Policies in Mean Field Models with Kullback-Leibler Regularization
Learning Optimal Policies in Mean Field Models with Kullback-Leibler Regularization Open
International audience
View article: Kullback–Leibler-Quadratic Optimal Control
Kullback–Leibler-Quadratic Optimal Control Open
This paper presents approaches to mean-field control, motivated by\ndistributed control of multi-agent systems. Control solutions are based on a\nconvex optimization problem, whose domain is a convex set of probability mass\nfunctions (pmf…
View article: Anomaly Detection in Power System State Estimation: Review and New Directions
Anomaly Detection in Power System State Estimation: Review and New Directions Open
Foundational and state-of-the-art anomaly-detection methods through power system state estimation are reviewed. Traditional components for bad data detection, such as chi-square testing, residual-based methods, and hypothesis testing, are …
View article: Convex Q Learning in a Stochastic Environment: Extended Version
Convex Q Learning in a Stochastic Environment: Extended Version Open
The paper introduces the first formulation of convex Q-learning for Markov decision processes with function approximation. The algorithms and theory rest on a relaxation of a dual of Manne's celebrated linear programming characterization o…
View article: The case for and against fixed step-size: Stochastic approximation algorithms in optimization and machine learning
The case for and against fixed step-size: Stochastic approximation algorithms in optimization and machine learning Open
Theory and application of stochastic approximation (SA) have become increasingly relevant due in part to applications in optimization and reinforcement learning. This paper takes a new look at SA with constant step-size $α>0$, defined by t…
View article: Anomaly Detection in Power System State Estimation: Review and New Directions
Anomaly Detection in Power System State Estimation: Review and New Directions Open
Foundational and state-of-the-art anomaly detection methods through power system state estimation are reviewed. The traditional components for bad data detection such as chi-square testing, residual-based methods, and hypothesis testing ar…
View article: Stability of Q-Learning Through Design and Optimism
Stability of Q-Learning Through Design and Optimism Open
Q-learning has become an important part of the reinforcement learning toolkit since its introduction in the dissertation of Chris Watkins in the 1980s. The purpose of this paper is in part a tutorial on stochastic approximation and Q-learn…
View article: High-Impedance Non-Linear Fault Detection via Eigenvalue Analysis with low PMU Sampling Rates
High-Impedance Non-Linear Fault Detection via Eigenvalue Analysis with low PMU Sampling Rates Open
This technique holds several advantages over contemporary techniques: It utilizes technology that is already deployed in the field, it offers a significant degree of generality, and so far it has displayed a very high-level of sensitivity …
View article: Uncertainty Error Modeling for Non-Linear State Estimation With Unsynchronized SCADA and $μ$PMU Measurements
Uncertainty Error Modeling for Non-Linear State Estimation With Unsynchronized SCADA and $μ$PMU Measurements Open
Distribution systems of the future smart grid require enhancements to the reliability of distribution system state estimation (DSSE) in the face of low measurement redundancy, unsynchronized measurements, and dynamic load profiles. Micro p…
View article: High Impedance Fault Detection Through Quasi-Static State Estimation: A Parameter Error Modeling Approach
High Impedance Fault Detection Through Quasi-Static State Estimation: A Parameter Error Modeling Approach Open
This paper presents a model for detecting high-impedance faults (HIFs) using parameter error modeling and a two-step per-phase weighted least squares state estimation (SE) process. The proposed scheme leverages the use of phasor measuremen…
View article: Sufficient Exploration for Convex Q-learning
Sufficient Exploration for Convex Q-learning Open
In recent years there has been a collective research effort to find new formulations of reinforcement learning that are simultaneously more efficient and more amenable to analysis. This paper concerns one approach that builds on the linear…
View article: Model-Free Characterizations of the Hamilton-Jacobi-Bellman Equation and Convex Q-Learning in Continuous Time
Model-Free Characterizations of the Hamilton-Jacobi-Bellman Equation and Convex Q-Learning in Continuous Time Open
Convex Q-learning is a recent approach to reinforcement learning, motivated by the possibility of a firmer theory for convergence, and the possibility of making use of greater a priori knowledge regarding policy or value function structure…
View article: Foreword
Foreword Open
who handled announcements, calls for papers, program preparation, Allerton House arrangements, registration, and the final editing of these Proceedings.The lion's share of the credit for the success of the Conference belongs to these indiv…
View article: Moment Constrained Optimal Transport for Control Applications
Moment Constrained Optimal Transport for Control Applications Open
This paper concerns the application of techniques from optimal transport (OT) to mean field control, in which the probability measures of interest in OT correspond to empirical distributions associated with a large collection of controlled…
View article: Research Trends and Applications of PMUs
Research Trends and Applications of PMUs Open
This work is a survey of current trends in applications of PMUs. PMUs have the potential to solve major problems in the areas of power system estimation, protection, and stability. A variety of methods are being used for these purposes, in…
View article: Duality for nonlinear filtering
Duality for nonlinear filtering Open
This thesis is concerned with the stochastic filtering problem for a hidden Markov model (HMM) with the white noise observation model. For this filtering problem, we make three types of original contributions: (1) dual controllability char…
View article: Markovian Foundations for Quasi-Stochastic Approximation with Applications to Extremum Seeking Control
Markovian Foundations for Quasi-Stochastic Approximation with Applications to Extremum Seeking Control Open
This paper concerns quasi-stochastic approximation (QSA) to solve root finding problems commonly found in applications to optimization and reinforcement learning. The general constant gain algorithm may be expressed as the time-inhomogeneo…
View article: Extremely Fast Convergence Rates for Extremum Seeking Control with Polyak-Ruppert Averaging
Extremely Fast Convergence Rates for Extremum Seeking Control with Polyak-Ruppert Averaging Open
Stochastic approximation is a foundation for many algorithms found in machine learning and optimization. It is in general slow to converge: the mean square error vanishes as $O(n^{-1})$. A deterministic counterpart known as quasi-stochasti…
View article: Control Systems and Reinforcement Learning
Control Systems and Reinforcement Learning Open
A high school student can create deep Q-learning code to control her robot, without any understanding of the meaning of 'deep' or 'Q', or why the code sometimes fails. This book is designed to explain the science behind reinforcement learn…
View article: The Conditional Poincaré Inequality for Filter Stability
The Conditional Poincaré Inequality for Filter Stability Open
This paper is concerned with the problem of nonlinear filter stability of ergodic Markov processes. The main contribution is the conditional Poincaré inequality (PI), which is shown to yield filter stability. The proof is based upon a rece…