Thinh T. Doan
YOU?
Author Swipe
View article: Accelerating Multi-Task Temporal Difference Learning under Low-Rank Representation
Accelerating Multi-Task Temporal Difference Learning under Low-Rank Representation Open
We study policy evaluation problems in multi-task reinforcement learning (RL) under a low-rank representation setting. In this setting, we are given $N$ learning tasks where the corresponding value function of these tasks lie in an $r$-dim…
View article: Nonasymptotic CLT and Error Bounds for Two-Time-Scale Stochastic Approximation
Nonasymptotic CLT and Error Bounds for Two-Time-Scale Stochastic Approximation Open
We consider linear two-time-scale stochastic approximation algorithms driven by martingale noise. Recent applications in machine learning motivate the need to understand finite-time error rates, but conventional stochastic approximation an…
View article: Accelerated Multi-Time-Scale Stochastic Approximation: Optimal Complexity and Applications in Reinforcement Learning and Multi-Agent Games
Accelerated Multi-Time-Scale Stochastic Approximation: Optimal Complexity and Applications in Reinforcement Learning and Multi-Agent Games Open
Multi-time-scale stochastic approximation is an iterative algorithm for finding the fixed point of a set of $N$ coupled operators given their noisy samples. It has been observed that due to the coupling between the decision variables and n…
View article: Resilient Two-Time-Scale Local Stochastic Gradient Descent for Byzantine Federated Learning
Resilient Two-Time-Scale Local Stochastic Gradient Descent for Byzantine Federated Learning Open
We study local stochastic gradient descent methods for solving federated optimization over a network of agents communicating indirectly through a centralized coordinator. We are interested in the Byzantine setting where there is a subset o…
View article: Bayesian meta learning for trustworthy uncertainty quantification
Bayesian meta learning for trustworthy uncertainty quantification Open
We consider the problem of Bayesian regression with trustworthy uncertainty quantification. We define that the uncertainty quantification is trustworthy if the ground truth can be captured by intervals dependent on the predictive distribut…
View article: Fast Two-Time-Scale Stochastic Gradient Method with Applications in Reinforcement Learning
Fast Two-Time-Scale Stochastic Gradient Method with Applications in Reinforcement Learning Open
Two-time-scale optimization is a framework introduced in Zeng et al. (2024) that abstracts a range of policy evaluation and policy optimization problems in reinforcement learning (RL). Akin to bi-level optimization under a particular type …
View article: Natural Policy Gradient and Actor Critic Methods for Constrained Multi-Task Reinforcement Learning
Natural Policy Gradient and Actor Critic Methods for Constrained Multi-Task Reinforcement Learning Open
Multi-task reinforcement learning (RL) aims to find a single policy that effectively solves multiple tasks at the same time. This paper presents a constrained formulation for multi-task RL where the goal is to maximize the average performa…
View article: Fast Nonlinear Two-Time-Scale Stochastic Approximation: Achieving $O(1/k)$ Finite-Sample Complexity
Fast Nonlinear Two-Time-Scale Stochastic Approximation: Achieving $O(1/k)$ Finite-Sample Complexity Open
This paper proposes to develop a new variant of the two-time-scale stochastic approximation to find the roots of two coupled nonlinear operators, assuming only noisy samples of these operators can be observed. Our key idea is to leverage t…
View article: Resilient Federated Learning under Byzantine Attack in Distributed Nonconvex Optimization with 2-f Redundancy
Resilient Federated Learning under Byzantine Attack in Distributed Nonconvex Optimization with 2-f Redundancy Open
We study the problem of Byzantine fault tolerance in a distributed optimization setting, where there is a group of $N$ agents communicating with a trusted centralized coordinator. Among these agents, there is a subset of $f$ agents that ma…
View article: Connected Superlevel Set in (Deep) Reinforcement Learning and its Application to Minimax Theorems
Connected Superlevel Set in (Deep) Reinforcement Learning and its Application to Minimax Theorems Open
The aim of this paper is to improve the understanding of the optimization landscape for policy optimization problems in reinforcement learning. Specifically, we show that the superlevel set of the objective function with respect to the pol…
View article: Byzantine Fault-Tolerance in Federated Local SGD Under $2f$-Redundancy
Byzantine Fault-Tolerance in Federated Local SGD Under $2f$-Redundancy Open
In this article, we study the problem of Byzantine fault-tolerance in a federated optimization setting, where there is a group of agents communicating with a centralized coordinator. We allow up to $f$ Byzantine-faulty agents, which may no…
View article: Convergence and Price of Anarchy Guarantees of the Softmax Policy Gradient in Markov Potential Games
Convergence and Price of Anarchy Guarantees of the Softmax Policy Gradient in Markov Potential Games Open
We study the performance of policy gradient methods for the subclass of Markov games known as Markov potential games (MPGs), which extends the notion of normal-form potential games to the stateful setting and includes the important special…
View article: Regularized Gradient Descent Ascent for Two-Player Zero-Sum Markov Games
Regularized Gradient Descent Ascent for Two-Player Zero-Sum Markov Games Open
We study the problem of finding the Nash equilibrium in a two-player zero-sum Markov game. Due to its formulation as a minimax optimization program, a natural approach to solve the problem is to perform gradient descent/ascent with respect…
View article: Convergence Rates of Two-Time-Scale Gradient Descent-Ascent Dynamics for Solving Nonconvex Min-Max Problems
Convergence Rates of Two-Time-Scale Gradient Descent-Ascent Dynamics for Solving Nonconvex Min-Max Problems Open
There are much recent interests in solving noncovnex min-max optimization problems due to its broad applications in many areas including machine learning, networked resource allocations, and distributed optimization. Perhaps, the most popu…
View article: Improved Convergence Rate for a Distributed Two-Time-Scale Gradient Method under Random Quantization
Improved Convergence Rate for a Distributed Two-Time-Scale Gradient Method under Random Quantization Open
We study the so-called distributed two-time-scale gradient method for solving convex optimization problems over a network of agents when the communication bandwidth between the nodes is limited, and so information that is exchanged between…
View article: Finite-Time Analysis of Decentralized Stochastic Approximation with Applications in Multi-Agent and Multi-Task Learning
Finite-Time Analysis of Decentralized Stochastic Approximation with Applications in Multi-Agent and Multi-Task Learning Open
Stochastic approximation, a data-driven approach for finding the fixed point of an unknown operator, provides a unified framework for treating many problems in stochastic optimization and reinforcement learning. Motivated by a growing inte…
View article: Sequencing of multi-robot behaviors using reinforcement learning
Sequencing of multi-robot behaviors using reinforcement learning Open
Given a collection of parameterized multi-robot controllers associated with individual behaviors designed for particular tasks, this paper considers the problem of how to sequence and instantiate the behaviors for the purpose of completing…
View article: Finite-Time Complexity of Online Primal-Dual Natural Actor-Critic Algorithm for Constrained Markov Decision Processes
Finite-Time Complexity of Online Primal-Dual Natural Actor-Critic Algorithm for Constrained Markov Decision Processes Open
We consider a discounted cost constrained Markov decision process (CMDP) policy optimization problem, in which an agent seeks to maximize a discounted cumulative reward subject to a number of constraints on discounted cumulative utilities.…
View article: Convergence Rates of Decentralized Gradient Methods over Cluster\n Networks
Convergence Rates of Decentralized Gradient Methods over Cluster\n Networks Open
We present an analysis for the performance of decentralized consensus-based\ngradient (DCG) methods for solving optimization problems over a cluster network\nof nodes. This type of network is composed of a number of densely connected\nclus…
View article: Convergence Rates of Decentralized Gradient Methods over Cluster Networks
Convergence Rates of Decentralized Gradient Methods over Cluster Networks Open
We present an analysis for the performance of decentralized consensus-based gradient (DCG) methods for solving optimization problems over a cluster network of nodes. This type of network is composed of a number of densely connected cluster…
View article: A Two-Time-Scale Stochastic Optimization Framework with Applications in Control and Reinforcement Learning
A Two-Time-Scale Stochastic Optimization Framework with Applications in Control and Reinforcement Learning Open
We study a new two-time-scale stochastic gradient method for solving optimization problems, where the gradients are computed with the aid of an auxiliary variable under samples generated by time-varying MDPs controlled by the underlying op…
View article: Byzantine Fault-Tolerance in Federated Local SGD under 2f-Redundancy
Byzantine Fault-Tolerance in Federated Local SGD under 2f-Redundancy Open
We consider the problem of Byzantine fault-tolerance in federated machine learning. In this problem, the system comprises multiple agents each with local data, and a trusted centralized coordinator. In fault-free setting, the agents collab…
View article: Distributed Dual Subgradient Methods with Averaging and Applications to Grid Optimization
Distributed Dual Subgradient Methods with Averaging and Applications to Grid Optimization Open
We study finite-time performance of a recently proposed distributed dual subgradient (DDSG) method for convex constrained multi-agent optimization problems. The algorithm enjoys performance guarantees on the last primal iterate, as opposed…
View article: Distributed Grid Optimization via Distributed Dual Subgradient Methods with Averaging.
Distributed Grid Optimization via Distributed Dual Subgradient Methods with Averaging. Open
A collection of optimization problems central to power system operation requires distributed solution architectures to avoid the need for aggregation of all information at a central location. In this paper, we study distributed dual subgra…
View article: Improved Convergence Rate for a Distributed Two-Time-Scale Gradient\n Method under Random Quantization
Improved Convergence Rate for a Distributed Two-Time-Scale Gradient\n Method under Random Quantization Open
We study the so-called distributed two-time-scale gradient method for solving\nconvex optimization problems over a network of agents when the communication\nbandwidth between the nodes is limited, and so information that is exchanged\nbetw…
View article: Byzantine Fault-Tolerance in Decentralized Optimization under 2f-Redundancy
Byzantine Fault-Tolerance in Decentralized Optimization under 2f-Redundancy Open
This paper considers the problem of exact Byzantine fault-tolerance in multi-agent decentralized optimization. We consider a complete peer-to-peer network of n agents; each agent has a local cost function, however, up to f of the agents ar…
View article: Distributed two-time-scale methods over clustered networks
Distributed two-time-scale methods over clustered networks Open
In this paper, we consider consensus problems over a network of nodes, where\nthe network is divided into a number of clusters. We are interested in the case\nwhere the communication topology within each cluster is dense as compared to\nth…
View article: Convergence Rates of Distributed Consensus over Cluster Networks: A Two-Time-Scale Approach
Convergence Rates of Distributed Consensus over Cluster Networks: A Two-Time-Scale Approach Open
We study the popular distributed consensus method over networks composed of a number of densely connected clusters with a sparse connection between them. In these cluster networks, the method often constitutes two-time-scale dynamics, wher…
View article: Finite-Time Convergence Rates of Nonlinear Two-Time-Scale Stochastic Approximation under Markovian Noise
Finite-Time Convergence Rates of Nonlinear Two-Time-Scale Stochastic Approximation under Markovian Noise Open
We study the so-called two-time-scale stochastic approximation, a simulation-based approach for finding the roots of two coupled nonlinear operators. Our focus is to characterize its finite-time performance in a Markov setting, which often…
View article: Finite Sample Analysis of Two-Time-Scale Natural Actor-Critic Algorithm
Finite Sample Analysis of Two-Time-Scale Natural Actor-Critic Algorithm Open
Actor-critic style two-time-scale algorithms are one of the most popular methods in reinforcement learning, and have seen great empirical success. However, their performance is not completely understood theoretically. In this paper, we cha…