Serdar Yüksel
YOU?
Author Swipe
View article: Sensitivity of Filter Kernels and Robustness Bounds to Transition and Measurement Kernel Perturbations in Partially Observable Stochastic Control
Sensitivity of Filter Kernels and Robustness Bounds to Transition and Measurement Kernel Perturbations in Partially Observable Stochastic Control Open
Studying the stability of partially observed Markov decision processes (POMDPs) with respect to perturbations in either transition or observation kernels is a significant problem. While asymptotic robustness/stability results as approximat…
View article: Kernel Mean Embedding Topology: Weak and Strong Forms for Stochastic Kernels and Implications for Model Learning
Kernel Mean Embedding Topology: Weak and Strong Forms for Stochastic Kernels and Implications for Model Learning Open
We introduce a novel topology, called Kernel Mean Embedding Topology, for stochastic kernels, in a weak and strong form. This topology, defined on the spaces of Bochner integrable functions from a signal space to a space of probability mea…
View article: Discrete-Time Approximations of Controlled Diffusions with Infinite Horizon Discounted and Average Cost
Discrete-Time Approximations of Controlled Diffusions with Infinite Horizon Discounted and Average Cost Open
We present discrete-time approximation of optimal control policies for infinite horizon discounted/ergodic control problems for controlled diffusions in $\Rd$\,. In particular, our objective is to show near optimality of optimal policies d…
View article: Unique ergodicity of non-linear filters via reachability and uniform weak continuity
Unique ergodicity of non-linear filters via reachability and uniform weak continuity Open
View article: Partially Observed Optimal Stochastic Control: Regularity, Optimality, Approximations, and Learning
Partially Observed Optimal Stochastic Control: Regularity, Optimality, Approximations, and Learning Open
In this review/tutorial article, we present recent progress on optimal control of partially observed Markov Decision Processes (POMDPs). We first present regularity and continuity conditions for POMDPs and their belief-MDP reductions, wher…
View article: Reinforcement Learning for Jointly Optimal Coding and Control Policies for a Controlled Markovian System over a Communication Channel
Reinforcement Learning for Jointly Optimal Coding and Control Policies for a Controlled Markovian System over a Communication Channel Open
We study the problem of joint optimization involving coding and control policies for a controlled Markovian sytem over a finite-rate noiseless communication channel. While structural results on the optimal encoding and control have been ob…
View article: Existence of $ε$-Nash Equilibria in Nonzero-Sum Borel Stochastic Games and Equilibria of Quantized Models
Existence of $ε$-Nash Equilibria in Nonzero-Sum Borel Stochastic Games and Equilibria of Quantized Models Open
Establishing the existence of exact or near Markov or stationary perfect Nash equilibria in nonzero-sum Markov games over Borel spaces remains a challenging problem, with few positive results to date. In this paper, we establish the existe…
View article: Robustness to Model Approximation, Model Learning From Data, and Sample Complexity in Wasserstein Regular MDPs
Robustness to Model Approximation, Model Learning From Data, and Sample Complexity in Wasserstein Regular MDPs Open
The paper studies the robustness properties of discrete-time stochastic optimal control under Wasserstein model approximation for both discounted cost and average cost criteria. Specifically, we study the performance loss when applying an …
View article: Probabilistic Satisfaction of Temporal Logic Constraints in Reinforcement Learning via Adaptive Policy-Switching
Probabilistic Satisfaction of Temporal Logic Constraints in Reinforcement Learning via Adaptive Policy-Switching Open
Constrained Reinforcement Learning (CRL) is a subset of machine learning that introduces constraints into the traditional reinforcement learning (RL) framework. Unlike conventional RL which aims solely to maximize cumulative rewards, CRL i…
View article: Near Optimal Approximations and Finite Memory Policies for POMPDs with Continuous Spaces
Near Optimal Approximations and Finite Memory Policies for POMPDs with Continuous Spaces Open
We study an approximation method for partially observed Markov decision processes (POMDPs) with continuous spaces. Belief MDP reduction, which has been the standard approach to study POMDPs requires rigorous approximation methods for pract…
View article: Robust decentralized control of coupled systems via risk sensitive control of decoupled or simple models with measure change
Robust decentralized control of coupled systems via risk sensitive control of decoupled or simple models with measure change Open
View article: Refined Bounds on Near Optimality Finite Window Policies in POMDPs and Their Reinforcement Learning
Refined Bounds on Near Optimality Finite Window Policies in POMDPs and Their Reinforcement Learning Open
Finding optimal policies for Partially Observable Markov Decision Processes (POMDPs) is challenging due to their uncountable state spaces when transformed into fully observable Markov Decision Processes (MDPs) using belief states. Traditio…
View article: Centralized Reduction of Decentralized Stochastic Control Models and their weak-Feller Regularity
Centralized Reduction of Decentralized Stochastic Control Models and their weak-Feller Regularity Open
Decentralized stochastic control problems involving general state/measurement/action spaces are intrinsically difficult to study because of the inapplicability of standard tools from centralized (single-agent) stochastic control. In this p…
View article: Robust Decentralized Control of Coupled Systems via Risk Sensitive Control of Decoupled or Simple Models with Measure Change
Robust Decentralized Control of Coupled Systems via Risk Sensitive Control of Decoupled or Simple Models with Measure Change Open
Decentralized stochastic control problems with local information involve problems where multiple agents and subsystems which are coupled via dynamics and/or cost are present. Typically, however, the dynamics of such couplings is complex an…
View article: Near Optimality of Lipschitz and Smooth Policies in Controlled Diffusions
Near Optimality of Lipschitz and Smooth Policies in Controlled Diffusions Open
For optimal control of diffusions under several criteria, due to computational or analytical reasons, many studies have a apriori assumed control policies to be Lipschitz or smooth, often with no rigorous analysis on whether this restricti…
View article: Decentralized Exchangeable Stochastic Dynamic Teams in Continuous-time, their Mean-Field Limits and Optimality of Symmetric Policies
Decentralized Exchangeable Stochastic Dynamic Teams in Continuous-time, their Mean-Field Limits and Optimality of Symmetric Policies Open
We study a class of stochastic exchangeable teams comprising a finite number of decision makers (DMs) as well as their mean-field limits involving infinite numbers of DMs. In the finite population regime, we study exchangeable teams under …
View article: Subjective Equilibria under Beliefs of Exogenous Uncertainty: Linear Quadratic Case
Subjective Equilibria under Beliefs of Exogenous Uncertainty: Linear Quadratic Case Open
We consider a stochastic dynamic game where players have their own linear state dynamics and quadratic cost functions. Players are coupled through some environment variables, generated by another linear system driven by the states and deci…
View article: Optimality of Symmetric Independent Policies under Decentralized Mean-Field Information Sharing for Stochastic Teams and Equivalence with McKean-Vlasov Control of a Representative Agent
Optimality of Symmetric Independent Policies under Decentralized Mean-Field Information Sharing for Stochastic Teams and Equivalence with McKean-Vlasov Control of a Representative Agent Open
We study a class of stochastic exchangeable teams with a finite number of decision makers (DMs) as well as their mean-field limits with infinitely many DMs. In the finite population regime, we study exchangeable teams under the centralized…
View article: Paths to Equilibrium in Games
Paths to Equilibrium in Games Open
In multi-agent reinforcement learning (MARL) and game theory, agents repeatedly interact and revise their strategies as new data arrives, producing a sequence of strategy profiles. This paper studies sequences of strategies satisfying a pa…
View article: Generalizing Better Response Paths and Weakly Acyclic Games
Generalizing Better Response Paths and Weakly Acyclic Games Open
Weakly acyclic games generalize potential games and are fundamental to the study of game theoretic control. In this paper, we present a generalization of weakly acyclic games, and we observe its importance in multi-agent learning when agen…
View article: Quantum Markov Decision Processes: General Theory, Approximations, and Classes of Policies
Quantum Markov Decision Processes: General Theory, Approximations, and Classes of Policies Open
In this paper, the aim is to develop a quantum counterpart to classical Markov decision processes (MDPs). Firstly, we provide a very general formulation of quantum MDPs with state and action spaces in the quantum domain, quantum transition…
View article: Quantum Markov Decision Processes: Dynamic and Semi-Definite Programs for Optimal Solutions
Quantum Markov Decision Processes: Dynamic and Semi-Definite Programs for Optimal Solutions Open
In this paper, building on the formulation of quantum Markov decision processes (q-MDPs) presented in our previous work [{\sc N.~Saldi, S.~Sanjari, and S.~Yüksel}, {\em Quantum Markov Decision Processes: General Theory, Approximations, and…
View article: Optimal Push and Pull-Based Edge Caching For Dynamic Content
Optimal Push and Pull-Based Edge Caching For Dynamic Content Open
We introduce a framework and optimal `fresh' caching for a content distribution network (CDN) comprising a front-end local cache and a back-end database. The data content is dynamically updated at a back-end database and end-users are inte…
View article: Continuity of cost in Borkar control topology and implications on discrete space and time approximations for controlled diffusions under several criteria
Continuity of cost in Borkar control topology and implications on discrete space and time approximations for controlled diffusions under several criteria Open
We first show that the discounted cost, cost up to an exit time, and ergodic cost involving controlled non-degenerate diffusions are continuous on the space of stationary control policies when the policies are given a topology introduced b…
View article: Average Cost Optimality of Partially Observed MDPS: Contraction of Non-linear Filters, Optimal Solutions and Approximations
Average Cost Optimality of Partially Observed MDPS: Contraction of Non-linear Filters, Optimal Solutions and Approximations Open
The average cost optimality is known to be a challenging problem for partially observable stochastic control, with few results available beyond the finite state, action, and measurement setup, for which somewhat restrictive conditions are …
View article: Reinforcement Learning for Near-Optimal Design of Zero-Delay Codes for Markov Sources
Reinforcement Learning for Near-Optimal Design of Zero-Delay Codes for Markov Sources Open
In the classical lossy source coding problem, one encodes long blocks of source symbols that enables the distortion to approach the ultimate Shannon limit. Such a block-coding approach introduces large delays, which is undesirable in many …
View article: Serdar Yüksel [People In Control]
Serdar Yüksel [People In Control] Open
Q. How did your education and early career lead to your initial and continuing interest in the control field?
View article: Controlled Diffusions under Full, Partial and Decentralized Information: Existence of Optimal Policies and Discrete-Time Approximations
Controlled Diffusions under Full, Partial and Decentralized Information: Existence of Optimal Policies and Discrete-Time Approximations Open
We present existence and discrete-time approximation results on optimal control policies for continuous-time stochastic control problems under a variety of information structures. These include fully observed models, partially observed mod…
View article: Q-Learning for Stochastic Control under General Information Structures and Non-Markovian Environments
Q-Learning for Stochastic Control under General Information Structures and Non-Markovian Environments Open
As a primary contribution, we present a convergence theorem for stochastic iterations, and in particular, Q-learning iterates, under a general, possibly non-Markovian, stochastic environment. Our conditions for convergence involve an ergod…
View article: Robustness of Optimal Controlled Diffusions with Near-Brownian Noise via Rough Paths Theory
Robustness of Optimal Controlled Diffusions with Near-Brownian Noise via Rough Paths Theory Open
In this article we show a robustness theorem for controlled stochastic differential equations driven by approximations of Brownian motion. Often, Brownian motion is used as an idealized model of a diffusion where approximations such as Won…