Dustin Morrill
YOU?
Author Swipe
View article: Composing Efficient, Robust Tests for Policy Selection
Composing Efficient, Robust Tests for Policy Selection Open
Modern reinforcement learning systems produce many high-quality policies throughout the learning process. However, to choose which policy to actually deploy in the real world, they must be tested under an intractable number of environmenta…
View article: Interpolating Between Softmax Policy Gradient and Neural Replicator Dynamics with Capped Implicit Exploration
Interpolating Between Softmax Policy Gradient and Neural Replicator Dynamics with Capped Implicit Exploration Open
Neural replicator dynamics (NeuRD) is an alternative to the foundational softmax policy gradient (SPG) algorithm motivated by online learning and evolutionary game theory. The NeuRD expected update is designed to be nearly identical to tha…
View article: Efficient Deviation Types and Learning for Hindsight Rationality in Extensive-Form Games: Corrections
Efficient Deviation Types and Learning for Hindsight Rationality in Extensive-Form Games: Corrections Open
Hindsight rationality is an approach to playing general-sum games that prescribes no-regret learning dynamics for individual agents with respect to a set of deviations, and further describes jointly rational behavior among multiple agents …
View article: The Partially Observable History Process
The Partially Observable History Process Open
We introduce the partially observable history process (POHP) formalism for reinforcement learning. POHP centers around the actions and observations of a single agent and abstracts away the presence of other players without reducing them to…
View article: Learning to Be Cautious
Learning to Be Cautious Open
A key challenge in the field of reinforcement learning is to develop agents that behave cautiously in novel situations. It is generally impossible to anticipate all situations that an autonomous system may face or what behavior would best …
View article: Hindsight and Sequential Rationality of Correlated Play
Hindsight and Sequential Rationality of Correlated Play Open
Driven by recent successes in two-player, zero-sum game solving and playing, artificial intelligence work on games has increasingly focused on algorithms that produce equilibrium-based strategies. However, this approach has been less effec…
View article: Efficient Deviation Types and Learning for Hindsight Rationality in Extensive-Form Games
Efficient Deviation Types and Learning for Hindsight Rationality in Extensive-Form Games Open
Hindsight rationality is an approach to playing general-sum games that prescribes no-regret learning dynamics for individual agents with respect to a set of deviations, and further describes jointly rational behavior among multiple agents …
View article: Efficient Deviation Types and Learning for Hindsight Rationality in\n Extensive-Form Games
Efficient Deviation Types and Learning for Hindsight Rationality in\n Extensive-Form Games Open
Hindsight rationality is an approach to playing general-sum games that\nprescribes no-regret learning dynamics for individual agents with respect to a\nset of deviations, and further describes jointly rational behavior among\nmultiple agen…
View article: Hindsight and Sequential Rationality of Correlated Play
Hindsight and Sequential Rationality of Correlated Play Open
Driven by recent successes in two-player, zero-sum game solving and playing, artificial intelligence work on games has increasingly focused on algorithms that produce equilibrium-based strategies. However, this approach has been less effec…
View article: The Advantage Regret-Matching Actor-Critic
The Advantage Regret-Matching Actor-Critic Open
Regret minimization has played a key role in online learning, equilibrium computation in games, and reinforcement learning (RL). In this paper, we describe a general model-free RL method for no-regret learning based on repeated reconsidera…
View article: Alternative Function Approximation Parameterizations for Solving Games: An Analysis of $f$-Regression Counterfactual Regret Minimization
Alternative Function Approximation Parameterizations for Solving Games: An Analysis of $f$-Regression Counterfactual Regret Minimization Open
Function approximation is a powerful approach for structuring large decision problems that has facilitated great achievements in the areas of reinforcement learning and game playing. Regression counterfactual regret minimization (RCFR) is …
View article: Bounds for Approximate Regret-Matching Algorithms
Bounds for Approximate Regret-Matching Algorithms Open
A dominant approach to solving large imperfect-information games is Counterfactural Regret Minimization (CFR). In CFR, many regret minimization problems are combined to solve the game. For very large games, abstraction is typically needed …
View article: OpenSpiel: A Framework for Reinforcement Learning in Games
OpenSpiel: A Framework for Reinforcement Learning in Games Open
OpenSpiel is a collection of environments and algorithms for research in general reinforcement learning and search/planning in games. OpenSpiel supports n-player (single- and multi- agent) zero-sum, cooperative and general-sum, one-shot an…
View article: Computing Approximate Equilibria in Sequential Adversarial Games by Exploitability Descent
Computing Approximate Equilibria in Sequential Adversarial Games by Exploitability Descent Open
In this paper, we present exploitability descent, a new algorithm to compute approximate equilibria in two-player zero-sum extensive-form games with imperfect information, by direct policy optimization against worst-case opponents. We prov…
View article: Neural Replicator Dynamics
Neural Replicator Dynamics Open
Policy gradient and actor-critic algorithms form the basis of many commonly used training techniques in deep reinforcement learning. Using these algorithms in multiagent environments poses problems such as nonstationarity and instability. …
View article: Computing Approximate Equilibria in Sequential Adversarial Games by\n Exploitability Descent
Computing Approximate Equilibria in Sequential Adversarial Games by\n Exploitability Descent Open
In this paper, we present exploitability descent, a new algorithm to compute\napproximate equilibria in two-player zero-sum extensive-form games with\nimperfect information, by direct policy optimization against worst-case\nopponents. We p…
View article: DeepStack: Expert-level artificial intelligence in heads-up no-limit poker
DeepStack: Expert-level artificial intelligence in heads-up no-limit poker Open
Computer code based on continual problem re-solving beats human professional poker players at a two-player variant of poker.
View article: Using Regret Estimation to Solve Games Compactly
Using Regret Estimation to Solve Games Compactly Open
Game theoretic solution concepts, such as Nash equilibrium strategies that are optimal against worst case opponents, provide guidance in finding desirable autonomous agent behaviour. In particular, we wish to approximate solutions to compl…
View article: Solving Games with Functional Regret Estimation
Solving Games with Functional Regret Estimation Open
We propose a novel online learning method for minimizing regret in large extensive-form games. The approach learns a function approximator online to estimate the regret for choosing a particular action. A no-regret algorithm uses these est…