Joel Veness
YOU?
Author Swipe
View article: Partition Tree Weighting for Non-Stationary Stochastic Bandits
Partition Tree Weighting for Non-Stationary Stochastic Bandits Open
This paper considers a generalisation of universal source coding for interaction data, namely data streams that have actions interleaved with observations. Our goal will be to construct a coding distribution that is both universal \emph{an…
View article: Compression via Pre-trained Transformers: A Study on Byte-Level Multimodal Data
Compression via Pre-trained Transformers: A Study on Byte-Level Multimodal Data Open
Foundation models are strong data compressors, but when accounting for their parameter size, their compression ratios are inferior to standard compression algorithms. Naively reducing the parameter count does not necessarily help as it det…
View article: Learning Universal Predictors
Learning Universal Predictors Open
Meta-learning has emerged as a powerful approach to train neural networks to learn new tasks quickly from limited data. Broad exposure to different tasks leads to versatile representations enabling general problem solving. But, what are th…
View article: Language Modeling Is Compression
Language Modeling Is Compression Open
It has long been established that predictive models can be transformed into lossless compressors and vice versa. Incidentally, in recent years, the machine learning community has focused on training increasingly large and powerful self-sup…
View article: Randomized Positional Encodings Boost Length Generalization of Transformers
Randomized Positional Encodings Boost Length Generalization of Transformers Open
Transformers have impressive generalization capabilities on tasks with a fixed context length. However, they fail to generalize to sequences of arbitrary length, even for seemingly simple tasks such as duplicating a string. Moreover, simpl…
View article: Memory-Based Meta-Learning on Non-Stationary Distributions
Memory-Based Meta-Learning on Non-Stationary Distributions Open
Memory-based meta-learning is a technique for approximating Bayes-optimal predictors. Under fairly general conditions, minimizing sequential prediction error, measured by the log loss, leads to implicit meta-learning. The goal of this work…
View article: Randomized Positional Encodings Boost Length Generalization of Transformers
Randomized Positional Encodings Boost Length Generalization of Transformers Open
Anian Ruoss, Grégoire Delétang, Tim Genewein, Jordi Grau-Moya, Róbert Csordás, Mehdi Bennani, Shane Legg, Joel Veness. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 2023.
View article: Beyond Bayes-optimality: meta-learning what you know you don't know
Beyond Bayes-optimality: meta-learning what you know you don't know Open
Meta-training agents with memory has been shown to culminate in Bayes-optimal agents, which casts Bayes-optimality as the implicit solution to a numerical optimization problem rather than an explicit modeling assumption. Bayes-optimal agen…
View article: Shaking the foundations: delusions in sequence models for interaction and control
Shaking the foundations: delusions in sequence models for interaction and control Open
The recent phenomenal success of language models has reinvigorated machine learning research, and large sequence models such as transformers are being applied to a variety of domains. One important problem class that has remained relativel…
View article: Shaking the foundations: delusions in sequence models for interaction\n and control
Shaking the foundations: delusions in sequence models for interaction\n and control Open
The recent phenomenal success of language models has reinvigorated machine\nlearning research, and large sequence models such as transformers are being\napplied to a variety of domains. One important problem class that has remained\nrelati…
View article: Reinforcement Learning with Information-Theoretic Actuation
Reinforcement Learning with Information-Theoretic Actuation Open
Reinforcement Learning formalises an embodied agent's interaction with the environment through observations, rewards and actions. But where do the actions come from? Actions are often considered to represent something external, such as the…
View article: Investigating Contingency Awareness Using Atari 2600 Games
Investigating Contingency Awareness Using Atari 2600 Games Open
Contingency awareness is the recognition that some aspects of a future observation are under an agent's control while others are solely determined by the environment. This paper explores the idea of contingency awareness in reinforcement l…
View article: Gated Linear Networks
Gated Linear Networks Open
This paper presents a new family of backpropagation-free neural architectures, Gated Linear Networks (GLNs). What distinguishes GLNs from contemporary neural networks is the distributed and local nature of their credit assignment mechanism…
View article: A rapid and efficient learning rule for biological neural circuits
A rapid and efficient learning rule for biological neural circuits Open
The dominant view in neuroscience is that changes in synaptic weights underlie learning. It is unclear, however, how the brain is able to determine which synapses should change, and by how much. This uncertainty stands in sharp contrast to…
View article: A Combinatorial Perspective on Transfer Learning
A Combinatorial Perspective on Transfer Learning Open
Human intelligence is characterized not only by the capacity to learn complex skills, but the ability to rapidly adapt and acquire new skills within an ever-changing environment. In this work we study how the learning of modular solutions …
View article: Gaussian Gated Linear Networks
Gaussian Gated Linear Networks Open
We propose the Gaussian Gated Linear Network (G-GLN), an extension to the recently proposed GLN family of deep neural networks. Instead of using backpropagation to learn features, GLNs have a distributed and local credit assignment mechani…
View article: Online Learning in Contextual Bandits using Gated Linear Networks
Online Learning in Contextual Bandits using Gated Linear Networks Open
We introduce a new and completely online contextual bandit algorithm called Gated Linear Contextual Bandits (GLCB). This algorithm is based on Gated Linear Networks (GLNs), a recently introduced deep learning architecture with properties w…
View article: Meta-learning of Sequential Strategies
Meta-learning of Sequential Strategies Open
In this report we review memory-based meta-learning as a tool for building sample-efficient strategies that learn from past experience to adapt to any task within a target class. Our goal is to equip the reader with the conceptual foundati…
View article: Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents (Extended Abstract)
Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents (Extended Abstract) Open
The Arcade Learning Environment (ALE) is an evaluation platform that poses the challenge of building AI agents with general competency across dozens of Atari 2600 games. It supports a variety of different problem settings and it has been r…
View article: Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents
Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents Open
The Arcade Learning Environment (ALE) is an evaluation platform that poses the challenge of building AI agents with general competency across dozens of Atari 2600 games. It supports a variety of different problem settings and it has been r…
View article: Reply to Huszár: The elastic weight consolidation penalty is empirically valid
Reply to Huszár: The elastic weight consolidation penalty is empirically valid Open
In our recent work on elastic weight consolidation (EWC) (1) we show that forgetting in neural networks can be alleviated by using a quadratic penalty whose derivation was inspired by Bayesian evidence accumulation. In his letter (2), Dr. …
View article: Online Learning with Gated Linear Networks
Online Learning with Gated Linear Networks Open
This paper describes a family of probabilistic architectures designed for online learning under the logarithmic loss. Rather than relying on non-linear transfer functions, our method gains representational power by the use of data conditio…
View article: Revisiting the Arcade Learning Environment: Evaluation Protocols and\n Open Problems for General Agents
Revisiting the Arcade Learning Environment: Evaluation Protocols and\n Open Problems for General Agents Open
The Arcade Learning Environment (ALE) is an evaluation platform that poses\nthe challenge of building AI agents with general competency across dozens of\nAtari 2600 games. It supports a variety of different problem settings and it\nhas bee…
View article: Overcoming catastrophic forgetting in neural networks
Overcoming catastrophic forgetting in neural networks Open
Significance Deep neural networks are currently the most successful machine-learning technique for solving a variety of tasks, including language translation, image classification, and image generation. One weakness of such models is that,…
View article: Overcoming catastrophic forgetting in neural networks
Overcoming catastrophic forgetting in neural networks Open
The ability to learn tasks in a sequential fashion is crucial to the development of artificial intelligence. Neural networks are not, in general, capable of this and it has been widely thought that catastrophic forgetting is an inevitable …
View article: Compress and Control
Compress and Control Open
This paper describes a new information-theoretic policy evaluation technique for reinforcement learning. This technique converts any compression or density model into a corresponding estimate of value. Under appropriate stationarity and er…