Diego de Las Casas
YOU?
Author Swipe
View article: Experimental research on the TCV tokamak
Experimental research on the TCV tokamak Open
Tokamak à configuration variable (TCV), recently celebrating 30 years of near-continual operation, continues in its missions to advance outstanding key physics and operational scenario issues for ITER and the design of future power plants …
View article: Training Compute-Optimal Large Language Models
Training Compute-Optimal Large Language Models Open
We investigate the optimal model size and number of tokens for training a transformer language model under a given compute budget. We find that current large language models are significantly undertrained, a consequence of the recent focus…
View article: Magnetic control of tokamak plasmas through deep reinforcement learning
Magnetic control of tokamak plasmas through deep reinforcement learning Open
View article: Improving language models by retrieving from trillions of tokens
Improving language models by retrieving from trillions of tokens Open
We enhance auto-regressive language models by conditioning on document chunks retrieved from a large corpus, based on local similarity with preceding tokens. With a $2$ trillion token database, our Retrieval-Enhanced Transformer (RETRO) ob…
View article: Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Scaling Language Models: Methods, Analysis & Insights from Training Gopher Open
Language modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world. In this paper, we present an analysis of Transformer-based…
View article: Transformation-based Adversarial Video Prediction on Large-Scale Data
Transformation-based Adversarial Video Prediction on Large-Scale Data Open
Recent breakthroughs in adversarial generative modeling have led to models capable of producing video samples of high quality, even on large and complex datasets of real-world video. In this work, we focus on the task of video prediction, …
View article: DeepMind Control Suite
DeepMind Control Suite Open
The DeepMind Control Suite is a set of continuous control tasks with a standardised structure and interpretable rewards, intended to serve as performance benchmarks for reinforcement learning agents. The tasks are written in Python and pow…
View article: Dawn of the Selfie Era: The Whos, Wheres, and Hows of Selfies on Instagram
Dawn of the Selfie Era: The Whos, Wheres, and Hows of Selfies on Instagram Open
Online interactions are increasingly involving images, especially those containing human faces, which are naturally attention grabbing and more effective at conveying feelings than text. To understand this new convention of digital culture…