Koray Kavukcuoglu
YOU?
Author Swipe
View article: Capabilities of Gemini Models in Medicine
Capabilities of Gemini Models in Medicine Open
Excellence in a wide variety of medical applications poses considerable challenges for AI, requiring advanced reasoning, access to up-to-date medical knowledge and understanding of complex multimodal data. Gemini models, with strong genera…
View article: RecurrentGemma: Moving Past Transformers for Efficient Open Language Models
RecurrentGemma: Moving Past Transformers for Efficient Open Language Models Open
We introduce RecurrentGemma, a family of open language models which uses Google's novel Griffin architecture. Griffin combines linear recurrences with local attention to achieve excellent performance on language. It has a fixed-sized state…
View article: Gemma: Open Models Based on Gemini Research and Technology
Gemma: Open Models Based on Gemini Research and Technology Open
This work introduces Gemma, a family of lightweight, state-of-the art open models built from the research and technology used to create Gemini models. Gemma models demonstrate strong performance across academic benchmarks for language unde…
View article: Competition-level code generation with AlphaCode
Competition-level code generation with AlphaCode Open
Programming is a powerful and ubiquitous problem-solving tool. Systems that can assist programmers or even generate programs themselves could make programming more productive and accessible. Recent transformer-based neural network models s…
View article: Improving alignment of dialogue agents via targeted human judgements
Improving alignment of dialogue agents via targeted human judgements Open
We present Sparrow, an information-seeking dialogue agent trained to be more helpful, correct, and harmless compared to prompted language model baselines. We use reinforcement learning from human feedback to train our models with two new a…
View article: Magnetic control of tokamak plasmas through deep reinforcement learning
Magnetic control of tokamak plasmas through deep reinforcement learning Open
View article: Unified Scaling Laws for Routed Language Models
Unified Scaling Laws for Routed Language Models Open
The performance of a language model has been shown to be effectively modeled as a power-law in its parameter count. Here we study the scaling behaviors of Routing Networks: architectures that conditionally use only a subset of their parame…
View article: Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Scaling Language Models: Methods, Analysis & Insights from Training Gopher Open
Language modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world. In this paper, we present an analysis of Transformer-based…
View article: Applying and improving <scp>AlphaFold</scp> at <scp>CASP14</scp>
Applying and improving <span>AlphaFold</span> at <span>CASP14</span> Open
We describe the operation and improvement of AlphaFold, the system that was entered by the team AlphaFold2 to the “human” category in the 14th Critical Assessment of Protein Structure Prediction (CASP14). The AlphaFold system entered in CA…
View article: Author response for "Applying and improving AlphaFold at CASP14"
Author response for "Applying and improving AlphaFold at CASP14" Open
View article: Highly accurate protein structure prediction for the human proteome
Highly accurate protein structure prediction for the human proteome Open
View article: Highly accurate protein structure prediction with AlphaFold
Highly accurate protein structure prediction with AlphaFold Open
Proteins are essential to life, and understanding their structure can facilitate a mechanistic understanding of their function. Through an enormous experimental effort 1–4 , the structures of around 100,000 unique proteins have been determ…
View article: Bootstrap your own latent: A new approach to self-supervised Learning
Bootstrap your own latent: A new approach to self-supervised Learning Open
We introduce Bootstrap Your Own Latent (BYOL), a new approach to self-supervised image representation learning. BYOL relies on two neural networks, referred to as online and target networks, that interact and learn from each other. From an…
View article: Protein structure prediction using multiple deep neural networks in the 13th Critical Assessment of Protein Structure Prediction (CASP13)
Protein structure prediction using multiple deep neural networks in the 13th Critical Assessment of Protein Structure Prediction (CASP13) Open
We describe AlphaFold, the protein structure prediction system that was entered by the group A7D in CASP13. Submissions were made by three free‐modeling (FM) methods which combine the predictions of three neural networks. All three systems…
View article: Human-level performance in 3D multiplayer games with population-based reinforcement learning
Human-level performance in 3D multiplayer games with population-based reinforcement learning Open
Artificial teamwork Artificially intelligent agents are getting better and better at two-player games, but most real-world endeavors require teamwork. Jaderberg et al. designed a computer program that excels at playing the video game Quake…
View article: The StreetLearn Environment and Dataset
The StreetLearn Environment and Dataset Open
Navigation is a rich and well-grounded problem domain that drives progress in many different areas of research: perception, planning, memory, exploration, and optimisation in particular. Historically these challenges have been separately c…
View article: Learning to Navigate in Cities Without a Map
Learning to Navigate in Cities Without a Map Open
Navigating through unstructured environments is a basic capability of intelligent creatures, and thus is of fundamental interest in the study and development of artificial intelligence. Long-range navigation is a complex cognitive task tha…
View article: Unsupervised Predictive Memory in a Goal-Directed Agent
Unsupervised Predictive Memory in a Goal-Directed Agent Open
Animals execute goal-directed behaviours despite the limited range and scope of their sensors. To cope, they explore environments and store memories maintaining estimates of important information that is not presently available. Recently, …
View article: Efficient Neural Audio Synthesis
Efficient Neural Audio Synthesis Open
Sequential models achieve state-of-the-art results in audio, visual and textual domains with respect to both estimating the data distribution and generating high-quality samples. Efficient sampling for this class of models has however rema…
View article: IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures Open
In this work we aim to solve a large collection of tasks using a single reinforcement learning agent with a single set of parameters. A key challenge is to handle the increased amount of data and extended training time. We have developed a…
View article: Parallel WaveNet: Fast High-Fidelity Speech Synthesis
Parallel WaveNet: Fast High-Fidelity Speech Synthesis Open
The recently-developed WaveNet architecture is the current state of the art in realistic speech synthesis, consistently rated as more natural sounding for many different languages than any previous system. However, because WaveNet relies o…
View article: Parallel WaveNet: Fast High-Fidelity Speech Synthesis
Parallel WaveNet: Fast High-Fidelity Speech Synthesis Open
The recently-developed WaveNet architecture is the current state of the art in realistic speech synthesis, consistently rated as more natural sounding for many different languages than any previous system. However, because WaveNet relies o…
View article: Population Based Training of Neural Networks
Population Based Training of Neural Networks Open
Neural networks dominate the modern machine learning landscape, but their training and success still suffer from sensitivity to empirical choices of hyperparameters such as model architecture, loss function, and optimisation algorithm. In …
View article: Neural Discrete Representation Learning
Neural Discrete Representation Learning Open
Learning useful representations without supervision remains a key challenge in machine learning. In this paper, we propose a simple yet powerful generative model that learns such discrete representations. Our model, the Vector Quantised-Va…
View article: Neural Discrete Representation Learning.
Neural Discrete Representation Learning. Open
Learning useful representations without supervision remains a key challenge in machine learning. In this paper, we propose a simple yet powerful generative model that learns such discrete representations. Our model, the Vector Quantised-Va…
View article: Hierarchical Representations for Efficient Architecture Search
Hierarchical Representations for Efficient Architecture Search Open
We explore efficient neural architecture search methods and show that a simple yet powerful evolutionary algorithm can discover new architectures with excellent performance. Our approach combines a novel hierarchical genetic representation…
View article: Automated Curriculum Learning for Neural Networks
Automated Curriculum Learning for Neural Networks Open
We introduce a method for automatically selecting the path, or syllabus, that a neural network follows through a curriculum so as to maximise learning efficiency. A measure of the amount that the network learns from each data sample is pro…
View article: FeUdal Networks for Hierarchical Reinforcement Learning
FeUdal Networks for Hierarchical Reinforcement Learning Open
We introduce FeUdal Networks (FuNs): a novel architecture for hierarchical reinforcement learning. Our approach is inspired by the feudal reinforcement learning proposal of Dayan and Hinton, and gains power and efficacy by decoupling end-t…
View article: Understanding Synthetic Gradients and Decoupled Neural Interfaces
Understanding Synthetic Gradients and Decoupled Neural Interfaces Open
When training neural networks, the use of Synthetic Gradients (SG) allows layers or modules to be trained without update locking - without waiting for a true error gradient to be backpropagated - resulting in Decoupled Neural Interfaces (D…
View article: Interaction Networks for Learning about Objects, Relations and Physics
Interaction Networks for Learning about Objects, Relations and Physics Open
Reasoning about objects, relations, and physics is central to human intelligence, and a key goal of artificial intelligence. Here we introduce the interaction network, a model which can reason about how objects in complex systems interact,…