Neuroevolution of Recurrent Architectures on Control Tasks Article Swipe

MAKE IMPACT

Maximilien Le Clei , Pierre Bellec · YOU? · · 2023 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2304.12431

Modern artificial intelligence works typically train the parameters of fixed-sized deep neural networks using gradient-based optimization techniques. Simple evolutionary algorithms have recently been shown to also be capable of optimizing deep neural network parameters, at times matching the performance of gradient-based techniques, e.g. in reinforcement learning settings. In addition to optimizing network parameters, many evolutionary computation techniques are also capable of progressively constructing network architectures. However, constructing network architectures from elementary evolution rules has not yet been shown to scale to modern reinforcement learning benchmarks. In this paper we therefore propose a new approach in which the architectures of recurrent neural networks dynamically evolve according to a small set of mutation rules. We implement a massively parallel evolutionary algorithm and run experiments on all 19 OpenAI Gym state-based reinforcement learning control tasks. We find that in most cases, dynamic agents match or exceed the performance of gradient-based agents while utilizing orders of magnitude fewer parameters. We believe our work to open avenues for real-life applications where network compactness and autonomous design are of critical importance. We provide our source code, final model checkpoints and full results at github.com/MaximilienLC/nra.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2304.12431

PDF

https://arxiv.org/pdf/2304.12431

OA Status

green

Related Works

10

OpenAlex ID

https://openalex.org/W4367060535

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4367060535

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2304.12431

Digital Object Identifier

Title

Neuroevolution of Recurrent Architectures on Control Tasks

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2023

Year of publication

Publication date

2023-04-03

Full publication date if available

Authors

Maximilien Le Clei, Pierre Bellec

List of authors in order

Landing page

https://arxiv.org/abs/2304.12431

Publisher landing page

PDF URL

https://arxiv.org/pdf/2304.12431

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2304.12431

Direct OA link when available

Concepts

Reinforcement learning, Computer science, Neuroevolution, Artificial intelligence, Artificial neural network, Massively parallel, Deep learning, Matching (statistics), Set (abstract data type), Evolutionary algorithm, Evolutionary computation, Code (set theory), Computation, Machine learning, Parallel computing, Algorithm, Statistics, Mathematics, Programming language

Top concepts (fields/topics) attached by OpenAlex

Cited by

0

Total citation count in OpenAlex

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload