Improving the Performance of Echo State Networks Through State Feedback Article Swipe

MAKE IMPACT

Peter J. Ehlers , Hendra I. Nurdin , Daniel Soh · YOU? · · 2023 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2312.15141

Reservoir computing, using nonlinear dynamical systems, offers a cost-effective alternative to neural networks for complex tasks involving processing of sequential data, time series modeling, and system identification. Echo state networks (ESNs), a type of reservoir computer, mirror neural networks but simplify training. They apply fixed, random linear transformations to the internal state, followed by nonlinear changes. This process, guided by input signals and linear regression, adapts the system to match target characteristics, reducing computational demands. A potential drawback of ESNs is that the fixed reservoir may not offer the complexity needed for specific problems. While directly altering (training) the internal ESN would reintroduce the computational burden, an indirect modification can be achieved by redirecting some output as input. This feedback can influence the internal reservoir state, yielding ESNs with enhanced complexity suitable for broader challenges. In this paper, we demonstrate that by feeding some component of the reservoir state back into the network through the input, we can drastically improve upon the performance of a given ESN. We rigorously prove that, for any given ESN, feedback will almost always improve the accuracy of the output. For a set of three tasks, each representing different problem classes, we find that with feedback the average error measures are reduced by $30\%-60\%$. Remarkably, feedback provides at least an equivalent performance boost to doubling the initial number of computational nodes, a computationally expensive and technologically challenging alternative. These results demonstrate the broad applicability and substantial usefulness of this feedback scheme.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2312.15141

PDF

https://arxiv.org/pdf/2312.15141

OA Status

green

Related Works

10

OpenAlex ID

https://openalex.org/W4390306026

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4390306026

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2312.15141

Digital Object Identifier

Title

Improving the Performance of Echo State Networks Through State Feedback

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2023

Year of publication

Publication date

2023-12-23

Full publication date if available

Authors

Peter J. Ehlers, Hendra I. Nurdin, Daniel Soh

List of authors in order

Landing page

https://arxiv.org/abs/2312.15141

Publisher landing page

PDF URL

https://arxiv.org/pdf/2312.15141

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2312.15141

Direct OA link when available

Concepts

Reservoir computing, Echo state network, Computer science, Nonlinear system, Computational complexity theory, Artificial neural network, State (computer science), Set (abstract data type), Process (computing), Recurrent neural network, Artificial intelligence, Identification (biology), Algorithm, Programming language, Botany, Quantum mechanics, Operating system, Biology, Physics

Top concepts (fields/topics) attached by OpenAlex

Cited by

0

Total citation count in OpenAlex

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload