Exploring unsupervised pre-training for echo state networks

Peter Steiner, Azarakhsh Jalalvand, Peter Birkholz

Research output: Contribution to journalArticlepeer-review

3 Scopus citations


Echo State Networks (ESNs) are a special type of Recurrent Neural Networks (RNNs), in which the input and recurrent connections are traditionally generated randomly, and only the output weights are trained. However, recent publications have addressed the problem that a purely random initialization may not be ideal. Instead, a completely deterministic or data-driven initialized ESN structure was proposed. In this work, an unsupervised training methodology for the hidden components of an ESN is proposed. Motivated by traditional Hidden Markov Models (HMMs), which have been widely used for speech recognition for decades, we present an unsupervised pre-training method for the recurrent weights and bias weights of ESNs. This approach allows for using unlabeled data during the training procedure and shows superior results for continuous spoken phoneme recognition, as well as for a large variety of time-series classification datasets.

Original languageEnglish (US)
Pages (from-to)24225-24242
Number of pages18
JournalNeural Computing and Applications
Issue number34
StatePublished - Dec 2023

All Science Journal Classification (ASJC) codes

  • Software
  • Artificial Intelligence


  • Clustering
  • ESN
  • RCN
  • State machine


Dive into the research topics of 'Exploring unsupervised pre-training for echo state networks'. Together they form a unique fingerprint.

Cite this