Noninvertibility in neural networks

Ramiro Rico-Martínez, Raymond A. Adomaitis, Ioannis G. Kevrekidis

Research output: Contribution to journalArticle

21 Scopus citations

Abstract

We present and discuss an inherent shortcoming of neural networks used as discrete-time models in system identification, time series processing, and prediction. Trajectories of nonlinear ordinary differential equations (ODEs) can, under reasonable assumptions, be integrated uniquely backward in time. Discrete-time neural network mappings derived from time series, on the other hand, can give rise to multiple trajectories when followed backward in time: they are in principle noninvertible. This fundamental difference can lead to model predictions that are not only slightly quantitatively different, but qualitatively inconsistent with continuous time series. We discuss how noninvertibility arises, present key analytical concepts and some of its phenomenology. Using two illustrative examples (one experimental and one computational), we demonstrate when noninvertibility becomes an important factor in the validity of artificial neural network (ANN) predictions, and show some of the overall complexity of the predicted pathological dynamical behavior. These concepts can be used to probe the validity of ANN time series models, as well as provide guidelines for the acquisition of additional training data. (C) 2000 Elsevier Science Ltd.

Original languageEnglish (US)
Pages (from-to)2417-2433
Number of pages17
JournalComputers and Chemical Engineering
Volume24
Issue number11
DOIs
StatePublished - Nov 1 2000

All Science Journal Classification (ASJC) codes

  • Chemical Engineering(all)
  • Computer Science Applications

Keywords

  • Artificial neural networks
  • Noninvertibility
  • System identification
  • Time-series processing

Fingerprint Dive into the research topics of 'Noninvertibility in neural networks'. Together they form a unique fingerprint.

  • Cite this

    Rico-Martínez, R., Adomaitis, R. A., & Kevrekidis, I. G. (2000). Noninvertibility in neural networks. Computers and Chemical Engineering, 24(11), 2417-2433. https://doi.org/10.1016/S0098-1354(00)00599-8