Learning continuous attractors in recurrent networks

Hyunjune Sebastian Seung

Learning continuous attractors in recurrent networks

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

One approach to invariant object recognition employs a recurrent neural network as an associative memory. In the standard depiction of the network's state space, memories of objects are stored as attractive fixed points of the dynamics. I argue for a modification of this picture: if an object has a continuous family of instantiations, it should be represented by a continuous attractor. This idea is illustrated with a network that learns to complete patterns. To perform the task of filling in missing information, the network develops a continuous attractor that models the manifold from which the patterns are drawn. Prom a statistical viewpoint, the pattern completion task allows a formulation of unsupervised learning in terms of regression rather than density estimation.

Original language	English (US)
Title of host publication	Advances in Neural Information Processing Systems 10 - Proceedings of the 1997 Conference, NIPS 1997
Publisher	Neural information processing systems foundation
Pages	654-660
Number of pages	7
ISBN (Print)	0262100762, 9780262100762
State	Published - Jan 1 1998
Externally published	Yes
Event	11th Annual Conference on Neural Information Processing Systems, NIPS 1997 - Denver, CO, United States Duration: Dec 1 1997 → Dec 6 1997

Other

Other	11th Annual Conference on Neural Information Processing Systems, NIPS 1997
Country/Territory	United States
City	Denver, CO
Period	12/1/97 → 12/6/97

All Science Journal Classification (ASJC) codes

Computer Networks and Communications
Information Systems
Signal Processing

Cite this

@inproceedings{38e06b0a3a4349a2a14cf56f2cd52b18,

title = "Learning continuous attractors in recurrent networks",

abstract = "One approach to invariant object recognition employs a recurrent neural network as an associative memory. In the standard depiction of the network's state space, memories of objects are stored as attractive fixed points of the dynamics. I argue for a modification of this picture: if an object has a continuous family of instantiations, it should be represented by a continuous attractor. This idea is illustrated with a network that learns to complete patterns. To perform the task of filling in missing information, the network develops a continuous attractor that models the manifold from which the patterns are drawn. Prom a statistical viewpoint, the pattern completion task allows a formulation of unsupervised learning in terms of regression rather than density estimation.",

author = "Seung, {Hyunjune Sebastian}",

year = "1998",

month = jan,

day = "1",

language = "English (US)",

isbn = "0262100762",

pages = "654--660",

booktitle = "Advances in Neural Information Processing Systems 10 - Proceedings of the 1997 Conference, NIPS 1997",

publisher = "Neural information processing systems foundation",

note = "11th Annual Conference on Neural Information Processing Systems, NIPS 1997 ; Conference date: 01-12-1997 Through 06-12-1997",

}

TY - GEN

T1 - Learning continuous attractors in recurrent networks

AU - Seung, Hyunjune Sebastian

PY - 1998/1/1

Y1 - 1998/1/1

N2 - One approach to invariant object recognition employs a recurrent neural network as an associative memory. In the standard depiction of the network's state space, memories of objects are stored as attractive fixed points of the dynamics. I argue for a modification of this picture: if an object has a continuous family of instantiations, it should be represented by a continuous attractor. This idea is illustrated with a network that learns to complete patterns. To perform the task of filling in missing information, the network develops a continuous attractor that models the manifold from which the patterns are drawn. Prom a statistical viewpoint, the pattern completion task allows a formulation of unsupervised learning in terms of regression rather than density estimation.

AB - One approach to invariant object recognition employs a recurrent neural network as an associative memory. In the standard depiction of the network's state space, memories of objects are stored as attractive fixed points of the dynamics. I argue for a modification of this picture: if an object has a continuous family of instantiations, it should be represented by a continuous attractor. This idea is illustrated with a network that learns to complete patterns. To perform the task of filling in missing information, the network develops a continuous attractor that models the manifold from which the patterns are drawn. Prom a statistical viewpoint, the pattern completion task allows a formulation of unsupervised learning in terms of regression rather than density estimation.

UR - http://www.scopus.com/inward/record.url?scp=0000938157&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0000938157&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:0000938157

SN - 0262100762

SN - 9780262100762

SP - 654

EP - 660

BT - Advances in Neural Information Processing Systems 10 - Proceedings of the 1997 Conference, NIPS 1997

PB - Neural information processing systems foundation

T2 - 11th Annual Conference on Neural Information Processing Systems, NIPS 1997

Y2 - 1 December 1997 through 6 December 1997

ER -

Learning continuous attractors in recurrent networks

Abstract

Other

All Science Journal Classification (ASJC) codes

Other files and links

Fingerprint

Cite this