Algebraic projection analysis for back-propagation learning

S. Y. Kung; J. N. Hwang

doi:10.1016/0893-6080(88)90567-9

Algebraic projection analysis for back-propagation learning

Research output: Contribution to journal › Conference article › peer-review

Abstract

Two critical issues in the back-propagation (BP) learning are the discrimination capability given a number of hidden units and the speed of convergence in learning. The number of hidden units must be sufficient to provide the discriminating capability required by the given application. On the other hand, the training of an excessively large number of synaptic weights may be computationally costly and unreliable. This makes it desirable to have an a priori estimate of an optimal number of hidden neurons. Another closely related issue is the learning rate of the BP scheme. In general, it is desirable to have a fast learning, but not so fast as incurring instability of the iterative computation. Based on this principle an optimal learning rate should be determined. An algebraic projection analysis (APA) is proposed that provides an analytical solution to both of the problems. The simulation results support the optimal hidden units size and learning rate as theoretically predicted by the APA.

Original language	English (US)
Pages (from-to)	547
Number of pages	1
Journal	Neural Networks
Volume	1
Issue number	1 SUPPL
DOIs	https://doi.org/10.1016/0893-6080(88)90567-9
State	Published - 1988
Event	International Neural Network Society 1988 First Annual Meeting - Boston, MA, USA Duration: Sep 6 1988 → Sep 10 1988

All Science Journal Classification (ASJC) codes

Cognitive Neuroscience
Artificial Intelligence

Access to Document

10.1016/0893-6080(88)90567-9

Cite this

@article{bba0fe39ed0f4092970d736207f2e37b,

title = "Algebraic projection analysis for back-propagation learning",

abstract = "Two critical issues in the back-propagation (BP) learning are the discrimination capability given a number of hidden units and the speed of convergence in learning. The number of hidden units must be sufficient to provide the discriminating capability required by the given application. On the other hand, the training of an excessively large number of synaptic weights may be computationally costly and unreliable. This makes it desirable to have an a priori estimate of an optimal number of hidden neurons. Another closely related issue is the learning rate of the BP scheme. In general, it is desirable to have a fast learning, but not so fast as incurring instability of the iterative computation. Based on this principle an optimal learning rate should be determined. An algebraic projection analysis (APA) is proposed that provides an analytical solution to both of the problems. The simulation results support the optimal hidden units size and learning rate as theoretically predicted by the APA.",

author = "Kung, {S. Y.} and Hwang, {J. N.}",

year = "1988",

doi = "10.1016/0893-6080(88)90567-9",

language = "English (US)",

volume = "1",

pages = "547",

journal = "Neural Networks",

issn = "0893-6080",

publisher = "Elsevier Ltd",

number = "1 SUPPL",

note = "International Neural Network Society 1988 First Annual Meeting ; Conference date: 06-09-1988 Through 10-09-1988",

}

TY - JOUR

T1 - Algebraic projection analysis for back-propagation learning

AU - Kung, S. Y.

AU - Hwang, J. N.

PY - 1988

Y1 - 1988

N2 - Two critical issues in the back-propagation (BP) learning are the discrimination capability given a number of hidden units and the speed of convergence in learning. The number of hidden units must be sufficient to provide the discriminating capability required by the given application. On the other hand, the training of an excessively large number of synaptic weights may be computationally costly and unreliable. This makes it desirable to have an a priori estimate of an optimal number of hidden neurons. Another closely related issue is the learning rate of the BP scheme. In general, it is desirable to have a fast learning, but not so fast as incurring instability of the iterative computation. Based on this principle an optimal learning rate should be determined. An algebraic projection analysis (APA) is proposed that provides an analytical solution to both of the problems. The simulation results support the optimal hidden units size and learning rate as theoretically predicted by the APA.

AB - Two critical issues in the back-propagation (BP) learning are the discrimination capability given a number of hidden units and the speed of convergence in learning. The number of hidden units must be sufficient to provide the discriminating capability required by the given application. On the other hand, the training of an excessively large number of synaptic weights may be computationally costly and unreliable. This makes it desirable to have an a priori estimate of an optimal number of hidden neurons. Another closely related issue is the learning rate of the BP scheme. In general, it is desirable to have a fast learning, but not so fast as incurring instability of the iterative computation. Based on this principle an optimal learning rate should be determined. An algebraic projection analysis (APA) is proposed that provides an analytical solution to both of the problems. The simulation results support the optimal hidden units size and learning rate as theoretically predicted by the APA.

UR - http://www.scopus.com/inward/record.url?scp=0024167380&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0024167380&partnerID=8YFLogxK

U2 - 10.1016/0893-6080(88)90567-9

DO - 10.1016/0893-6080(88)90567-9

M3 - Conference article

AN - SCOPUS:0024167380

SN - 0893-6080

VL - 1

SP - 547

JO - Neural Networks

JF - Neural Networks

IS - 1 SUPPL

T2 - International Neural Network Society 1988 First Annual Meeting

Y2 - 6 September 1988 through 10 September 1988

ER -

Algebraic projection analysis for back-propagation learning

Abstract

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this