Parallel algorithms/architectures for neural networks

This paper advocates digital VLSI architectures for implementing a wide variety of artificial neural networks (ANNs). A programmable systolic array is proposed, which maximizes the strength of VLSI in terms of intensive and pipelined computing and yet circumvents the limitation on communication. The array is meant to be more general purpose than most other ANN architectures proposed. It may be used for a variety of algorithms in both the retrieving and learning phases of ANNs: e.g., single layer feedback networks, competitive learning networks, and multilayer feed-forward networks. A unified approach to modeling of existing neural networks is proposed. This unified formulation leads to a basic structure for a universal simulation tool and neurocomputer architecture. Fault-tolerance approach and partitioning scheme for large or non-homogeneous networks are also proposed. Finally, the implementations based on commercially available VLSI chips (e.g., Inmos T800) and custom VLSI technology are discussed in great detail.