Linear and logarithmic capacities in associative neural networks
- Creators
- Venkatesh, Santosh S.
- Psaltis, Demetri
Abstract
A model of associate memory incorporating global linearity and pointwise nonlinearities in a state space of n-dimensional binary vectors is considered. Attention is focused on the ability to store a prescribed set of state vectors as attractors within the model. Within the framework of such associative nets, a specific strategy for information storage that utilizes the spectrum of a linear operator is considered in some detail. Comparisons are made between this spectral strategy and a prior scheme that utilizes the sum of Kronecker outer products of the prescribed set of state vectors, which are to function nominally as memories. The storage capacity of the spectral strategy is linear in n (the dimension of the state space under consideration), whereas an asymptotic result of n/4 log n holds for the storage capacity of the outer product scheme. Computer-simulated results show that the spectral strategy stores information more efficiently. The preprocessing costs incurred in the two algorithms are estimated, and recursive strategies are developed for their computation.
Additional Information
© Copyright 1989 IEEE. Reprinted with permission. Manuscript received March 20, 1985; revised January 20, 1988. This work was supported in part by the National Science Foundation under Grant EET-8709198. The material in this paper was partially presented at the Workshop on Neural Networks for Computing, Santa Barbara, CA, April 1985. We wish to thank our colleagues Profs. Y.S. Abu-Mostafa, N. Farhat, J.J. Hopfield, R.J. McEliece, and E.C. Posner, and Mr. J. Hong with whom we have had many illuminating discussions on this subject. Our thanks also go to Dr. R. Winslow who kindly made available to us an analytical proof demonstrating that the degenerate spectral scheme always enters a stable state, and to our reviewers who made several useful suggestions for improving the clarity of the presentation. Correction; IEEE Transactions on Information Theory 53(2):854 February 2007.Files
| Name | Size | Download all |
|---|---|---|
|
md5:c793967bf4a87db7c014610a4e40a9fe
|
1.1 MB | Preview Download |
|
md5:e63f7ab9578a1ca50850a74af753d151
|
50.5 kB | Preview Download |
Additional details
- Eprint ID
- 7551
- DOI
- 10.1109/TWC.2006.256975
- Resolver ID
- CaltechAUTHORS:VENieeetit89
- Created
-
2007-03-02Created from EPrint's datestamp field
- Updated
-
2021-11-08Created from EPrint's last_modified field