Welcome to the new version of CaltechAUTHORS. Login is currently restricted to library staff. If you notice any issues, please email coda@library.caltech.edu
Published 1994 | Published
Book Section - Chapter Open

Optimal Brain Surgeon: Extensions and performance comparison.

Abstract

We extend Optimal Brain Surgeon (OBS) - a second-order method for pruning networks - to allow for general error measures, and explore a reduced computational and storage implementation via a dominant eigenspace decomposition. Simulations on nonlinear, noisy pattern classification problems reveal that OBS does lead to improved generalization, and performs favorably in comparison with Optimal Brain Damage (OBD). We find that the required retraining steps in OBD may lead to inferior generalization, a result that can be interpreted as due to injecting noise back into the system. A common technique is to stop training of a large network at the minimum validation error. We found that the test error could be reduced even further by means of OBS (but not OBD) pruning. Our results justify the t → o approximation used in OBS and indicate why retraining in a highly pruned network may lead to inferior performance.

Additional Information

Copyright 1994. Thanks to T. Kailath for support of B.H. through grants AFOSR 91-0060 and DAAL03-91-C-0010.

Attached Files

Published - 749-optimal-brain-surgeon-extensions-and-performance-comparisons.pdf

Files

749-optimal-brain-surgeon-extensions-and-performance-comparisons.pdf
Files (1.7 MB)

Additional details

Created:
August 20, 2023
Modified:
March 5, 2024