Welcome to the new version of CaltechAUTHORS. Login is currently restricted to library staff. If you notice any issues, please email coda@library.caltech.edu
Published March 1994 | Published
Journal Article Open

Statistical physics, mixtures of distributions, and the EM algorithm

Abstract

We show that there are strong relationships between approaches to optmization and learning based on statistical physics or mixtures of experts. In particular, the EM algorithm can be interpreted as converging either to a local maximum of the mixtures model or to a saddle point solution to the statistical physics system. An advantage of the statistical physics approach is that it naturally gives rise to a heuristic continuation method, deterministic annealing, for finding good solutions.

Additional Information

© 1994 Massachusetts Institute of Technology. Posted Online April 10, 2008. We would like to thank Eric Mjolsness and Anand Rangarajan for helpful conversations and encouragement. One of us (A.L.Y.) thanks DARPA and the Air Force for support under contract F49620-92-J-0466 and Geoffrey Hinton for a helpful conversation.

Attached Files

Published - YUInc94.pdf

Files

YUInc94.pdf
Files (319.0 kB)
Name Size Download all
md5:abbbbef73abadd8b6192aaf596728787
319.0 kB Preview Download

Additional details

Created:
August 20, 2023
Modified:
October 18, 2023