Welcome to the new version of CaltechAUTHORS. Login is currently restricted to library staff. If you notice any issues, please email coda@library.caltech.edu
Published January 2022 | Submitted + Published
Journal Article Open

Predictive Coding, Variational Autoencoders, and Biological Connections

Abstract

We present a review of predictive coding, from theoretical neuroscience, and variational autoencoders, from machine learning, identifying the common origin and mathematical framework underlying both areas. As each area is prominent within its respective field, more firmly connecting these areas could prove useful in the dialogue between neuroscience and machine learning. After reviewing each area, we discuss two possible correspondences implied by this perspective: cortical pyramidal dendrites as analogous to (nonlinear) deep networks and lateral inhibition as analogous to normalizing flows. These connections may provide new directions for further investigations in each field.

Additional Information

© 2021 Massachusetts Institute of Technology. Received: March 14 2021; Accepted: August 14 2021. Sam Gershman and Rajesh Rao provided helpful comments on this manuscript, and Karl Friston engaged in useful early discussions related to these ideas. We also thank the anonymous reviewers for their feedback and suggestions.

Attached Files

Published - neco_a_01458.pdf

Submitted - 2011-07464.pdf

Files

2011-07464.pdf
Files (10.1 MB)
Name Size Download all
md5:94e99afbd93dcb104a2ae4c615638aeb
6.6 MB Preview Download
md5:832d31305970ec0f7ae5b5a347eb1459
3.4 MB Preview Download

Additional details

Created:
October 5, 2023
Modified:
October 24, 2023