Welcome to the new version of CaltechAUTHORS. Login is currently restricted to library staff. If you notice any issues, please email coda@library.caltech.edu
Published December 9, 2014 | Supplemental Material + Published + Accepted Version
Journal Article Open

Provable Tensor Methods for Learning Mixtures of Generalized Linear Models

Abstract

We consider the problem of learning mixtures of generalized linear models (GLM) which arise in classification and regression problems. Typical learning approaches such as expectation maximization (EM) or variational Bayes can get stuck in spurious local optima. In contrast, we present a tensor decomposition method which is guaranteed to correctly recover the parameters. The key insight is to employ certain feature transformations of the input, which depend on the input generative model. Specifically, we employ score function tensors of the input and compute their cross-correlation with the response variable. We establish that the decomposition of this tensor consistently recovers the parameters, under mild non-degeneracy conditions. We demonstrate that the computational and sample complexity of our method is a low order polynomial of the input and the latent dimensions.

Additional Information

© 2016 by the authors. This work was done while H. Sedghi was a visiting researcher at UC Irvine and was supported by NSF Career award FG15890. M. Janzamin is supported by NSF BIGDATA award FG16455. A. Anandkumar is supported in part by Microsoft Faculty Fellowship, NSF Career award CCF-1254106, and ONR Award N00014-14-1-0665.

Attached Files

Published - sedghi16.pdf

Accepted Version - 1412.3046.pdf

Supplemental Material - sedghi16-supp.pdf

Files

sedghi16-supp.pdf
Files (811.8 kB)
Name Size Download all
md5:19feb04956f7167c4090dd31be38bad4
269.4 kB Preview Download
md5:05d2fd2e4f5fab52082aa3a9506d6ea2
286.2 kB Preview Download
md5:5c4b2b94fc5dcfb312e352b75f1bbb25
256.3 kB Preview Download

Additional details

Created:
August 20, 2023
Modified:
October 20, 2023