Parameter identifiability in Sparse Bayesian Learning

Abstract

The problem of parameter identifiability in linear underdetermined models is addressed, where the observed data vectors follow a multivariate Gaussian distribution. The problem is underdetermined because the dimension of parameters characterizing the distribution of the data is larger than the dimension of the observed vectors. Such models arise frequently in Bayesian Compressive sensing and Sparse Bayesian Learning problems, where the parameter vector to be estimated, is assumed to be sparse. We establish explicit conditions for parameter identifiability in such models, by relating the ambient dimension of the hyperparameter space and that of the data. We establish a crucial result that in such underdetermined models, even without requiring the parameter to be sparse, it is possible to guarantee unique identifiability of the parameters as long as these two dimensions satisfy a certain condition. When such a condition is violated, the unconstrained statistical model is no more identifiable and additional constraints in the form of sparsity need to be enforced to recover the true parameter.

Additional details