Latent Variable Augmented Sparse Regress

As a powerful tool for producing meaningful and interpretable models, sparse modeling has gained increasing popularity for analyzing large-scale data sets. However, the key assumption of sparse feature effects which underlie high-dimensional statistical inference has been questioned in real applications. On the other hand, most of existing methods assume implicitly that all features in a model are observable. Yet some latent confounding factors may potentially exist in the hidden structure of the original model. In this paper, we consider nonsparse feature effects under the conditional sparsity structure that the original coefficient vector is sparse only after taking out the effects of latent factors. A new framework, latent variable augmented sparse regression (LAVAR), is proposed to simultaneously recover the significant observable predictors and latent factors. In particular, one potential family of latent variables that incorporate the population principal components of observable features is explored and asymptotic properties of sample principal components are established for a wide class of distributions. With the aid of these properties, we prove that the proposed framework can enjoy model selection consistency and oracle inequalities under various prediction and variable selection losses for both observable predictors and latent confounding factors. Our new method and results are evidenced by simulation and real data examples.

主讲人简介：