Multiple modes in high dimensional regression settings

 

We adopt a Bayesian approach with priors for the regression coefficients that are scale mixtures of normal distributions and embody a high prior probability of proximity to zero. By seeking modal estimates we generalise the lasso in a way that provides automated variable selection is adaptive through a range of models from ridge to lass to quasi-Cauchy through extreme spiked versions to the limiting normal-Jeffreys.

Properties of the priors and their resultant posteriors are explored in the context of the linear and generalized linear model especially when there are more variables than observations. We develop MAP algorithms that embrace the need to explore multiple modes of the non log-concave posterior distributions using multiple simulated perfectly fitting starting values. In the context of more variables than observations this multimodality is argued to be a desirable aspect of inference. The methodology is illustrated with a simulation study and a proteomic example. This work extends and further develops Griffin and Brown (2005).

References:
Griffin, J. E. and Brown, P. J. "Alternative prior distributions for variable selection with very many variables than observations," Technical Report, University of Kent.

Joint work with: J. E. Griffin and C. J. Hoggart