Bayesian Relaxation methods for the exploratory analysis of gene expression data

 

Modern genomics has reinforced the need for statistical methods which can explore low-dimensional structure in very high dimensional data. In this respect, classical Relaxation methods, including Boosting and Regularisation, have proved remarkably successful in solving similar problems and have arguably had an important influence within the field of regression analysis. Relaxation methods are best described as forward stagewise model fitting procedures which start from a simple null model and then walk along a path of increasing model complexity by adding components to the model at each stage. Such methods are ideally suited to the analysis of genomics data because they usually provide "automatic'' selection and shrinkage of predictors. In this paper, we consider relaxation methods from a fully probabilistic standpoint, accommodating model uncertainty through a prior distribution on the set of all paths for a given relaxation type. We make inference by Importance Sampling over paths in order to retain the principle characteristics of Relaxation. This appears to provide qualitative and quantitative improvements over MCMC. These new probabilistic relaxation methods are built upon a connection between Regularisation and Generalised Ridge Regression. Results seem to suggest that these algorithms are of practical importance as alternatives to standard Bayesian methods and add weight to our belief that modelling the variance components of Bayesian GLMs provides a powerful and attractive approach to model determination in modern (genomic) data analysis.