Effect of Model Space Priors on Statistical Inference with Model Uncertainty
Volume 1, Issue 2 (2023), pp. 149–158
Pub. online: 16 November 2022 Type: Statistical Methodology Open Access
18 October 2022
18 October 2022
16 November 2022
16 November 2022
Bayesian model averaging (BMA) provides a coherent way to account for model uncertainty in statistical inference tasks. BMA requires specification of model space priors and parameter space priors. In this article we focus on comparing different model space priors in the presence of model uncertainty. We consider eight reference model space priors used in the literature and three adaptive parameter priors recommended by Porwal and Raftery . We assess the performance of these combinations of prior specifications for variable selection in linear regression models for the statistical tasks of parameter estimation, interval estimation, inference, point and interval prediction. We carry out an extensive simulation study based on 14 real datasets representing a range of situations encountered in practice. We found that beta-binomial model space priors specified in terms of the prior probability of model size performed best on average across various statistical tasks and datasets, outperforming priors that were uniform across models. Recently proposed complexity priors performed relatively poorly.
Supplementary materialSupplementary Material
The supplementary material contains detailed summary results for each metric and dataset used in the study. It also contains a summary of data-generating models for each of the datasets.
Filzmoser, P. and Varmuza, K. (2017). chemometrics: Multivariate Statistical Analysis in Chemometrics. R package version 1.4.2. https://CRAN.R-project.org/package=chemometrics.
George, E. (1999). Discussion of “Model averaging and model search strategies” by M. Clyde. In Bayesian Statistics 6–Proceedings of the Sixth Valencia International Meeting. MR1723497
George, E. I. (2010). Dilution priors: Compensating for model space redundancy. In Borrowing Strength: Theory Powering Applications–A Festschrift for Lawrence D. Brown 158–165 Institute of Mathematical Statistics. MR2798517
Leamer, E. E. (1978) Specification Searches: Ad hoc Inference with Nonexperimental Data 53. Wiley. MR0471118
Lumley, T. (2020). leaps: Regression Subset Selection. R package version 3.1. https://CRAN.R-project.org/package=leaps.
Newman, D. J., Hettich, S., Blake, C. L. and Merz, C. J. (1998). UCI Repository of machine learning databases. http://www.ics.uci.edu/~mlearn/MLRepository.html.
Raftery, A. E. (1988). Approximate Bayes factors for generalized linear models. Technical Report No. 121, Department of Statistics, University of Washington. https://stat.uw.edu/sites/default/files/files/reports/1988/tr121.pdf.
Rossell, D., Abril, O. and Bhattacharya, A. (2021). Approximate Laplace approximations for scalable model selection. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 83(4) 853–879. MR4320004
Zellner, A. (1986). On assessing prior distributions and Bayesian regression analysis with g-prior distributions. In Bayesian Inference and Decision Techniques 6. MR0881437