Lessons Learned from the Bayesian Design and Analysis for the BNT162b2 COVID-19 Vaccine Phase 3 Trial

Ji, Yuan; Yuan, Shijie

doi:10.51387/26-NEJSDS93

The New England Journal of Statistics in Data Science

Lessons Learned from the Bayesian Design and Analysis for the BNT162b2 COVID-19 Vaccine Phase 3 Trial

Volume 3, Issue 2 (2025), pp. 159–163

Yuan Ji Shijie Yuan

https://doi.org/10.51387/26-NEJSDS93

Pub. online: 22 January 2026 Type: Commentary And/or Historical Perspective

Open Access

Area: Cancer Research

Accepted
8 September 2025

Published
22 January 2026

Abstract

The phase III BNT162b2 mRNA COVID-19 vaccine trial is based on a Bayesian design and analysis, and the main evidence of vaccine efficacy is presented in Bayesian statistics. Confusion and mistakes arise in the presentation of the Bayesian results. Some key statistics, such as Bayesian credible intervals, are mislabeled and stated as confidence intervals. Posterior probabilities of the vaccine efficacy are not reported as the main results. We illustrate the main differences in the reporting of Bayesian analysis results for a clinical trial and provide four recommendations. We argue that statistical evidence from a Bayesian trial, when presented properly, is easier to interpret and directly addresses the main clinical questions, thereby better supporting regulatory decision making. We also recommend using the abbreviation “BI” to represent Bayesian credible interval as a differentiation to “CI” which stands for confidence interval.

Supplementary material

Supplementary Material

The supplementary R file contains fully reproducible code for all Bayesian analyses presented in this paper. The code implements (i) the betabinomial model used to reproduce the reported Bayesian credible intervals in Polack et al. (2020), (ii) the alternative binomial model with independent priors on infection probabilities for the vaccine and placebo groups, and (iii) posterior samplingbased computation of vaccine efficacy, credible intervals, and posterior probabilities under different prior specifications. The file also includes code for generating all numerical results and figures reported in the main text and supplementary material.

References

[1]

BioNTech, S. (2020). Study to describe the safety, tolerability, immunogenicity, and efficacy of RNA vaccine candidates against COVID-19 in healthy individuals. ClinicalTrials. gov: NCT04368728.

[2]

Chen, Y. and Pei, J. (2010). Factors influencing the association between CYP17 T34C polymorphism and the risk of breast cancer: meta-regression and subgroup analysis. Breast Cancer Research and Treatment 122 471–481.

[3]

Cox, D. R. and Hinkley, D. V. (1979) Theoretical statistics. CRC Press. MR0370837

[4]

de Sire, A., Moggio, L., Demeco, A., Fortunato, F., Spanò, R., Aiello, V., Marotta, N. and Ammendolia, A. (2022). Efficacy of rehabilitative techniques in reducing hemiplegic shoulder pain in stroke: Systematic review and meta-analysis. Annals of physical and rehabilitation medicine 65(5) 101602.

[5]

Egorov, V., Kearney, T., Pollak, S. B., Rohatgi, C., Sarvazyan, N., Airapetian, S., Browning, S. and Sarvazyan, A. (2009). Differentiation of benign and malignant breast lesions by mechanical imaging. Breast cancer research and treatment 118 67–80.

[6]

Muehlemann, N., Zhou, T., Mukherjee, R., Hossain, M. I., Roychoudhury, S. and Russek-Cohen, E. (2023). A tutorial on modern Bayesian methods in clinical trials. Therapeutic Innovation & Regulatory Science 57(3) 402–416.

[7]

Orenstein, W. A., Bernier, R. H., Dondero, T. J., Hinman, A. R., Marks, J. S., Bart, K. J. and Sirotkin, B. (1985). Field evaluation of vaccine efficacy. Bulletin of the World Health Organization 63(6) 1055.

[8]

Polack, F. P., Thomas, S. J., Kitchin, N., Absalon, J., Gurtman, A., Lockhart, S., Perez, J. L., Pérez Marc, G., Moreira, E. D., Zerbini, C. et al. (2020). Safety and efficacy of the BNT162b2 mRNA Covid-19 vaccine. New England Journal of Medicine.

[9]

U. S. FDA (2010). Guidance for the Use of Bayesian Statistics in Medical Device Clinical Trials. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/guidance-use-bayesian-statistics-medical-device-clinical-trials-pdf-version.

[10]

U. S. FDA (2020). Emergency Use Authorization for Vaccines to Prevent COVID-19 – Guidance for Industry. https://www.fda.gov/news-events/fda-brief/fda-brief-fda-issues-guidance-emergency-use-authorization-covid-19-vaccines.