1. Home
  2. Issues
  3. Volume 2, Issue 3 (2024)
  4. Meta-analysis of Censored Adverse Events

The New England Journal of Statistics in Data Science


Meta-analysis of Censored Adverse Events
Volume 2, Issue 3 (2024), pp. 380–392
Pub. online: 11 June 2024      Type: Statistical Methodology      Open accessOpen Access
Accepted
3 January 2024
Published
11 June 2024

Abstract

Meta-analysis is a powerful tool for assessing drug safety by combining treatment-related toxicological findings across multiple studies, as clinical trials are typically underpowered for detecting adverse drug effects. However, incomplete reporting of adverse events (AEs) in published clinical studies is frequently encountered, especially if the observed number of AEs is below a pre-specified study-dependent threshold. Ignoring the censored AE information, often found in lower frequency, can significantly bias the estimated incidence rate of AEs. Despite its importance, this prevalent issue in meta-analysis has received little statistical or analytic attention in the literature. To address this challenge, we propose a Bayesian approach to accommodating the censored and possibly rare AEs for meta-analysis of safety data. Through simulation studies, we demonstrate that the proposed method can improve accuracy in point and interval estimation of incidence probabilities, particularly in the presence of censored data. Overall, the proposed method provides a practical solution that can facilitate better-informed decisions regarding drug safety.

Supplementary material

 Supplementary Material
The Supplementary Material includes the JAGS model specification (Section A) for the application in Section 4, the additional results of Table 2 (Section B) under Scenarios 5 and 6 in Section 3.2, and a comprehensive forest plot across all 125 clinical studies (Section C).

References

[1] 
Berlin, J., Crowe, B., Whalen, E., Xia, H., Koro, C. and Kuebler, J. (2013). Meta-analysis of clinical trial safety data in a drug development program: answers to frequently asked questions. Clinical Trials 10(1) 20–31.
[2] 
Bhaumik, D. K., Amatya, A., Normand, S. q. L. T., Greenhouse, J., Kaizar, E., Neelon, B. and Gibbons, R. D. (2012). Meta-analysis of rare binary adverse event data. Journal of the American Statistical Association 107(498) 555–567. https://doi.org/10.1080/01621459.2012.664484. MR2980067
[3] 
Bradburn, M. J., Deeks, J. J., Berlin, J. A. and Russell Localio, A. (2007). Much ado about nothing: a comparison of the performance of meta-analytical methods with rare events. Statistics in Medicine 26(1) 53–77. https://doi.org/10.1002/sim.2528. MR2312699
[4] 
Brahmer, J. R., Drake, C. G. et al. (2010). Phase I study of single-agent anti–programmed death-1 (MDX-1106) in refractory solid tumors: safety, clinical activity, pharmacodynamics, and immunologic correlates. Journal of Clinical Oncology 28(19) 3167.
[5] 
Burke, D. L., Ensor, J. and Riley, R. D. (2017). Meta-analysis using individual participant data: one-stage and two-stage approaches, and why they may differ. Statistics in Medicine 36(5) 855–875. https://doi.org/10.1002/sim.7141. MR3597661
[6] 
Cai, T., Parast, L. and Ryan, L. (2010). Meta-analysis for rare events. Statistics in Medicine 29(20) 2078–2089. https://doi.org/10.1002/sim.3964. MR2756556
[7] 
Carriere, K. C. (2001). How good is a normal approximation for rates and proportions of low incidence events? Communications in Statistics – Simulation and Computation 30(2) 327–337. https://doi.org/10.1081/SAC-100002370. MR1864235
[8] 
Casella, G. and Berger, R. L. (2002) Statistical Inference 2. Duxbury Pacific Grove, CA. MR1051420
[9] 
Chuzi, S., Tavora, F., Cruz, M., Costa, R., Chae, Y. K., Carneiro, B. A. and Giles, F. J. (2017). Clinical features, diagnostic challenges, and management strategies in checkpoint inhibitor-related pneumonitis. Cancer Management and Research 9 207.
[10] 
Council for International Organizations of Medical Sciences (1999) Guidelines for Preparing Core Clinical-safety Information on Drugs: Report of CIOMS Working Groups III and V: Including New Proposals for Investigator’s Brochures. CIOMS.
[11] 
Debray, T. P., Moons, K. G., Abo-Zaid, G. M. A., Koffijberg, H. and Riley, R. D. (2013). Individual participant data meta-analysis for a binary outcome: one-stage or two-stage? PloS ONE 8(4) 60650.
[12] 
Deeks, J. J. (2002). Issues in the selection of a summary statistic for meta-analysis of clinical trials with binary outcomes. Statistics in Medicine 21(11) 1575–1600.
[13] 
FDA (2006). Guidance for Industry: Adverse Reactions Section of Labeling for Human Prescription Drug and Biological Products – Content and Format. Technical Report.
[14] 
Gelman, A. (2006). Prior distributions for variance parameters in hierarchical models. Bayesian Analysis 1(3) 515–534. https://doi.org/10.1214/06-BA117A. MR2221284
[15] 
Gelman, A., Carlin, J. B., Stern, H. S. and Rubin, D. B. (1995) Bayesian Data Analysis. Chapman and Hall/CRC. MR1385925
[16] 
Gelman, A., Jakulin, A., Pittau, M. G. and Su, Y.-S. (2008). A weakly informative default prior distribution for logistic and other regression models. The Annals of Applied Statistics 2(4) 1360–1383. https://doi.org/10.1214/08-AOAS191. MR2655663
[17] 
Golder, S., Loke, Y. K., Wright, K. and Norman, G. (2016). Reporting of adverse events in published and unpublished studies of health care interventions: a systematic review. PLoS Medicine 13(9) 1002127.
[18] 
Hamza, T. H., van Houwelingen, H. C. and Stijnen, T. (2008). The binomial distribution of meta-analysis was preferred to model within-study variability. Journal of Clinical Epidemiology 61(1) 41–51.
[19] 
Higgins, J. P., White, I. R. and Wood, A. M. (2008). Imputation methods for missing outcome data in meta-analysis of clinical trials. Clinical Trials 5(3) 225–239.
[20] 
Higgins, J. P., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M. J. and Welch, V. A. (2019) Cochrane Handbook for Systematic Reviews of Interventions. John Wiley & Sons.
[21] 
Hill, H. A., Qi, X., Jain, P., Nomie, K., Wang, Y., Zhou, S. and Wang, M. L. (2020). Genetic mutations and features of mantle cell lymphoma: a systematic review and meta-analysis. Blood Advances 4(13) 2927–2938.
[22] 
Horn, L., Gettinger, S. N., Gordon, M. S., Herbst, R. S., Gandhi, L., Felip, E., Sequist, L. V., Spigel, D. R., Antonia, S. J., Balmanoukian, A. et al. (2018). Safety and clinical activity of atezolizumab monotherapy in metastatic non-small-cell lung cancer: final results from a phase I study. European Journal of Cancer 101 201–209.
[23] 
Hsu, C., Lee, S.-H., Ejadi, S., Even, C., Cohen, R. B., Le Tourneau, C., Mehnert, J. M., Algazi, A., van Brummelen, E. M., Saraf, S. et al. (2017). Safety and antitumor activity of pembrolizumab in patients with programmed death-ligand 1–positive nasopharyngeal carcinoma: results of the KEYNOTE-028 study. Journal of Clinical Oncology 35(36) 4050–4056.
[24] 
Jewell, N. P. (2003) Statistics for Epidemiology. Chapman and Hall/CRC.
[25] 
Johnson, T. R. and Kuhn, K. M. (2013). Bayesian Thurstonian models for ranking data using JAGS. Behavior Research Methods 45(3) 857–872.
[26] 
Kalaycioglu, O., Copas, A., King, M. and Omar, R. Z. A comparison of multiple-imputation methods for handling missing data in repeated measurements observational studies. Journal of the Royal Statistical Society. Series A (Statistics in Society) 179(3) 683–706 (2016). https://doi.org/10.1111/rssa.12140. MR3501428
[27] 
Kruschke, J. (2014) Doing Bayesian Data Analysis: A Tutorial with R, JAGS, and Stan. Academic Press.
[28] 
Lane, P. W. (2013). Meta-analysis of incidence of rare events. Statistical Methods in Medical Research 22(2) 117–132. https://doi.org/10.1177/0962280211432218. MR3190650
[29] 
Le, D. T., Uram, J. N., Wang, H., Bartlett, B. R., Kemberling, H., Eyring, A. D., Skora, A. D., Luber, B. S., Azad, N. S., Laheru, D., Biedrzycki, B., Donehower, R. C., Zaheer, A., Fisher, G. A., Crocenzi, T. S., Lee, J. J., Duffy, S. M., Goldberg, R. M., de la Chapelle, A., Koshiji, M., Bhaijee, F., Huebner, T., Hruban, R. H., Wood, L. D., Cuka, N., Pardoll, D. M., Papadopoulos, N., Kinzler, K. W., Zhou, S., Cornish, T. C., Taube, J. M., Anders, R. A., Eshleman, J. R., Vogelstein, B. and Diaz, L. A. (2015). PD-1 Blockade in Tumors with Mismatch-Repair Deficiency. New England Journal of Medicine 372(26) 2509–2520. PMID: 26028255. https://doi.org/10.1056/NEJMoa1500596.
[30] 
Lee, J. S., Lee, K. H., Cho, E. K., Kim, D.-W., Kim, S.-W., Kim, J.-H., Cho, B. C., Kang, J. H., Han, J.-Y., Min, Y. J. et al. (2018). Nivolumab in advanced non-small-cell lung cancer patients who failed prior platinum-based chemotherapy. Lung Cancer 122 234–242.
[31] 
Little, R. J. and Rubin, D. B. (2019) Statistical Analysis with Missing Data 793. John Wiley & Sons. https://doi.org/10.1002/9781119013563. MR1925014
[32] 
Liu, D., Liu, R. Y. and Xie, M.-g. (2014). Exact meta-analysis approach for discrete data and its application to $2\times 2$ tables with rare events. Journal of the American Statistical Association 109(508) 1450–1465. https://doi.org/10.1080/01621459.2014.946318. MR3293603
[33] 
Luft, H. S. and Brown Jr, B. (1993). Calculating the probability of rare events: why settle for an approximation? Health Services Research 28(4) 419.
[34] 
Ma, Y., Chu, H. and Mazumdar, M. (2016). Meta-analysis of proportions of rare events–a comparison of exact likelihood methods with robust variance estimation. Communications in Statistics – Simulation and Computation 45(8) 3036–3052. https://doi.org/10.1080/03610918.2014.911901. MR3514856
[35] 
Mathes, T. and Kuss, O. (2018). A comparison of methods for meta-analysis of a small number of studies with binary outcomes. Research Synthesis Methods 9(3) 366–381.
[36] 
Mavridis, D., Chaimani, A., Efthimiou, O., Leucht, S. and Salanti, G. (2014). Addressing missing outcome data in meta-analysis. Evidence-Based Mental Health 17(3) 85–89.
[37] 
Nanda, R., Chow, L. Q., Dees, E. C., Berger, R., Gupta, S., Geva, R., Pusztai, L., Pathiraja, K., Aktan, G., Cheng, J. D. et al. (2016). Pembrolizumab in patients with advanced triple-negative breast cancer: phase Ib KEYNOTE-012 study. Journal of Clinical Oncology 34(21) 2460.
[38] 
Peizer, D. B. and Pratt, J. W. (1968). A normal approximation for binomial, F, beta, and other common, related tail probabilities, I. Journal of the American Statistical Association 63(324) 1416–1456. MR0235650
[39] 
Pigott, T. D. Methods for handling missing data in research synthesis. In The Handbook of Research Synthesis 163–176 (1994).
[40] 
Plummer, M. (2003). JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling. In Proceedings of the 3rd International Workshop on Distributed Statistical Computing 124. Vienna, Austria.
[41] 
Plummer, M. (2008). Penalized loss functions for Bayesian model comparison. Biostatistics 9(3) 523–539.
[42] 
Plummer, M. (2012). SOURCEFORGE JAGS: Just Another Gibbs Sampler Help Forum: dinterval(). https://sourceforge.net/p/mcmc-jags/discussion/610037/thread/fd7f3f7e/.
[43] 
Plummer, M. (2017). JAGS version 4.3. 0 user manual.
[44] 
Powles, T., Durán, I., Van der Heijden, M. S., Loriot, Y., Vogelzang, N. J., De Giorgi, U., Oudard, S., Retz, M. M., Castellano, D., Bamias, A. et al. (2018). Atezolizumab versus chemotherapy in patients with platinum-treated locally advanced or metastatic urothelial carcinoma (IMvigor211): a multicentre, open-label, phase 3 randomised controlled trial. The Lancet 391(10122) 748–757.
[45] 
Qi, X., Zhou, S. and Plummer, M. (2022). On Bayesian modeling of censored data in JAGS. BMC Bioinformatics 23(1) 1–13.
[46] 
Robert, C., Schachter, J., Long, G. V., Arance, A., Grob, J. J., Mortier, L., Daud, A., Carlino, M. S., McNeil, C., Lotem, M. et al. (2015). Pembrolizumab versus ipilimumab in advanced melanoma. New England Journal of Medicine 372(26) 2521–2532.
[47] 
Silva, M. A., Swanson, A. C., Gandhi, P. J. and Tataronis, G. R. (2006). Statin-related adverse events: a meta-analysis. Clinical Therapeutics 28(1) 26–35.
[48] 
Simmonds, M. C., Higginsa, J. P., Stewartb, L. A., Tierneyb, J. F., Clarke, M. J. and Thompson, S. G. (2005). Meta-analysis of individual patient data from randomized trials: a review of methods used in practice. Clinical Trials 2(3) 209–217.
[49] 
Spiegelhalter, D. J., Best, N. G., Carlin, B. P. and Van Der Linde, A. (2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 64(4) 583–639. https://doi.org/10.1111/1467-9868.00353. MR1979380
[50] 
Stoto, M. (2015). Drug safety meta-analysis: promises and pitfalls. Drug Safety 38(3) 233–243.
[51] 
Sutton, A. J., Cooper, N. J., Lambert, P. C., Jones, D. R., Abrams, K. R. and Sweeting, M. J. (2002). Meta-analysis of rare and adverse event data. Expert Review of Pharmacoeconomics & Outcomes Research 2(4) 367–379.
[52] 
Sweeting, M. J., Sutton, A. J. and Lambert, P. C. (2004). What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. Statistics in Medicine 23(9) 1351–1375.
[53] 
Vehtari, A., Gelman, A. and Gabry, J. (2017). Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Statistics and Computing 27(5) 1413–1432. https://doi.org/10.1007/s11222-016-9696-4. MR3647105
[54] 
Wang, Y., Zhou, S., Yang, F., Qi, X., Wang, X., Guan, X., Shen, C., Duma, N., Aguilera, J. V., Chintakuntlawar, A. et al. (2019). Treatment-related adverse events of PD-1 and PD-L1 inhibitors in clinical trials: a systematic review and meta-analysis. JAMA Oncology 5(7) 1008–1019.
[55] 
Watanabe, S. Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory. Journal of Machine Learning Research 11(12) 3571–3594 (2010). MR2756194
[56] 
White, I. R., Higgins, J. P. and Wood, A. M. (2008). Allowing for uncertainty due to missing data in meta-analysis—part 1: two-stage methods. Statistics in Medicine 27(5) 711–727. https://doi.org/10.1002/sim.3008. MR2418509
[57] 
White, I. R., Welton, N. J., Wood, A. M., Ades, A. and Higgins, J. P. (2008). Allowing for uncertainty due to missing data in meta-analysis—part 2: hierarchical models. Statistics in Medicine 27(5) 728–745. https://doi.org/10.1002/sim.3007. MR2418510
[58] 
Zhang, Z. and Sun, J. (2010). Interval censoring. Statistical Methods in Medical Research 19(1) 53–70. https://doi.org/10.1177/0962280209105023. MR2744492
[59] 
Zhong, L., Altan, M., Shannon, V. R. and Sheshadri, A. (2020). Immune-related adverse events: pneumonitis. Immunotherapy 1244 255–269.
[60] 
Zhou, S. (2023). Posterior Averaging Information Criterion. Entropy 25(3) 468. https://doi.org/10.3390/e25030468. MR4575164
[61] 
Zhou, S. and Shen, C. (2022). Statistical concerns for meta-analysis of rare events and small sample sizes. The Lancet Infectious Diseases 22(8) 1111–1112.

Full article Related articles PDF XML
Full article Related articles PDF XML

Copyright
© 2024 New England Statistical Society
Open access article under the CC BY license.

Keywords
Adverse drug reaction Bayesian inference Drug safety Incomplete reporting MAGEC Meta-analysis

Funding
This work is supported in part by NIH/NCI CCSG Grant P30CA016672 and NIH/NLM Grant R21LM014534. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Metrics
since December 2021
139

Article info
views

 47

Full article
views
55

PDF
downloads

 19

XML
downloads


Share


RSS