The New England Journal of Statistics in Data Science logo


  • Help
Login Register

  1. Home
  2. To appear
  3. Optimal Design of Controlled Experiments ...

The New England Journal of Statistics in Data Science

Submit your article Information Become a Peer-reviewer
  • Article info
  • Full article
  • More
    Article info Full article

Optimal Design of Controlled Experiments for Personalized Decision Making in the Presence of Observational Covariates
Yezhuo Li   Qiong Zhang   Amin Khademi     All authors (4)

Authors

 
Placeholder
https://doi.org/10.51387/23-NEJSDS22
Pub. online: 26 January 2023      Type: Statistical Methodology      Open accessOpen Access

Accepted
12 January 2023
Published
26 January 2023

Abstract

Controlled experiments are widely applied in many areas such as clinical trials or user behavior studies in IT companies. Recently, it is popular to study experimental design problems to facilitate personalized decision making. In this paper, we investigate the problem of optimal design of multiple treatment allocation for personalized decision making in the presence of observational covariates associated with experimental units (often, patients or users). We assume that the response of a subject assigned to a treatment follows a linear model which includes the interaction between covariates and treatments to facilitate precision decision making. We define the optimal objective as the maximum variance of estimated personalized treatment effects over different treatments and different covariates values. The optimal design is obtained by minimizing this objective. Under a semi-definite program reformulation of the original optimization problem, we use a YALMIP and MOSEK based optimization solver to provide the optimal design. Numerical studies are provided to assess the quality of the optimal design.

References

[1] 
ApS, M. (2019). The MOSEK optimization toolbox for MATLAB manual. Version 9.0. http://docs.mosek.com/9.0/toolbox/index.html.
[2] 
Atkinson, A. (2015). Optimum designs for two treatments with unequal variances in the presence of covariates. Biometrika 102(2) 494–499. https://doi.org/10.1093/biomet/asu071. MR3371019
[3] 
Ben-Tal, A. and Nemirovski, A. (2001) Lectures on modern convex optimization: analysis, algorithms, and engineering applications. SIAM. https://doi.org/10.1137/1.9780898718829. MR1857264
[4] 
Berry, D. A., Mueller, P., Grieve, A. P., Smith, M., Parke, T., Blazek, R., Mitchard, N. and Krams, M. (2002). Adaptive Bayesian designs for dose-ranging drug trials. In Case studies in Bayesian statistics 99–181 Springer. https://doi.org/10.1007/978-1-4613-0035-9_2. MR1931867
[5] 
Bhat, N., Farias, V. F., Moallemi, C. C. and Sinha, D. (2020). Near-Optimal A-B Testing. Management Science 66(10) 4477–4495.
[6] 
Biswas, A. and López–Fidalgo, J. (2013). Compound designs for dose-finding in the presence of nondesignable covariates. Pharmaceutical Statistics 12(2) 92–101.
[7] 
Blackwelder, W. C. (1982). “Proving the null hypothesis” in clinical trials. Controlled clinical trials 3(4) 345–353.
[8] 
Boyd, S. and Vandenberghe, L. (2004) Convex optimization. Cambridge university press. https://doi.org/10.1017/CBO9780511804441. MR2061575
[9] 
Dette, H. and Studden, W. J. (1993). Geometry of E-optimality. The Annals of Statistics 21(1) 416–433. https://doi.org/10.1214/aos/1176349034. MR1212185
[10] 
Edelman, A. (1988). Eigenvalues and condition numbers of random matrices. SIAM journal on matrix analysis and applications 9(4) 543–560. https://doi.org/10.1137/0609045. MR0964668
[11] 
Edelman, A. (1991). The distribution and moments of the smallest eigenvalue of a random matrix of Wishart type. Linear algebra and its applications 159 55–80. https://doi.org/10.1016/0024-3795(91)90076-9. MR1133335
[12] 
Feit, E. M. and Berman, R. (2019). Test & Roll: Profit-Maximizing A/B Tests. Marketing Science 38(6) 1038–1058.
[13] 
Gally, T., Pfetsch, M. E. and Ulbrich, S. (2018). A framework for solving mixed-integer semidefinite programs. Optimization Methods and Software 33(3) 594–632. https://doi.org/10.1080/10556788.2017.1322081. MR3783675
[14] 
Goemans, M. X. and Williamson, D. P. (1995). Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. Journal of the ACM (JACM) 42(6) 1115–1145. https://doi.org/10.1145/227683.227684. MR1412228
[15] 
Harville, D. A. (1974). Nearly optimal allocation of experimental units using observed covariate values. Technometrics 16(4) 589–599. https://doi.org/10.2307/1267612. MR0418353
[16] 
Hore, S., Dewanji, A. and Chatterjee, A. (2014). Design issues related to allocation of experimental units with known covariates into two treatment groups. Journal of Statistical Planning and Inference 155 117–126. https://doi.org/10.1016/j.jspi.2014.06.002. MR3264459
[17] 
Kohavi, R. and Longbotham, R. (2017). Online Controlled Experiments and A/B Testing. Encyclopedia of machine learning and data mining 7(8) 922–929.
[18] 
Koning, R., Hasan, S. and Chatterji, A. (2022). Experimentation and start-up performance: Evidence from A/B testing. Management Science.
[19] 
Lee, K. M. and Wason, J. (2019). Design of experiments for a confirmatory trial of precision medicine. Journal of Statistical Planning and Inference 199 179–187. https://doi.org/10.1016/j.jspi.2018.06.004. MR3857821
[20] 
Löfberg, J. (2004). YALMIP: A Toolbox for Modeling and Optimization in MATLAB. In In Proceedings of the CACSD Conference.
[21] 
MATLAB (2021) 9.10.0.1710957 (R2021a) Update 4. The MathWorks Inc., Natick, Massachusetts.
[22] 
McFarland, C. (2012) Experiment!: Website conversion rate optimization with A/B and multivariate testing. New Riders.
[23] 
Merikoski, J. K. and Virtanen, A. (1997). Bounds for eigenvalues using the trace and determinant. Linear algebra and its applications 264 101–108. https://doi.org/10.1016/S0024-3795(97)00067-0. MR1465858
[24] 
Pukelsheim, F. (2006) Optimal design of experiments. SIAM. https://doi.org/10.1137/1.9780898719109. MR2224698
[25] 
Rosa, S. (2020). Optimal experimental designs for treatment contrasts in heteroscedastic models with covariates. Journal of Statistical Planning and Inference 209 280–291. https://doi.org/10.1016/j.jspi.2020.04.005. MR4096268
[26] 
Schork, N. J. (2015). Personalized medicine: time for one-person trials. Nature News 520(7549) 609.
[27] 
Silverstein, J. W. et al.(1985). The smallest eigenvalue of a large dimensional Wishart matrix. The Annals of Probability 13(4) 1364–1368. MR0806232
[28] 
Simon, S. J. and Peppas, S. C. (2005). Attitudes towards product website design: a study of the effects of gender. Journal of Marketing Communications 11(2) 129–144.
[29] 
Siroker, D. and Koomen, P. (2013) A/B testing: The most powerful way to turn clicks into customers. John Wiley & Sons.
[30] 
Wang, Y. and Ai, M. (2016). Optimal designs for multiple treatments with unequal variances. Journal of Statistical Planning and Inference 171 175–183. https://doi.org/10.1016/j.jspi.2015.10.005. MR3458076
[31] 
Zhang, Q., Khademi, A. and Song, Y. (2022). Min-max optimal design of two-armed trials with side information. INFORMS Journal on Computing 34(1) 165–182. https://doi.org/10.1287/ijoc.2021.1068. MR4401464

Full article PDF XML
Full article PDF XML

Copyright
© 2023 New England Statistical Society
by logo by logo
Open access article under the CC BY license.

Keywords
E-optimal design Multiple treatments allocation Semi-definite program

Funding
Khademi is supported by NSF Award 1651912.

Metrics (since February 2017)
106

Article info
views

21

Full article
views

25

PDF
downloads

11

XML
downloads

Export citation

Copy and paste formatted citation
Placeholder

Download citation in file


Share


RSS

The New England Journal of Statistics in Data Science

  • ISSN: 2693-7166
  • Copyright © 2021 New England Statistical Society

About

  • About journal

For contributors

  • Submit
  • OA Policy
  • Become a Peer-reviewer
Powered by PubliMill  •  Privacy policy