Optimal Design of Controlled Experiments for Personalized Decision Making in the Presence of Observational Covariates

Li, Yezhuo; Zhang, Qiong; Khademi, Amin; Yang, Boshi

doi:10.51387/23-NEJSDS22

The New England Journal of Statistics in Data Science

Optimal Design of Controlled Experiments for Personalized Decision Making in the Presence of Observational Covariates

Volume 1, Issue 3 (2023), pp. 386–393

Yezhuo Li Qiong Zhang Amin Khademi All authors (4)

https://doi.org/10.51387/23-NEJSDS22

Pub. online: 26 January 2023 Type: Statistical Methodology

Open Access

Accepted
12 January 2023

Published
26 January 2023

Abstract

Controlled experiments are widely applied in many areas such as clinical trials or user behavior studies in IT companies. Recently, it is popular to study experimental design problems to facilitate personalized decision making. In this paper, we investigate the problem of optimal design of multiple treatment allocation for personalized decision making in the presence of observational covariates associated with experimental units (often, patients or users). We assume that the response of a subject assigned to a treatment follows a linear model which includes the interaction between covariates and treatments to facilitate precision decision making. We define the optimal objective as the maximum variance of estimated personalized treatment effects over different treatments and different covariates values. The optimal design is obtained by minimizing this objective. Under a semi-definite program reformulation of the original optimization problem, we use a YALMIP and MOSEK based optimization solver to provide the optimal design. Numerical studies are provided to assess the quality of the optimal design.

References

[1]

ApS, M. (2019). The MOSEK optimization toolbox for MATLAB manual. Version 9.0. http://docs.mosek.com/9.0/toolbox/index.html.

[2]

Atkinson, A. (2015). Optimum designs for two treatments with unequal variances in the presence of covariates. Biometrika 102(2) 494–499. https://doi.org/10.1093/biomet/asu071. MR3371019

[3]

Ben-Tal, A. and Nemirovski, A. (2001) Lectures on modern convex optimization: analysis, algorithms, and engineering applications. SIAM. https://doi.org/10.1137/1.9780898718829. MR1857264

[4]

Berry, D. A., Mueller, P., Grieve, A. P., Smith, M., Parke, T., Blazek, R., Mitchard, N. and Krams, M. (2002). Adaptive Bayesian designs for dose-ranging drug trials. In Case studies in Bayesian statistics 99–181 Springer. https://doi.org/10.1007/978-1-4613-0035-9_2. MR1931867

[5]

Bhat, N., Farias, V. F., Moallemi, C. C. and Sinha, D. (2020). Near-Optimal A-B Testing. Management Science 66(10) 4477–4495.

[6]

Biswas, A. and López–Fidalgo, J. (2013). Compound designs for dose-finding in the presence of nondesignable covariates. Pharmaceutical Statistics 12(2) 92–101.

[7]

Blackwelder, W. C. (1982). “Proving the null hypothesis” in clinical trials. Controlled clinical trials 3(4) 345–353.

[8]

Boyd, S. and Vandenberghe, L. (2004) Convex optimization. Cambridge university press. https://doi.org/10.1017/CBO9780511804441. MR2061575

[9]

Dette, H. and Studden, W. J. (1993). Geometry of E-optimality. The Annals of Statistics 21(1) 416–433. https://doi.org/10.1214/aos/1176349034. MR1212185

[10]

Edelman, A. (1988). Eigenvalues and condition numbers of random matrices. SIAM journal on matrix analysis and applications 9(4) 543–560. https://doi.org/10.1137/0609045. MR0964668

[11]

Edelman, A. (1991). The distribution and moments of the smallest eigenvalue of a random matrix of Wishart type. Linear algebra and its applications 159 55–80. https://doi.org/10.1016/0024-3795(91)90076-9. MR1133335

[12]

Feit, E. M. and Berman, R. (2019). Test & Roll: Profit-Maximizing A/B Tests. Marketing Science 38(6) 1038–1058.

[13]

Gally, T., Pfetsch, M. E. and Ulbrich, S. (2018). A framework for solving mixed-integer semidefinite programs. Optimization Methods and Software 33(3) 594–632. https://doi.org/10.1080/10556788.2017.1322081. MR3783675

[14]

Goemans, M. X. and Williamson, D. P. (1995). Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. Journal of the ACM (JACM) 42(6) 1115–1145. https://doi.org/10.1145/227683.227684. MR1412228

[15]

Harville, D. A. (1974). Nearly optimal allocation of experimental units using observed covariate values. Technometrics 16(4) 589–599. https://doi.org/10.2307/1267612. MR0418353

[16]

Hore, S., Dewanji, A. and Chatterjee, A. (2014). Design issues related to allocation of experimental units with known covariates into two treatment groups. Journal of Statistical Planning and Inference 155 117–126. https://doi.org/10.1016/j.jspi.2014.06.002. MR3264459

[17]

Kohavi, R. and Longbotham, R. (2017). Online Controlled Experiments and A/B Testing. Encyclopedia of machine learning and data mining 7(8) 922–929.

[18]

Koning, R., Hasan, S. and Chatterji, A. (2022). Experimentation and start-up performance: Evidence from A/B testing. Management Science.

[19]

Lee, K. M. and Wason, J. (2019). Design of experiments for a confirmatory trial of precision medicine. Journal of Statistical Planning and Inference 199 179–187. https://doi.org/10.1016/j.jspi.2018.06.004. MR3857821

[20]

Löfberg, J. (2004). YALMIP: A Toolbox for Modeling and Optimization in MATLAB. In In Proceedings of the CACSD Conference.

[21]

MATLAB (2021) 9.10.0.1710957 (R2021a) Update 4. The MathWorks Inc., Natick, Massachusetts.

[22]

McFarland, C. (2012) Experiment!: Website conversion rate optimization with A/B and multivariate testing. New Riders.

[23]

Merikoski, J. K. and Virtanen, A. (1997). Bounds for eigenvalues using the trace and determinant. Linear algebra and its applications 264 101–108. https://doi.org/10.1016/S0024-3795(97)00067-0. MR1465858

[24]

Pukelsheim, F. (2006) Optimal design of experiments. SIAM. https://doi.org/10.1137/1.9780898719109. MR2224698

[25]

Rosa, S. (2020). Optimal experimental designs for treatment contrasts in heteroscedastic models with covariates. Journal of Statistical Planning and Inference 209 280–291. https://doi.org/10.1016/j.jspi.2020.04.005. MR4096268

[26]

Schork, N. J. (2015). Personalized medicine: time for one-person trials. Nature News 520(7549) 609.

[27]

Silverstein, J. W. et al. (1985). The smallest eigenvalue of a large dimensional Wishart matrix. The Annals of Probability 13(4) 1364–1368. MR0806232

[28]

Simon, S. J. and Peppas, S. C. (2005). Attitudes towards product website design: a study of the effects of gender. Journal of Marketing Communications 11(2) 129–144.

[29]

Siroker, D. and Koomen, P. (2013) A/B testing: The most powerful way to turn clicks into customers. John Wiley & Sons.

[30]

Wang, Y. and Ai, M. (2016). Optimal designs for multiple treatments with unequal variances. Journal of Statistical Planning and Inference 171 175–183. https://doi.org/10.1016/j.jspi.2015.10.005. MR3458076

[31]

Zhang, Q., Khademi, A. and Song, Y. (2022). Min-max optimal design of two-armed trials with side information. INFORMS Journal on Computing 34(1) 165–182. https://doi.org/10.1287/ijoc.2021.1068. MR4401464

Full article

Open access article under the CC BY license.

Keywords

E-optimal design Multiple treatments allocation Semi-definite program

Funding

Khademi is supported by NSF Award 1651912.

Metrics

since December 2021

456 Article info views	148 Full article views
179 PDF downloads	87 XML downloads

RSS

Authors

Abstract

References

Export citation

Copy and paste formatted citation

Download citation in file