1. Home
  2. Issues
  3. Volume 1, Issue 3 (2023)
  4. Bayesian D-Optimal Design of Experiments ...

The New England Journal of Statistics in Data Science


Bayesian D-Optimal Design of Experiments with Quantitative and Qualitative Responses
Volume 1, Issue 3 (2023), pp. 371–385
Pub. online: 21 April 2023      Type: Methodology Article      Open accessOpen Access
Area: Statistical Methodology
Accepted
17 April 2023
Published
21 April 2023

Abstract

Systems with both quantitative and qualitative responses are widely encountered in many applications. Design of experiment methods are needed when experiments are conducted to study such systems. Classic experimental design methods are unsuitable here because they often focus on one type of response. In this paper, we develop a Bayesian D-optimal design method for experiments with one continuous and one binary response. Both noninformative and conjugate informative prior distributions on the unknown parameters are considered. The proposed design criterion has meaningful interpretations regarding the D-optimality for the models for both types of responses. An efficient point-exchange search algorithm is developed to construct the local D-optimal designs for given parameter values. Global D-optimal designs are obtained by accumulating the frequencies of the design points in local D-optimal designs, where the parameters are sampled from the prior distributions. The performances of the proposed methods are evaluated through two examples.

Supplementary material

 Supplementary Material
The supplementary material contains the proofs and derivations for equations (3.3), Theorem 1, Proposition 1 and 2, Theorem 2, the shortcut formulas, update formulas, and the Δ ( x , x i ) function in Section 5.1. The supplement material also includes the table of five different designs for the artificial example in Section 6.1. The codes and data for all the algorithms and examples are available from https://github.com/lulukang/BayesianQQDoE.git.

References

[1] 
Breiman, L. and Friedman, J. H. (1997). Predicting multivariate responses in multiple linear regression. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 59(1) 3–54. https://doi.org/10.1111/1467-9868.00054. MR1436554
[2] 
Carnell, R. (2012). lhs: Latin Hypercube Samples. R package version 0.10. http://CRAN.R-project.org/package=lhs.
[3] 
Chaloner, K. and Verdinelli, I. (1995). Bayesian experimental design: A review. Statistical Science 10(3) 273–304. MR1390519
[4] 
Chen, M. H. and Ibrahim, J. G. (2003). Conjugate priors for generalized linear models. Statistica Sinica 13(2) 461–476. MR1977737
[5] 
Cook, R. D. and Nachtrheim, C. J. (1980). A comparison of algorithms for constructing exact D-optimal designs. Technometrics 22(3) 315–324.
[6] 
Cox, D. R. and Wermuth, N. (1992). Response models for mixed binary and quantitative variables. Biometrika 79(3) 441–461. https://doi.org/10.1093/biomet/79.3.441. MR1187603
[7] 
Deng, X. and Jin, R. (2015). QQ Models: Joint Modeling for Quantitative and Qualitative Quality Responses in Manufacturing Systems. Technometrics 57(3) 320–331. https://doi.org/10.1080/00401706.2015.1029079. MR3384947
[8] 
Denman, N., McGree, J., Eccleston, J. and Duffull, S. B. (2011). Design of experiments for bivariate binary responses modelled by Copula functions. Computational Statistics & Data Analysis 55(4) 1509–1520. https://doi.org/10.1016/j.csda.2010.07.025. MR2748657
[9] 
Dette, H., Pepelyshev, A., Zhigljavsky, A. et al. (2013). Optimal design for linear models with correlated observations. The Annals of Statistics 41(1) 143–176. https://doi.org/10.1214/12-AOS1079. MR3059413
[10] 
Diaconis, P. and Ylvisaker, D. (1979). Conjugate priors for exponential families. The Annals of Statistics 7(2) 269–281. MR0520238
[11] 
Draper, N. R. and Hunter, W. G. (1966). Design of experiments for parameter estimation in multiresponse situations. Biometrika 53(3-4) 525–533. https://doi.org/10.1093/biomet/53.3-4.525. MR0214240
[12] 
Dror, H. A. and Steinberg, D. M. (2005). Approximate Local D-optimal Experimental Design for Binary Response. Technical Report No. RP SOR-0501, Tel Aviv University.
[13] 
Dror, H. A. and Steinberg, D. M. (2006). Robust experimental design for multivariate generalized linear models. Technometrics 48(4) 520–529. https://doi.org/10.1198/004017006000000318. MR2328621
[14] 
Fang, K. -T., Lin, D. K., Winker, P. and Zhang, Y. (2000). Uniform design: theory and application. Technometrics 42(3) 237–248. https://doi.org/10.2307/1271079. MR1801031
[15] 
Fang, X. and Hedayat, A. (2008). Locally D-optimal designs based on a class of composed models resulted from blending Emax and one-compartment models. The Annals of Statistics 36 428–444. https://doi.org/10.1214/009053607000000776. MR2387978
[16] 
Fedorov, V. V. (1972). Theory of Optimal Experiments 12. Academic Press, New York. MR0403103
[17] 
Girard, P. and Parent, E. (2001). Bayesian analysis of autocorrelated ordered categorical data for industrial quality monitoring. Technometrics 43(2) 180–191. https://doi.org/10.1198/004017001750386297. MR1954138
[18] 
Hainy, M., Müller, W. G. and Wynn, H. P. (2013). Approximate Bayesian computation design (ABCD), an introduction. In mODa 10–Advances in Model-Oriented Design and Analysis 135–143. Springer, Switzerland. https://doi.org/10.3390/e16084353. MR3255991
[19] 
Herzberg, A. M. and Andrews, D. F. (1976). Some considerations in the optimal design of experiments in non-optimal situations. Journal of the Royal Statistical Society. Series B (Methodological) 38(3) 284–289. MR0654007
[20] 
Joseph, V. R. (2006). A Bayesian approach to the design and analysis of fractionated experiments. Technometrics 48(2) 219–229. https://doi.org/10.1198/004017005000000652. MR2277676
[21] 
Joseph, V. R. and Hung, Y. (2008). Orthogonal-maximin Latin hypercube designs. Statistica Sinica 18(1) 171–186. MR2416907
[22] 
Kang, L. (2014). Improved Coordiante-Exchange Algorithm for Constrained Optimal Design. Revision submitted to Technometrics.
[23] 
Kang, L. and Huang, X. (2019). Bayesian A-Optimal Design of Experiment with Quantitative and Qualitative Responses. Journal of Statistical Theory and Practice 13(4) 64. https://doi.org/10.1007/s42519-019-0063-6. MR4021858
[24] 
Kang, L. and Joseph, V. R. (2009). Bayesian optimal single arrays for robust parameter design. Technometrics 51(3) 250–261. https://doi.org/10.1198/tech.2009.08057. MR2751071
[25] 
Kang, L., Kang, X., Deng, X. and Jin, R. (2018). A Bayesian hierarchical model for quantitative and qualitative responses. Journal of Quality Technology 50(3) 290–308. https://doi.org/10.1080/00224065.2018.1489042.
[26] 
Kang, X., Ranganathan, S., Kang, L., Gohlke, J. and Deng, X. (2021). Bayesian auxiliary variable model for birth records data with qualitative and quantitative responses. Journal of Statistical Computation and Simulation 91(16) 3283–3303. https://doi.org/10.1080/00949655.2021.1926459. MR4332587
[27] 
Kang, X., Kang, L., Chen, W. and Deng, X. (2022). A generative approach to modeling data with quantitative and qualitative responses. Journal of Multivariate Analysis 190 104952. https://doi.org/10.1016/j.jmva.2022.104952. MR4379329
[28] 
Khuri, A. I., Mukherjee, B., Sinha, B. K. and Ghosh, M. (2006). Design issues for generalized linear models: A review. Statistical Science 21(3) 376–399. https://doi.org/10.1214/088342306000000105. MR2339137
[29] 
Li, W. W. and Wu, C. F. J. (1997). Columnwise-Pairwise Algorithms with Applications to the Construction of Supersaturated Designs. Technometrics 39(2) 171–179. https://doi.org/10.2307/1270905. MR1452345
[30] 
Li, X., Sudarsanam, N. and Frey, D. D. (2006). Regularities in data from factorial experiments. Complexity 11(5) 32–45.
[31] 
Lin, C. D., Bingham, D., Sitter, R. R. and Tang, B. (2010). A new and flexible method for constructing designs for computer experiments. The Annals of Statistics 38 1460–1477. https://doi.org/10.1214/09-AOS757. MR2662349
[32] 
McCullagh, P. (1980). Regression models for ordinal data. Journal of the Royal Statistical Society. Series B (Methodological) 42 109–142. MR0583347
[33] 
Meyer, R. K. and Nachtsheim, C. J. (1995). The Coordinate-Exchange Algorithm for Constructing Exact Optimal Experimental Designs. Technometrics 37(1) 60–69. https://doi.org/10.2307/1269153. MR1322047
[34] 
Morris, M. D. and Mitchell, T. J. (1995). Exploratory designs for computational experiments. Journal of Statistical Planning and Inference 43(3) 381–402.
[35] 
Nguyen, N. -K. and Miller, A. J. (1992). A review of some exchange algorithms for constructing discrete D-optimal designs. Computational Statistics and Data Analysis 14(4) 489–498. https://doi.org/10.1016/0167-9473(92)90064-M. MR1192218
[36] 
Ning, X. and Tsung, F. (2012). A density-based statistical process control scheme for high-dimensional and mixed-type observations. IIE transactions 44(4) 301–311.
[37] 
Qian, P. Z. (2012). Sliced Latin hypercube designs. Journal of the American Statistical Association 107(497) 393–399. https://doi.org/10.1080/01621459.2011.644132. MR2949368
[38] 
Russell, K. G., Woods, D. C., Lewis, S. and Eccleston, J. (2009). D-optimal designs for Poisson regression models. Statistica Sinica 19 721–730. MR2514184
[39] 
Silvey, S. (1980). Optimal designs: an introduction to the theory for parameter estimation. London: Chapman and Hall. MR0606742
[40] 
Sitter, R. R. and Torsney, B. (1995). Optimal designs for binary response experiments with two design variables. Statistica Sinica 5 405–419. MR1347597
[41] 
Wheeler, B. (2014). AlgDesign: Algorithmic Experimental Design. R package version 1.1-7.2. http://CRAN.R-project.org/package=AlgDesign.
[42] 
Woods, D. C. and Van de Ven, P. (2012). Blocked designs for experiments with correlated non-normal response. Technometrics 53(2) 173–182. https://doi.org/10.1198/TECH.2011.09197. MR2808343
[43] 
Woods, D. (2010). Robust designs for binary data: applications of simulated annealing. Journal of Statistical Computation and Simulation 80(1) 29–41. https://doi.org/10.1080/00949650802445367. MR2757033
[44] 
Woods, D., Lewis, S., Eccleston, J. and Russell, K. (2006). Designs for generalized linear models with several variables and model uncertainty. Technometrics 48(2) 284–292. https://doi.org/10.1198/004017005000000571. MR2277681
[45] 
Wu, C. F. J. (1985). Efficient Sequential Designs With Binary Data. Journal of the American Statistical Association 80(392) 974–984. MR0819603
[46] 
Wu, C. J. and Hamada, M. S. (2011). Experiments: planning, analysis, and optimization 552. John Wiley & Sons, New Jersey. MR2583259
[47] 
Yang, E., Baker, Y., Ravikumar, P., Allen, G. and Liu, Z. (2014). Mixed Graphical Models via Exponential Families. In Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics 1042–1050.
[48] 
Yang, M. and Stufken, J. (2009). Support points of locally optimal designs for nonlinear models with two parameters. The Annals of Statistics 37(1) 518–541. https://doi.org/10.1214/07-AOS560. MR2488361
[49] 
Yang, M., Biedermann, S. and Tang, E. (2013). On optimal designs for nonlinear models: a general and efficient algorithm. Journal of the American Statistical Association 108(504) 1411–1420. https://doi.org/10.1080/01621459.2013.806268. MR3174717

Full article PDF XML
Full article PDF XML

Copyright
© 2023 New England Statistical Society
Open access article under the CC BY license.

Keywords
Bayesian D-optimal design conjugate prior generalized linear model multivariate responses noninformative prior point-exchange

Funding
The authors were partly supported by U.S. National Science Foundation for this research project. Dr. Lulu Kang was supported by grants CMMI-1435902, DMS-1916467, and DMS-2153029, Dr. Xinwei Deng by CMMI-1233571 and CMMI-1435996, and Dr. Ran Jin by CMMI-1435996.

Metrics
since December 2021
487

Article info
views

142

Full article
views

271

PDF
downloads

76

XML
downloads


Share


RSS