The New England Journal of Statistics in Data Science

We are pleased to introduce this special issue dedicated to “Design and Analysis of Experiments for Data Science”. The statistical methodology for design and analysis of experiments has played a pivotal role in various scientific fields for over a century. It guides researchers to make valid inference from experiments with optimized resource allocation. This enduring discipline remains a vibrant and dynamic field of modern research. This collection of papers present cutting-edge research and innovative methodologies related to experimental designs. They provide valuable insights to address challenges of modern experimental designs and demonstrate the pivotal role of experimental designs in various domains, from clinical trials to online experimentation and beyond, in the increasingly data-driven world.

Chen et al. [2] showcase how Particle Swarm Optimization (PSO) can efficiently find optimal designs for longitudinal studies with diverse correlation structures and different models. Longitudinal studies present a unique set of challenges when it comes to experimental designs. The application of PSO in this context is a game-changer. The potential applications are far-reaching, from the Michaelis-Menten model to growth curve studies and HIV dynamic modeling. The optimization provided by PSO opens up new avenues for the scientific community.

Clinical trialists often grapple with the need to balance statistical power, ethical considerations, and control over the type I error rate. Zhu et al. [11] propose an adaptive seamless design (ASD) in conjunction with response adaptive randomization (RAR) to address these challenges. This innovative approach demonstrates how it is possible to achieve efficient and ethical objectives while maintaining control over the type I error rate, a crucial aspect in clinical trials.

The task of discovering significant factors in experiments with a large number of variables and limited observations is a common problem in data science. Qi and Chien [7] introduce a new approach to construct supersaturated designs with low coherence, enhancing their usability for variable selection, particularly in methods like the Lasso. The real-world examples provided illustrate the practical value of this approach.

Hyperparameter tuning is a key factor in the success of deep learning models. However, the computational expense involved can be prohibitive. Shi et al. [9] delve into the exploration of hyperparameters and the collection of informative data for deep learning techniques. The findings demonstrate the superiority of the technique of strong orthogonal array in efficiently collecting data to improve the test accuracy with deep learning, a critical measure of learning performance.

Bui et al. [1] propose a general additive network effect (GANE) model. Innovative experiments are essential for businesses, and social network companies are at the forefront of testing new ideas and product changes. The unique challenges these experiments pose require specialized approaches. The introduction of the GANE model provides a comprehensive framework for understanding treatment effects and network influence. The proposed power-degree specification showcases how specialized experiments can yield precise results, even in the face of model misspecification.

Crossover and interference models have diverse applications, but their complex nature has limited their practical utility. Hao et al. [3] present an algorithm designed to efficiently generate crossover designs under various conditions. The inclusion of a user-friendly interface and an R package broadens its accessibility and usability in a wide range of applications.

Systems with both quantitative and qualitative responses require specialized experimental designs. Kang et al. [4] introduce a Bayesian D-optimal design method that caters to such systems, providing an efficient approach to constructing both local and global D-optimal designs. The inclusion of prior distributions and a point-exchange search algorithm allows for meaningful interpretations and practical application in various contexts.

Personalized decision-making in controlled experiments is of growing interest, particularly in clinical trials and user behavior studies. Li et al. [5] tackle the challenge of optimizing treatment allocation while considering observational covariates. The proposed method seeks to maximize the variance of personalized treatment effects, enhancing precision in decision-making processes. Numerical studies validate the method’s quality and applicability.

Pronzato and Rendas [6] address integrated squared error (ISE) estimation for predicting unknown functions. They calculate ISE estimators as weighted averages of predictor residuals at selected points. Their study shows that minimizing mean squared error of ISE estimators is equivalent to minimizing maximum mean discrepancy (MMD). Sequential Bayesian quadrature is utilized to create nested validation designs that minimize MMD at each step. The optimal ISE estimate is expressed as the integral of a linear reconstruction of interpolator residual squares over the function’s domain. The validation designs maintain a space-filling property. Numerical experiments validate the method’s strong performance and robustness.

Online experimentation frequently encounters incomplete metric data, making imputation an essential step for analysis. Shen et al. [8] introduce a clustering-based imputation method, considering both experiment-specific features and user activities, to improve the analysis of online experiments. This research lays the foundation for more efficient imputation of large-scale data in online experimentation, benefiting both simulations and real-world experiments.

The explosion of big data requires efficient subdata selection methods to perform inferences while managing computational costs. Singh and Stufken [10] present the information-based optimal subdata selection method for selecting subdata with strong statistical properties in linear regression models. The idea of combining lasso and subdata selection extends the capabilities of subdata selection, offering an efficient solution for handling large datasets with many variables.

We hope that this special issue, featuring eleven diverse and innovative research papers, will inspire further exploration and innovation in the field of experimental designs for data science. The methodologies and insights presented in these papers have the potential to transform how experiments are conducted and analyzed in our data-driven world. We invite you to delve into these papers to discover the latest advancements and ideas on experimental designs in data science.