NEJSDS The New England Journal of Statistics in Data Science 2693-71662693-7166 New England Statistical Society NEJSDS41 10.51387/23-NEJSDS41 Methodology Article Statistical Methodology Algorithm-Based Optimal and Efficient Exact Experimental Designs for Crossover and Interference Models HaoShuaishuai.hao@abbvie.com∗ YangMinmyang2@uic.edu ZhengWeiwzheng9@utk.edu AbbVie Inc, USA. E-mail address: shuai.hao@abbvie.com Department of Mathematics, Statistics, and Computer Science, University of Illinois at Chicago, USA. E-mail address: myang2@uic.edu Department of Business Analytics and Statistics, University of Tennessee-Knoxville, USA. E-mail address: wzheng9@utk.edu Corresponding author. 2023266202313361370962023 © 2023 New England Statistical Society2023 Open access article under the CC BY license.

The crossover models and interference models are frequently used in clinical trials, agriculture studies, social studies, etc. While some theoretical optimality results are available, it is still challenging to apply these results in practice. The available theoretical results, due to the complexity of exact optimal designs, typically require some specific combinations of the number of treatments (t), periods (p), and subjects (n). A more flexible method is to build integer programming based on theories in approximate design theory, which can handle general cases of ( t , p , n ). Nonetheless, those results are generally derived for specific models or design problems and new efforts are needed for new problems. These obstacles make the application of the theoretical results rather difficult. Here we propose a new algorithm, a revision of the optimal weight exchange algorithm by [1]. It provides efficient crossover designs quickly under various situations, for different optimality criteria, different parameters of interest, different configurations of ( t , p , n ), as well as arbitrary dropout scenarios. To facilitate the usage of our algorithm, the corresponding R package and an R Shiny app as a more user-friendly interface has been developed.

 Crossover Dropout Proportional Interference Min Yang was supported by NSF Grant DMS-22-10546. Introduction

Experimental design plays a fundamental role in scientific research. An optimal design thoughtfully organizes limited experimental resources to ensure reproducible and trustworthy outcomes. This paper focuses on searching for optimal designs for crossover and interference models using a general numerical algorithm proposed by [1].

In a typical experiment with a crossover design, there are n subjects, t treatments, p periods, and m ( ≤ n ) unique sequences s i of length p. Here, n, t, p, m are positive integers and we defer the mathematical rigorous definition of the experimental design to a later section. In the experiment, each of the subjects is assigned to a sequence and will receive treatments in a specific order defined in that sequence. Since subjects receive multiple treatments and have data collected throughout all periods, they serve as their own control and typically yield more data compared to other designs with the same number of subjects. Hence, crossover designs are usually more statistically efficient.

However, the limitations are noteworthy as well. First, it is usually unknown if the effects of previous treatments still exist in the following periods. These effects are often referred to as “carryover effects”. It is sensible to use a model that accounts for the carryover effects. There are variants of crossover models, which differ in the way of modeling carryover effects. Section 2 provides details of two of them, and we refer to [2] for an extended list of models. Second, the competing advantage of crossover designs is contingent on no or minimum dropouts. However in reality, as [3] pointed out “Experience suggests that a dropout rate of between 5% and 10% is not uncommon and, in some areas, can be as high as 25%”. While remedies for missing values due to dropouts have been established, the dropout issue is so common that it has to be prudently considered at the design stage. Related work in this direction includes [3, 4, 5, 6, 7, 8], with a focus on preserving the symmetric structure of the design in the presence of subject dropout. Recently in [9], Kushner’s type of linear equations (see [10]) are developed as necessary and sufficient conditions for a design being universal ϕ 1 optimal where ϕ 1 is defined as a new surrogate objective function considering dropout. It was later extended to unified results that apply to any configurations of the experiment. Optimal or efficient designs are obtained by either solving linear equations or integer optimization problems modified from equation systems.

The interference model is commonly employed in agricultural studies to mitigate the systematic bias caused by neighbor effects within blocks. A design for interference model consists of n subjects, t treatments, and m ( ≤ n ) distinct blocks s i of size k. Optimal designs can be found in [11, 12, 13], and other sources.

The literature contains abundant theoretical work on optimal exact designs for crossover and interference models. However, despite the elegance of the theorems, they are in general not convenient for use. The optimal or efficient designs are usually derived for a certain combination of ( n , t , p ) or ( n , t , k ). In most cases, experimenters need to search for the theory and construct the design themselves. In addition, the relevant theory may not even exist for certain experiment configurations, and this presents a major obstacle to utilizing optimal or efficient designs. Therefore, a general and efficient algorithm easily implemented via a computer program is necessary.

Numerical algorithms have been developed for decades, and it aims at solving the optimal design problem from a general perspective. Existing algorithms are typically modifications of either Fedorov-Wynn algorithm (FWA, [14], [15]), or the multiplicative algorithm (MA, [16]). For more information, refer to [17, 18, 19, 20, 21, 22]. [23] combined multiple existing designs with modifications, named “cocktail algorithm”, and achieved dramatic speed improvement. However, all the aforementioned algorithms only focus on D-optimal designs whereas the different objective of experiments requires properly chosen optimality criterion. [1] proposed an optimal weights exchange algorithm (OWEA) for nonlinear models. It updates the support points in the same way as FWA and optimizes weights via the Newton’s method. This is the algorithm we will use.

Nevertheless, the development of an efficient algorithm for crossover designs is faced with two main obstacles. Firstly, it is worth noting that unlike general statistical models where the number of parameters is fixed, the number of parameters in a crossover design varies depending on the number of subjects. This variability extends to the dimension of the information matrix, presenting a significant challenge for algorithm-based optimization. Secondly, constructing optimal crossover designs in the presence of possible subject dropout mechanisms is challenging, considering the variety of crossover design formats. In this paper, we aim to address these challenges.

Specifically, through the Schur complement operation, we derived a novel information matrix that is linear in the design points, with corresponding weights serving as linear coefficients, paving the way for the successful implementation of the OWEA algorithm. Based on [1], we developed a general and efficient algorithm framework for optimal designs of crossover models and later extended it to interference models. Our algorithm is capable of handling any configuration of designs for the two models, including possible subject dropout mechanisms. The designs generated by our approach are highly efficient compared to existing designs in the literature. Additionally, to promote wider usage of our algorithm, we have developed a corresponding R package and R Shiny app, which provides a more user-friendly interface.

The remaining sections of this paper are organized as follows. Section 2 briefly introduces the necessary notations, statistical models and the derivations of the information matrices to fit into the framework of the algorithm. Section 3 gives details of the algorithm and the developed R-package. Numerical results are presented in section 4, and a short discussion is in section 5.

 The Framework and Notations

In this section, we will introduce the general framework starting with basic concepts of optimal designs followed by demonstrations of crossover models and interference models as well as their information matrix. We will show how the information matrices, utilizing Schur complement, fit into the algorithmic framework in [1].

Optimal Designs and Objective Functions

Despite the various types of designs, generally, an experimental design with n runs can be written as d = { ( s i , n i ) | s i ∈ χ , ∑ i = 1 m n i = n , n i > 0 } , where ( s i , n i ) is the pair of the design point and its associated repetition. The s i is a vector of length p ( ≥ 1 ). The design space, which is a collection of the design points, is represented by χ. The m is the number of distinct design points. If all of the repetitions ( n 1 , … , n m ) are required to be positive integers, then the design (2.1) is called exact design and m ≤ n.

To derive an optimal design is to find a set of pairs ( s i , n i ) that optimize an objective function. Due to the discrete nature of the exact design, integer solutions of n i are generally intractable. In that case, one may drop the integer requirement and search for an approximate design: ξ = { ( s i , w i ) | s i ∈ χ , ∑ i = 1 m w i = 1 , w i ∈ ( 0 , 1 ] } . where w i is the weight associated to design point s i and all weights sum up to unity.

In after-experiment data analysis, usually a differentiable function of θ, say g ( θ ), is of interest. Notice that g must be an estimable function. In the rest of this paper, we always have this assumption. The choice of g grants the flexibility on parameter of interest. For example, when we target on all parameters, g is an identity function, i.e. g ( θ ) = θ; and when comparison is the goal, one of many options of g is g ( θ ) = ( θ 1 − θ ν , θ 2 − θ ν , … , θ ν − 1 − θ ν ) ), where ν is the length of θ. Denote by θ ˆ the maximum likelihood estimator (MLE) of θ, then it is well known that g ( θ ˆ ) is also the MLE of g ( θ ). Under mild assumptions, the Delta method yields the asymptotic covariance matrix of g ( θ ˆ ). With I ξ ( θ ) being the information matrix of θ under design ξ, C ξ ( g ) is the asymptotic variance-covariance matrix of g ( θ ˆ ) obtained using the Delta method under mild conditions, C ξ ( g ) = ∂ g ∂ θ ′ I ξ − ( θ ) ( ∂ g ∂ θ ′ ) ′ .

With the notations of (2.1) and (2.2), we follow the definition in [24] of the objective function, Φ r ( C ξ ( g ) ) = 1 v T r ( C ξ ( g ) ) r 1 / r , 0 ≤ r < ∞ , where r is a non-negative real number, v is the dimension of g, g is a differentiable function of parameter vector θ, and T r ( C ξ ( g ) ) is the trace of the matrix C ξ ( g ).

Note that Φ r ( C ξ ( g ) ) is equivalent to many prevailing optimality criteria on various values of r. For example, r = 0, Φ r ( C ξ ( g ) ) is lim r ↓ 0 1 v T r ( C ξ ( g ) ) r 1 / r = | C ξ ( g ) | 1 / v , which is D-optimality; when r = 1, we have Φ r ( C ξ ( g ) ) = T r ( C ξ ( g ) ) / v, and it is A-optimality.

For the purpose of verifying a design is optimal, the following theorem, adapted from Theorem 1 of [1], provides Kiefer’s type general equivalence theorem (GET) which serves as a sufficient and necessary condition.

Let g ( θ ) be an estimable function of θ. Suppose an arbitrary design ξ has an information matrix I ξ . ξ is Φ r optimal for g ( θ ) if and only if the directional derivative of Φ r , denoted by d r ( s , ξ ) satisfies d r ( s , ξ ) ≤ 0 , for any s ∈ χ, with equality holds if s belongs to the support of ξ.

In addition, the d r ( s , ξ ) can be calculated by:

when r = 0, d r ( s , ξ ) = T r [ ( C ξ ( g ) ) − 1 ( ∂ g ∂ θ ′ ) I ξ − ( I s − I ξ ) I ξ − ( ∂ g ∂ θ ′ ) ′ ] . when r > 1, d r ( s , ξ ) = ( 1 v ) 1 / r T r [ ( C ξ ( g ) ) ] 1 − r × T r [ ( C ξ ( g ) ) r − 1 ( ∂ g ∂ θ ′ ) I ξ − ( I s − I ξ ) I ξ − ( ∂ g ∂ θ ′ ) ′ ] .

Models and Information Matrix Crossover Model with Carryover Effect

Suppose a crossover design d is conducted with n subjects, t treatments and p periods, the observation from subject i at period j is typically modeled by y i j = μ + ζ i + π j + τ d ( i , j ) + γ d ( i , j − 1 ) + ϵ i j ; i = 1 , … , n ; j = 1 , … , p , where μ is the grand mean, ζ i is the effect from ith subject, π j stands for the effects from jth period, d ( i , j ) denotes the treatment assignment of jth period for ith subject from design d, τ d ( i , j ) is the treatment effect from d ( i , j ), and γ d ( i , j − 1 ) is the carryover effect due to treatment d ( i , j − 1 ) where γ d ( i , 0 ) is set to 0 by convention. ϵ i j is the error term.

If the collection of observations from the design d is organized in a vector Y d = ( y 11 , y 12 , … , y 1 p , y 21 , … , y n p ) ′ , then model (2.8) can be written in matrices, Y d = 1 n p μ + U ζ + Z π + T τ + R γ + ϵ , where ϵ is a vector of independent and identically distributed errors with a mean vector 0 and a variance-covariance matrix σ 2 I n p , which can be relaxed to a form of Kronecker’s product, I n ⊗ Σ, in most cases when the variance-covariance matrix of ( ϵ i 1 , … , ϵ i p ) is Σ for i = 1 , … , n. Besides, we also have π = ( π 1 , … , π p ) ′ , ζ = ( ζ 1 , … , ζ n ) ′ , τ = ( τ 1 , … , τ t ) ′ , γ = ( γ 1 , … , γ t ) ′ , U = I n ⊗ 1 p = ( U 1 ′ , … , U n ′ ) ′ , Z = 1 n ⊗ I p = ( Z 1 ′ , … , Z n ′ ) ′ , T = ( T 1 ′ , … , T n ′ ) ′ and R = ( R 1 ′ , … , R n ′ ) ′ . Here U i ’s are p × n incidence matrices for subjects. Z i is p × p subject-to-period incidence matrix, T i and R i are p × t period-to-treatment incidence matrices for the ith subject. The structure of matrices Z i , T i , and R i depend on the design d. I n is an identity matrix of dimension n and 1 p stands for a p × 1 vector of 1’s. Then the information matrix for full parameter vector Θ = ( ζ ′ , π ′ , τ ′ , γ ′ ) ′ is I ( Θ ) = U Z T R ′ U Z T R = ∑ i = 1 n U i Z i T i R i ′ U i Z i T i R i = ∑ i = 1 n I i = ∑ i = 1 m n i I i , where the positive integer m ( ≤ n ) represents the number of unique design point (sequence of treatments) of the design, and m = n holds when there is only one subject assigned to each of the unique design points. I i is the information matrix for the ith unique design point, and n i is the number of subjects assigned to that design point or alternatively the “repetition” of a design point. Additionally, ∑ i n i = n. For the remaining of this paper, unless otherwise specified, the term “design point” refers to “unique design point” in an experimental design.

An optimal exact design is then to choose a collection of sequences so that it optimizes an objective function related to (2.10). In this paper, we obtain optimal approximate design by a numerical algorithm and carefully round it to an exact design which is either efficient or optimal. However two issues regarding information matrix has to be settled prior to the implementation of the algorithm. First, the direct treatment is the target to estimate, and yet its information matrix is not additive with respect to design points. As a result, it does not fit for the prerequisite of the algorithm. On the other hand, although information matrix for full parameters is additive, its dimension is changing all the time because of the inclusion of subject effects ζ as well as the iterative nature of numerical algorithm. Optimizing on an information matrix of varying dimensions is problematic. Therefore, to fit the crossover design problem to our framework, we exclude the subject effect from (2.10) through the Schur complement operation. Excluding them and the general mean (noticing that the unity vector 1 n p belongs to the column space of U) and utilizing the Schur complement of a matrix, we reach to the information matrix of the marginal parameter vector θ = ( π ′ , τ ′ , γ ′ ) ′ as I ( θ ) = Z T R ′ p r ⊥ ( U ) Z T R = Z T R ′ [ I n p − 1 p I n ⊗ 1 p 1 p ′ ] Z T R = ∑ i = 1 n Z i T i R i ′ [ I p − 1 p 1 p 1 p ′ ] Z i T i R i = ∑ i = 1 n I i = ∑ i = 1 m n i I i = n ∑ i = 1 m w i I i , where, n i = n w i , for i = 1 , 2 , … , m, and w i is called the “weight” associated to a design point. p r ⊥ ( · ) is an orthogonal projection operator. Notice that I i is the information matrix associated with the ith design point. For any matrix X, p r ⊥ ( X ) = I − X ( X ′ X ) − 1 X ′ . With such derivation, the information matrix is linear in the design points with corresponding weights as linear coefficients.

 Crossover Model with Dropout

To account for the dropouts, following [3, 9], define the dropout mechanism of a subject as ℓ = ( ℓ 1 , … , ℓ p ), where ℓ i is the probability that the longest time of stay is i, and ∑ i = 1 p ℓ i = 1 and assume

Subject’s dropout is independent of the choice of design as well as the outcome of the design.

Once a subject is dropped, the chance of returning is zero.

Subject’s dropout mechanisms are i.i.d.

 Given a realization of ℓ, the information matrix (2.12) becomes I ( θ , ℓ ) = ∑ i = 1 n Z i T i R i ′ [ M i − M i 1 p ( 1 p ′ M i 1 p ) − 1 1 p ′ M i ] ( Z i , T i , R i ) = ∑ i = 1 n I i ( ℓ ) = ∑ i = 1 m n i I i ( ℓ ) = n ∑ i = 1 m w i I i ( ℓ ) , where M i is an indicator matrix depending on the dropout mechanism ℓ, M i ( ℓ ) = I ( a i × a i ) O 1 O 1 ′ O p × p , I ( a i × a i ) is the a i × a i identity matrix, a i is the longest period of stay for subject i, and O and O 1 are zero matrices with proper dimensions.

The information matrix is random due to the modeling of the dropout mechanism. The intuitively reasonable objective function, E Φ p ( C ξ ( g ) ) (defined in (2.4)), is not easy to deal with, and Φ p ( E ( C ξ ( g ) ) ) is chosen as a feasible replacement. Here the operator E means expectation with respect to the dropout mechanism. Suppose we obtain an optimal approximate design ξ ∗ by minimizing Φ p ( E ( C ξ ( g ) ) ), and round it to exact design ξ 1 . The efficiency of ξ 1 should be defined as E Φ p ( C ξ o p t ( g ) ) / E Φ p ( C ξ 1 ( g ) ), where ξ o p t = a r g m i n ξ E Φ p ( C ξ ( g ) ). However, the efficiency is not accessible because ξ o p t is unknown. Nevertheless, one is still able to obtain the lower bound. (Zheng (2013a)).

For crossover designs under subject dropout, define the following approximate designs ξ o p t = a r g m i n ξ E Φ p ( C ξ ( g ) ) , ξ ∗ = a r g m i n ξ Φ p ( E ( C ξ ( g ) ) ) . For an arbitrary design ξ 1 , E Φ p ( C ξ o p t ( g ) ) E Φ p ( C ξ 1 ( g ) ) ≥ Φ p ( E ( C ξ o p t ( g ) ) ) E Φ p ( C ξ 1 ( g ) ) ≥ Φ p ( E ( C ξ ∗ ( g ) ) ) E Φ p ( C ξ 1 ( g ) ) ,

where the first ‘≥’ is due to the convexity of the function Φ p and Jensen’s inequality, and the second ‘≥’ is because of (2.15). Note that the problem in (2.16) fits the framework of (2.12) and hence could be solved by our algorithm. Therefore, the rightmost term, Φ p ( E ( C ξ ∗ ( g ) ) ) E Φ p ( C ξ 1 ( g ) ) , becomes evaluable for any given arbitrary design ξ 1 and dropout mechanism and will serve as the lower bound efficiency. Furthermore, [9] proposed to convert the results of (2.16) to an exact design and gauge its performance by the lower bound of efficiency. We are using the same approach except that our algorithm to derive the exact design is much faster.

 Crossover Model with Proportional Carryover Effects

Another variant of the classical crossover design model is the proportional model. For the design d with p periods and t treatments, individual outcomes are modeled as y i j = μ + ζ i + π j + τ d ( i , j ) + λ τ d ( i , j − 1 ) + ϵ i j ; i = 1 , … , n ; j = 1 , … , p , where the notations are exactly the same as in (2.8), except that the additional real-valued λ, which is the proportion that carryover effects accounting for direct effects. The proportional model can also be written in matrices, Y d = 1 n p μ + U ζ + Z π + T τ + λ R τ + ϵ , where ϵ is a vector of independent errors with mean 0 and variance covariance matrix I n ⊗ Σ, Σ is an p × p positive definite matrix. Utilizing Schur complement operations, the information matrix for the parameter vector θ = ( π ′ , τ ′ , λ ) ′ is I ( θ , λ , τ ) = ∑ i = 1 n ( Z i , T i + λ R i , R i τ ) ′ [ Σ − 1 − Σ − 1 1 p [ 1 p ′ Σ − 1 1 p ] − 1 p ′ Σ − 1 ] ( Z i , T i + λ R i , R i τ ) = ∑ i = 1 n I i = ∑ i = 1 m n i I i = n ∑ i = 1 m w i I i . Therefore, the design problem for the proportional model fits into the framework of (2.12). Note that the proportional model is nonlinear due to the term λ τ, and its information matrix depends on unknown parameters τ and λ.

 Interference Model

The individual observations of the interference model with t treatments, n blocks of size k are written as y i j = μ + γ i + τ d ( i , j ) + λ d ( i , j − 1 ) + ρ d ( i , j + 1 ) + ϵ i j i = 1 , … , n ; j = 1 , … , k , where y i j denotes the response from jth plot of ith block, μ is general mean, d ( i , j ) stands for the treatment assignment of ith block and jth plot according to design d, and τ d ( i , j ) , λ d ( i , j − 1 ) , and ρ d ( i , j + 1 ) are treatment effects from treatment itself, its left plot and right plot. By convention, we set λ d ( i , 0 ) = ρ d ( i , p + 1 ) = 0. Again, the responses can be gathered in a vector and modeled in terms of matrices, Y d = 1 n k μ + U γ + T d τ + L d λ + R d ρ + ϵ , where Y d = ( y 11 , . . , y 1 k , y 21 , … , y n p ) ′ , γ = ( γ 1 , … , γ n ) ′ , τ = ( τ 1 , … , τ t ) ′ , λ = ( λ 1 , … , λ t ) ′ , ρ = ( ρ 1 , … , ρ t ) ′ , U = I n ⊗ 1 k , T d = ( T 1 ′ , … , T n ′ ) ′ , L d = ( L 1 ′ , … , L n ′ ) ′ , R d = ( R 1 ′ , … , R n ′ ) ′ . Here U i ’s are incidence matrices for block, T i and L i and R i are k × t plot-to-treatment incidence matrices for ith block that depends on the design d. The error vector ϵ is assumed to follow N ( 0 , I n ⊗ Σ ), where Σ is an arbitrary positive definite matrix of order k. Let θ = ( τ ′ , λ ′ , ρ ′ ) ′ be the vector of parameters of interest. Note that block effects are excluded due to similar reasons to that of the subject effect for the crossover model. The information matrix for θ, after applying Schur complement, is I ( θ ) = ∑ i = 1 n ( T i , L i , R i ) ′ [ Σ − 1 − Σ − 1 1 k ( 1 k ′ Σ − 1 1 k ) − 1 1 k ′ Σ − 1 ] ( T i , L i , R i ) = ∑ i = 1 n I i = ∑ i = 1 m n i I i = n ∑ i = 1 m w i I i .

At this point, we have fitted the design problem for the crossover model with dropout, the proportional model and the interference model into the framework of (2.12), to which our algorithm would apply. In the next section, we shall introduce the algorithm and the developed R-package.

 The Algorithm and R-package

Numerical algorithms are powerful and convenient tools for finding optimal designs. [1] proposed an optimal weights exchange algorithm (OWEA) for nonlinear models. It updates the support points in the same way as Fedorov-Wynn algorithm and optimizes weights via the Newton’s method. This is the algorithm we are going to use. The procedures of OWEA’s implementation is briefly introduced as follows.

Phase I: Finding optimal approximate designs

Start with initial support S ( 1 ) with ν + 1 randomly picked sequences and equal weights w ( 1 ) , at iteration t,

Input support S ( t ) , w ( t ) , and update weights using the Newton’s method. Note that support points with zero weights will be deleted in optimizing process.

Newton’s Method:

Input: Start with the initial input by setting S 1 ( t ) = S ( t ) , and w 1 ( t ) = w ( t ) . After j − 1 iterations, with the S j ( t ) and w j ( t ) , updating the weight by the following steps:

update the weights by the equation w j + 1 ( t ) = w j ( t ) − α ( ∂ 2 Φ ∂ w ∂ w ′ ) − 1 ∂ Φ ∂ w | w j ( t ) .

If w j + 1 ( t ) has negative component, go to step ( c ), otherwise proceed to step ( d ).

Set α to α / 2 and go back to step ( a ). If α < ϵ α , remove the point with smallest weight and go back to step ( a ).

Check if ∂ Φ ∂ w | w j + 1 ( t ) < ϵ 0 , if true, w ˜ ( t ) = w j + 1 ( t ) is the optimal weights otherwise go to next iteration.

 Output: Support S ˜ ( t ) and optimal weights w ˜ ( t ) , where S ˜ ( t ) = S ( t ) if no points are removed.

Derive s t ∗ = a r g m a x s ∈ χ d r ( s , ξ ( t ) ), where ξ ( t ) = { ( s i , w i ) | s i ∈ S ˜ ( t ) , w i ∈ w ˜ ( t ) , ∑ i w i = 1 }

Check d r ( s t ∗ , ξ ( t ) ) < ϵ d , where ϵ d is a pre-selected small positive value. If true, ξ ( t ) is the desired design. Otherwise, let S ( t + 1 ) = S ˜ ( t ) ⋃ { s t ∗ }, w ( t + 1 ) = w ˜ ( t ) ⋃ { 0 }, and go to step 1.

 Phase II: Rounding

t = p = 4, n = 16, a = (0,0,1/2,1/2).

Optimality Rel-Eff (to Gurobi) Rel-Eff (rounding) Efficiency (Lower Bound) Rel-Eff (to  d 2  in [25]) Time (sec) Gurobi Time (sec)
A 0.9994 0.9917 0.9366 0.9844 2.18 20.84
D 0.9979 0.9757 0.9519 0.9538 2.63 20.84

Suppose optimal design from Phase I is ξ ∗ = { ( s i , w i ∗ ) | s i ∈ S ∗ , w i ∈ ( 0 , 1 ) , ∑ i m w i = 1 }, for a given number of total runs, say N,

Compute N w i for i = 1 , …, and round them to their nearest integers. This results in a tentative exact design d = { ( s i , n i ) | s i ∈ S ∗ , ∑ i m n i = N }

For every s j ∈ Ω / S ∗ ,

For every unique s i ∈ S ∗ , subtract n i by 1 and append s j to the design with repetition 1, this results in a new design, say d ( − i , j ) .

Calculate two values V 1 = Φ ( d ) and V 2 = Φ ( d ( − i , j ) ), if V 1 > V 2, keep d ( − i , j ) as the current tentative design, otherwise, keep the tentative design unchanged.

Repeat these two steps for i = 1 , … , m;

After traversing all possible j’s, output the current design as efficient design d ∗ .

Note that tuning parameters ϵ d , ϵ α and ϵ 0 may affect the time for convergence. In all examples tested in this paper, the combination of ϵ d = 10 − 15 , ϵ α = 10 − 6 and ϵ 0 = 10 − 10 work well and optimal or highly efficient designs can be found within a relatively short period of time. Besides, it is known that the Newton’s method is sensitive to the choice of initial points and sometimes has overshooting issue. However, based on our experience, efficient designs are always found.

We have developed an R Package called OWEA to apply the algorithm to the three models as mentioned in the earlier section, namely (i) crossover design accounting for subject dropouts; ( i i) proportional model; ( i i i) interference model. It can be readily applicable to all these three optimal design problems. To implement it, a user only needs to input very basic information such as numbers of periods, treatments, and subject, and of course the dropout probability for the model in (i). The detailed implementation is available at https://cran.r-project.org/package=OWEA.

 Examples

In this section, utilizing the information matrices derived in section 2 and the OWEA algorithm in Section 3, exact designs are presented for three scenarios: crossover model with dropouts, crossover model with proportional carryover effects, and interference models. For convenience, we call the design obtained from OWEA algorithm the OWEA design, optimal design available from the literature the literature design. For example, the A- optimal design obtained from OWEA is “A-optimal OWEA design”. Moreover, for all the three scenarios, [9, 13, 25] developed linear equation systems as necessary and sufficient conditions for universally optimal designs and suggested searching for optimal designs using integer optimizer, like [26]. More specifically, the conditions are in the form of linear equations regarding the replication numbers of each treatment sequence. Since there is no guarantee of integer solutions, we transform the problem into a quadratic programming which minimizes the total square of loss for any equation that does not hold. In this section, we call designs derived such way as the Gurobi design.

Relative efficiency, shown as “Rel-Eff (to xx)”, is provided when the intention is to compare the OWEA designs to existing designs. In addition, the relative efficiency due to rounding from the approximate design to the exact design, shown as “Rel-Eff (rounding)”, is also presented.

All the algorithms and examples are programmed in R (R Core Team, 2022) [27] and are executed on an Apple Desktop with 8GB RAM. Computing times are recorded and provided.

Crossover with Subject Dropout

( t , p ) = ( 4 , 4 ), n = 16, ℓ = ( 0 , 0 , 1 / 2 , 1 / 2 ). Exact designs and their performance under A- and D- optimality are summarized in Table 1. It is evident that there is only a tiny loss in efficiency when rounding from approximate design to exact design, though the lower bound efficiencies are only 0.9366 and 0.9519 for the A- and D- criteria. Compared to d 2 from [9] which is proved to be highly efficient, the relative efficiency of the OWEA designs are 0.9844 and 0.9538 respectively. They are not very high but still satisfactory. The design obtained utilizing the integer programming from [9] is also available and it is almost identical to the OWEA designs in efficiency, but the OWEA framework only takes a small fraction of computing time. A − o p t = 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4 2 3 4 4 1 1 3 4 1 2 2 4 1 2 2 3 3 4 3 3 3 4 1 1 2 4 4 2 2 1 3 1 4 2 2 2 4 3 4 3 4 1 1 1 3 3 1 2 D − o p t = 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4 2 3 4 4 1 1 3 4 1 2 2 4 1 2 2 3 3 4 3 3 3 4 1 1 2 4 4 2 2 1 3 1 4 2 2 2 4 3 4 3 4 1 1 1 3 3 1 2 d 2 = 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4 2 2 3 4 1 3 4 4 1 1 2 4 1 2 3 3 3 3 4 2 4 1 1 3 2 4 4 1 3 1 2 2 4 4 2 2 4 1 3 3 2 2 4 1 3 3 1 1 Gurobi Design = 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4 2 2 3 4 1 3 4 4 1 2 4 4 1 2 3 3 3 3 2 2 4 1 1 3 2 4 1 2 3 1 1 2 4 4 4 2 4 4 3 3 2 4 1 1 3 3 1 1

( t , p ) = ( 4 , 4 ), n = 19, ℓ = ( 0 , 0 , 1 / 2 , 1 / 2 ). This is a case where n is not a multiple of either t or p, and the optimal designs in literature are not available. Exact OWEA designs and the Gurobi designs are summarized in Table 2. Still, the lower bound efficiency is around 0.95, but the efficiency loss due to rounding is less than 0.015. In addition, the OWEA designs outperform the Gurobi designs with improved efficiency of 1.0261 and 1.0785 under A- and D- optimal criteria but with only no more than 3% of computing time. A − o p t = 1 1 1 1 1 2 2 2 2 2 3 3 3 3 4 4 4 4 4 3 3 4 4 4 1 1 3 3 4 1 2 2 4 1 1 2 2 3 2 4 3 3 3 3 4 1 4 1 2 4 4 2 2 2 1 3 1 4 2 2 2 2 4 3 4 1 3 4 1 1 1 3 3 3 1 2 D − o p t = 1 1 1 1 1 2 2 2 2 2 3 3 3 3 3 4 4 4 4 2 3 3 4 4 1 1 3 3 4 1 2 2 4 4 1 1 2 3 3 2 4 3 3 3 4 1 1 1 4 4 4 2 2 2 2 1 1 4 4 2 2 2 4 3 4 4 3 2 1 1 1 1 3 3 3 2 Gurobi Design = 1 1 1 1 1 2 2 2 2 2 3 3 3 3 4 4 4 4 4 2 3 4 4 4 1 3 3 4 4 1 2 2 4 1 1 3 2 2 4 4 2 2 3 3 1 1 1 3 2 1 4 1 2 2 2 1 1 3 4 3 3 3 4 1 4 3 3 2 4 4 1 3 3 2 3 3

t = p = 4, n = 19, a = (0,0,1/2,1/2).

Optimality Rel-Eff (to Gurobi) Rel-Eff (Rounding) Efficiency (Lower Bound) Time (sec) Gurobi Time (sec)
A 1.0261 0.9944 0.9489 2.28 100.41
D 1.0785 0.9855 0.9576 2.71 100.41

( t , p ) = ( 4 , 4 ), n = 250. This is a real world application based on [28]. The Best African American Response to Asthma Drugs (BARD) Trial, contained two four-treatment, four-period, four-sequence crossover trials, one involving 294 adolescents and adults, and the other one involving 280 children. For the purpose of illustration, we focus on the BARD trial for the children. The four treatment regimens were represented by letters Q, R, S, and T. A total of 280 participants were evenly randomized to one of the four sequences, QRST, RTQS, SQTR and TSRQ. The study design was uniform and balanced. Out of the 280 children: 30 dropped out before period 1, 16 dropped out after period 1, 17 dropped out after period 2, 19 dropped out after period 3, and the remaining 198 completed the study. Since the subjects who drop out before period 1 provides no information at all, for the illustration of our method, we shall study the optimal design of crossover design with ( 280 − 30 = ) 250 subjects, 4 periods, 4 treatments, and the adjusted dropout mechanism as ℓ = ( 16 , 17 , 19 , 198 ) / 250 = ( 0.064 , 0.068 , 0.076 , 0.792 ). The resultant OWEA designs as well as their performances are available in Table 3.

A & D optimal
Sequence Repetition
QRST 9
QRTS 12
QSTR 15
QTRS 26
QTSR 1
RQTS 8
RSTQ 28
RTQS 5
RTSQ 22
SQRT 24
SRQT 29
STRQ 9
TQSR 30
TRQS 12
TSQR 16
TSRQ 4

t = p = 4, n = 250, a = (0.064, 0.068, 0.076, 0.792).

Optimality Rel-Eff (rounding) Efficiency (Lower Bound) Rel-Eff (to [28] with n = 250) Time (sec)
A 0.9999 0.9977 1.0023 2.24
D 0.9999 0.9916 1.0063 2.15

t = p = 3, n = 36.

Optimality Rel-Eff (to Gurobi) Rel-Eff (to d 2 in [25]) Time (sec) Gurobi Time (sec)
A 1.0390 1 0.01 0.06
D 1.0794 1 0.01 0.06

t = 4, p = 3, n = 12.

Optimality Rel-Eff (to Gurobi) Rel-Eff (to d 1 in [29]) Time (sec) Gurobi Time (sec)
A 1.009 0.9980 0.17 0.03
D 1.027 0.9997 0.22 0.03

t = k = 4, n = 10.

Optimality Rel-Eff (to Gurobi) Rel-Eff (rounding) Time (sec) Gurobi Time (sec)
A 0.9807 0.9756 1.21 2.86
D 0.9265 0.9116 5.34 2.86

The OWEA exact design under A- and D- optimality happened to be exactly the same. Their relative efficiency due to rounding are almost 1, with lower bound efficiencies being all above 0.99. The relative efficiencies as compared to designs in [28] are 1.0023 for A-criterion and 1.0063 for D-criterion. This means that the OWEA framework improves the design efficiency after incorporating the dropout mechanism. Finally, the computing time is around 2.20 seconds under both criteria.

 Crossover Model with Proportional Carryover Effects

( t , p ) = ( 3 , 3 ), n = 36. The proportional model is nonlinear. All OWEA designs in this paper are locally optimal. For this particular example, we assume all direct treatment effects to be 2 and λ = 0.2, and the covariance structure is taken to be the identity matrix. According to Table 4, the OWEA designs have identical efficiency under A- and D- optimal criteria compared to the optimal designs in the literature. In fact, the OWEA design is equivalent to the symmetric block ⟨ 123 ⟩ defined in [25] and it is shown to have unity efficiency for A- D- T- criteria and 0.9931 under E-optimality. Besides, the Gurobi design is also calculated, and comparably it is less efficient. Trivially, the difference between computing times is negligible as both algorithms take less than 0.1 seconds. A − o p t = 1 2 3 2 3 1 3 1 2 × 12 D − o p t = 1 2 3 2 3 1 3 1 2 × 12 Gurobi Design = 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 1 1 1 1 1 1 2 3 3 3 3 3 2 2 2 2 2 3 1 3 3 3 3 3 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 1 1 1 1 1 1 2 2 2 2 2 2 1 1 1 1 1 3 1 2 2 2 2 2 1 1 1 1 1 2

( t , p ) = ( 4 , 4 ), n = 12. Similarly, we assumed all direct treatment effects to be 1.732 and λ = 0.1, and the covariance structure is taken to be the identity matrix. As shown in Table 5, the efficiency of the OWEA designs relative to the Gurobi designs is 1.009 and 1.027 under A- and D- optimal criteria. There are minor differences in computing time, but all the computing times are below 0.3 seconds. In addition, design 1 (shown as d 1 in Table 5) in [29] is a set of 3 mutually orthogonal Latin square, which is balanced and neighbor-balanced. Comparably, the OWEA design is almost identical in efficiency under A- and D- optimal criteria. A − o p t = 1 1 1 2 2 2 3 3 3 4 4 4 2 3 3 1 3 4 2 4 4 1 1 2 4 2 4 3 4 3 1 1 1 2 2 3 3 4 2 4 1 1 4 2 2 3 3 1 D − o p t = 1 1 1 2 2 2 3 3 3 4 4 4 2 2 3 3 3 4 1 4 4 1 1 2 4 4 2 1 1 3 4 1 2 2 3 3 3 3 4 4 4 1 2 2 1 3 2 1 Gurobi Design = 1 1 1 2 2 2 3 3 3 4 4 4 2 3 4 1 3 4 1 2 4 1 2 2 4 2 3 3 4 3 4 1 1 2 1 3 3 4 2 4 1 1 2 4 2 3 3 1

Interference Model

( t , k ) = ( 4 , 4 ), n = 10. As it is shown in [11], the optimal weights of universally optimal design are not integers, and hence the Gurobi designs proposed therein are used as benchmarks. As it is indicated in Table 6, the relative efficiency of A- and D- optimal designs are 0.9807 and 0.9265 compared to the Gurobi designs. Note that the relatively lower D- efficiency is the limitation of the OWEA framework, especially when n is small. There is no obvious winner in computing times, as OWEA framework is faster searching for A-optimal designs but slower in D-optimal designs compared to those using Gurobi. A − o p t = 1 1 1 2 2 3 3 4 4 4 2 3 4 1 2 1 3 1 2 3 4 2 3 3 3 4 4 2 1 2 4 4 3 4 1 2 1 3 1 2 D − o p t = 1 1 1 2 2 3 3 4 4 4 2 3 4 2 4 2 3 2 3 4 3 4 2 4 1 1 1 3 2 1 3 4 2 3 3 1 4 1 1 2 Gurobi Design = 1 1 2 2 2 3 3 4 4 4 1 2 1 2 3 1 3 1 3 4 4 4 3 4 1 4 2 3 2 2 4 3 4 1 1 2 1 3 2 3

( t , k ) = ( 4 , 5 ), n = 24. Based on Theorem 6 (ii) of [13], the universally optimal design is a symmetric design comprising equal weights to sequences and their dual sequences, where the dual sequence has the same elements as its original sequence but in reversed order. The design referred as “Universal” in Table 7 is a design consists of symmetric blocks ⟨ 11234 ⟩, ⟨ 12344 ⟩. From the summary in Table 7, the OWEA designs are as efficient as universally optimal designs from [13] and the Gurobi design, with the relative efficiency being 0.9970 and 0.9981 under A- and D- optimality. The computing times, in this case, are 14.17 and 7.89 seconds. Interestingly, integer programming for the Gurobi design only takes 0.20 seconds. This is because of the existence of universally optimal designs and solving for a linear equation systems is straightforward and fast utilizing integer programming. A − o p t = 1 1 1 1 1 1 2 2 2 2 2 2 3 3 3 3 3 3 3 4 4 4 4 4 1 1 1 1 3 2 2 2 2 4 4 4 2 2 2 2 3 3 4 1 3 4 4 4 2 3 3 3 4 4 1 3 3 1 1 3 1 1 1 2 1 1 1 3 2 2 2 3 3 4 4 4 2 3 4 4 4 3 3 3 4 4 4 1 2 2 2 2 1 3 3 1 4 2 2 2 3 3 3 1 1 3 3 1 4 4 4 4 4 4 2 2 1 1 1 2 D − o p t = 1 1 1 1 1 1 2 2 2 2 2 2 3 3 3 3 3 3 4 4 4 4 4 4 1 1 1 3 3 3 2 2 2 4 4 4 2 2 2 3 3 3 1 1 1 4 4 4 4 4 4 4 4 4 3 3 3 3 3 3 1 1 1 1 1 1 2 2 2 2 2 2 3 3 3 2 2 2 4 4 4 1 1 1 4 4 4 2 2 2 3 3 3 1 1 1 2 2 2 2 2 2 1 1 1 1 1 1 4 4 4 4 4 4 3 3 3 3 3 3 Gurobi Design = 1 1 1 1 1 1 2 2 2 2 2 2 3 3 3 3 3 3 4 4 4 4 4 4 1 1 1 2 2 4 2 2 2 3 3 4 1 2 3 3 3 4 1 1 3 4 4 4 2 2 3 3 4 3 3 4 4 1 4 1 4 1 1 1 4 2 2 3 2 1 2 3 3 4 4 4 3 2 1 1 3 4 1 3 2 4 2 4 2 1 3 2 1 2 1 2 4 3 2 4 3 2 4 3 1 4 1 3 2 4 4 2 1 1 3 2 1 3 3 1

t = 4, k = 5, n = 24.

Optimality Rel-Eff (to Universal) Rel-Eff (to Gurobi) Time (sec) Gurobi Time (sec)
A 0.9970 0.9970 14.17 0.20
D 0.9981 0.9981 7.89 0.20
Discussion

In this paper, we provide an algorithm framework for optimal or efficient crossover designs, and later extend it to the interference model. Information matrices are derived in order to apply the optimal weight exchange algorithm. The OWEA exact designs in the numerical examples are shown to be efficient or identical to the optimal exact designs. The most noteworthy advantages of the OWEA algorithm are the generality and convenience. It works on a myriad of models, as long as the information matrix is linear with respect to the design point. Although the requirement on the information matrix is regarded as a limitation of the algorithm, in practice, it is typically met when using linear models or some non-linear model. From one model to another, the OWEA algorithm is applicable with only minor modifications to the information matrix. Within the same model, one only needs to adjust the values of case configurations, such as t, p, and n in those examples. Note that these configurations can be arbitrary, including the inclusion of a dropout mechanism. To facilitate the usage of our algorithm, we have prepared an R package and provided an online R Shiny app as a more user-friendly interface (https://cran.r-project.org/package=OWEA).

Although the theoretical result indicates optimal designs within a subclass of pseudo-symmetric designs are automatically global optimal, the OWEA algorithm does not guarantee symmetry in its output. However, based on our experience, OWEA exact designs are close to symmetric optimal designs and always have relatively high efficiency or satisfactory lower bound. In addition, we also observed the limitation of the framework when n is a small integer, where the efficiency is not very high. In our opinion, one of the reasons is that approximate optimal design usually consists of many distinct sequences with small weights, and rounding them into a small n exact design would result in losing many sequences that significantly contributes to the information matrix. Furthermore, the OWEA algorithm offers flexibility in the choice of parameter vectors. Note that all the designs in this paper focus on direct treatment effects. In other cases, for example, [25] derived corresponding results for estimating λ in the proportional model, whereas using OWEA algorithm, one only needs to change the function g.

Speed is also an important aspect of numerical algorithms. In the numerical examples, the OWEA algorithm only takes a reasonable amount of time to complete the tasks. However, there is still space for improvement. The current search spaces that we implemented in the numerical examples comprise almost all possible sequences or blocks. According to theory of Kushner’s approximate design theory, for example, [11, 13, 25], one can narrow the search space down to a much smaller one depending on combinations of ( t , p , n ) or ( t , k , n ). However, there is a trade-off between convenience and computing time. Another possible improvement lies in the Newton’s method. In some of the cases, we do encounter overshooting problems, but the resultant design after reaching maximum iterations is still highly efficient. If this issue could be resolved, the computing time would be faster than the reported time.

Finally, although we compared examples and algorithms in [9, 13, 25], one has to point out those papers are aiming at developing powerful theoretical tools while this paper is prone to be more practical oriented. Often, practitioners even statisticians cannot easily understand and use good theoretical results in design studies. This inability to use results greatly hinders the application of excellent theoretical work. The developed R package R Shiny app not only offers a user-friendly interface but also generates optimal and efficient designs that are not available in existing publications. Deriving such designs theoretically is often challenging or impossible, but our user-friendly package simplifies the process. This development is essential for making theoretical results more accessible to practitioners, and it will undoubtedly contribute to the continued advancement of this important field.

 Acknowledgments

We would like to extend our sincere appreciation to the editors for their dedicated efforts in reviewing our research paper. Their insightful comments and suggestions have undoubtedly contributed to the overall improvement of the manuscript.

 References Yang, M., Biedermann, S. and Tang, E. (2013). On optimal designs for nonlinear models: a general and efficient algorithm. Journal of the American Statistical Association 108(504) 1411–1420. https://doi.org/10.1080/01621459.2013.806268. MR3174717 Yang, M. and Stufken, J. (2008). Optimal and efficient crossover designs for comparing test treatments to a control treatment under various models. Journal of Statistical Planning and Inference 138(1) 278–285. arXiv:0505641. https://doi.org/10.1016/j.jspi.2007.05.005. MR2369633 Low, B. Y. J. L., Lewis, S. M. and Prescott, P. (1999). Assessing robustness of crossover designs to subjects dropping out. Statistics and Computing 9(3) 219–227. https://doi.org/10.1023/A:1008974031782 Matthews, J. N. S. (1988). Recent developments in crossover designs. International Statistical Review / Revue Internationale de Statistique 56(2) 117. https://doi.org/10.2307/1403636. MR0963525 Godolphin, J. D. (2004). Simple pilot procedures for the avoidance of disconnected experimental designs. Journal of the Royal Statistical Society. Series C: Applied Statistics 53(1) 133–147. https://doi.org/10.1046/j.0035-9254.2003.05054.x. MR2043764 Majumdar, D., Dean, A. M. and Lewis, S. M. (2008). Uniformly balanced repeated measurements designs in the presence of subject dropout. Statistica Sinica 18 235–253. arXiv:0710.0685. MR2384987 Bose, M. and Bagchi, S. (2008). Optimal crossover designs under premature stopping. Utilitas Mathematica 75 273–285. MR2392763 Zhao, S. and Majumdar, D. (2012). On uniformly balanced crossover designs efficient under subject dropout. Journal of Statistical Theory and Practice 6(1) 178–189. https://doi.org/10.1080/15598608.2012.647579. MR3196538 Zheng, W. (2013). Universally optimal crossover designs under subject dropout. The Annals of Statistics 41(1) 63–90. arXiv:1303.2800v1. https://doi.org/10.1214/12-AOS1074 Kushner, H. B. (1997). Optimal repeated measurements designs: the linear optimality equations. The Annals of Statistics 25(6) 2328–2344. https://doi.org/10.1214/aos/1030741075. MR1604457 Kunert, J. and Martin, R. J. (2000). On the determination of optimal designs for an interference model. The Annals of Statistics 28(6) 1728–1742. https://doi.org/10.1214/aos/1015957478. MR1835039 Kunert, J. and Mersmann, S. (2011). Optimal designs for an interference model. Journal of Statistical Planning and Inference 141(4) 1623–1632. https://doi.org/10.1016/j.jspi.2010.11.020. MR2747931 Zheng, W. (2015). Universally optimal designs for two interference models. Ann. Statist. 43(2) 501–518. https://doi.org/10.1214/14-AOS1287 Fedorov, V. (1972) Theory of Optimal Experiments Designs. New York: Academic Press. MR0403103 Wynn, H. P. (1970). The sequential generation of D-optimum experimental designs. The Annals of Mathematical Statistics 41(5) 1655–1664. https://doi.org/10.1214/aoms/1177696809 Silvey, S. D., Titterington, D. H. and Torsney, B. (1978). An algorithm for optimal designs on a design space. Communications in Statistics – Theory and Methods 7(14) 1379–1389. https://doi.org/10.1080/03610927808827719 Torsney, B. Algorithms for a constrained optimization problem with applications in statistics and optimum design (1981). PhD thesis, University of Glasgow. Hettich, R. (1983). A review of numerical methods for semi-infinite optimization. In Semi-Infinite Programming and Applications (A. V. Fiacco and K. O. Kortanek, eds.) 158–178. Springer Berlin Heidelberg, Berlin, Heidelberg. Böhning, D. (1986). A vertex-exchange-method in D-optimal design theory. Metrika 33(1) 337–347. https://doi.org/10.1007/BF01894766. MR0868043 Harman, R. and Pronzato, L. (2007). Improvements on removing nonoptimal support points in D-optimum design algorithms. Statistics and Probability Letters 77(1) 90–94. arXiv:0706.4394. https://doi.org/10.1016/j.spl.2006.05.014. MR2339022 Torsney, B. and Martín-Martín, R. (2009). Multiplicative algorithms for computing optimum designs. Journal of Statistical Planning and Inference 139(12) 3947–3961. https://doi.org/10.1016/j.jspi.2009.05.007. MR2558341 Martín-Martín, R., Rodríguez-Aragón, L. J. and Torsney, B. (2012). Multiplicative algorithm for computing D-optimum designs for pVT measurements. Chemometrics and Intelligent Laboratory Systems 111(1) 20–27. https://doi.org/10.1016/j.chemolab.2011.10.014. MR2558341 Yu, Y. (2010). Monotonic convergence of a general algorithm for computing optimal designs. The Annals of Statistics 38(3) 1593–1606. arXiv:0905.2646. https://doi.org/10.1214/09-AOS761 Kiefer, J. (1974). General equivalence theory for optimum designs (approximate theory). The Annals of Statistics 2(5) 849–879. https://doi.org/10.1214/aos/1176342810. MR0356386 Zheng, W. (2013). Optimal crossover designs for the proportional model. The Annals of Statistics 41(4) 2218–2235. https://doi.org/10.1214/13-AOS1148. MR3127864 Gurobi Optimization, LLC (2022). Gurobi Optimizer Reference Manual. https://www.gurobi.com. R Core Team (2022). R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/. Wang, X. and Chinchilli, V. M. (2021). Analysis of crossover designs with nonignorable dropout. Statistics in Medicine 40(1) 64–84. https://doi.org/10.1002/sim.8762. MR4194572 Kempton, R. A., Ferris, S. J. and David, O. (2001). Optimal change-over designs when carry-over effects are proportional to direct effects of treatments. Biometrika 88(2) 391–399. https://doi.org/10.1093/biomet/88.2.391. MR1844839