The composite endpoints have been used in many clinical trials to increase the chances of collecting more data from many domains of a disease to increase the study power. Although this idea sounds feasible and can be useful, having a clear understanding of when a composite endpoint should be considered and how to use it properly is very important. We use Figure 1 as a toy example to illustrate that if a composite endpoint is not constructed wisely, the results can be misleading.

Figure 1 displays a composite endpoint with two components. We assume that all the eight patients in the drug group respond to event A but not B. For the eight placebo patients, we assume that half of them respond to both events A and B, and the other half don’t respond to neither A nor B.

When we consider the composite endpoint by winning either A or B, results tell us that the drug response rate is 100% and the placebo response rate is 50%. However, if we further study the two individual events, we can see that this result is mainly driven by the event A, because the drug performs worse than the placebo on the event B. In particular, although for the composite endpoint A or B and the component A, the placebo response rate is 50% and drug response rate is 100%, for the component B the placebo response rate is still 50% but the drug response rate is 0%. In other words, if we do not consider any specific winning criteria, Event A and Event B should be equally important. Otherwise, results can be very misleading, and the study will not be powerful.

The idea of WRs is not new and has been extensively studied. This type of endpoint has also been utilized in many large cardiovascular and renal clinical trials [15, 6]. The basic idea of constructing a WR is first to pair all patients in two treatment arms and compare their performance according to pre-defined criteria to determine their winning status. At the end, combine all pairs’ winning status for making the final statistical inference. These pairs can be either coming from matched or unmatched samples [12, 19, 1, 13]. More details regarding how we applied the WR methods in either unmatched or matched pairs will be discussed and illustrated in Section 3. As noted in our toy example, how all the components are prioritized in the composite endpoint will affect the performance and interpretability of the WR results.

To generalize the use of the WR method in a composite endpoint with more than two components, we consider the situation in which a composite endpoint has multiple equally important components. For example, a composite endpoint with three equally important continuous components has notations described as follows

Suppose y p , j , i is the ith patient’s time to its jth component improvement in the placebo group, y t , j , i is the ith patient’s time to its jth component improvement in the treatment group, and y b a s e is a baseline. We identify the indicators of successful improvement for patients in the placebo group via the following indicators: I p , j , i = 1 y p , j , i / y b a s e , i < c t , 0 y p , j . i / y b a s e , i ≥ c t , I p , i = 1 ∑ j = 1 3 I p , j , i ≥ 1 , 0 ∑ j = 1 3 I p , j , i = 0 , where I p , j , i is an indicator that implies whether the ith patient in placebo group successfully improves on the jth component with cutoff c t , and I p , i is an indicator that implies whether the ith patient in placebo group successfully improves on at least one component. Similarly, we identify the indicators of successful improvement I t , j , i and I t , i for patients in the treatment group via the following indicators: I t , j , i = 1 y t , j , i / y b a s e , i < c t , 0 y t , j , i / y b a s e , i ≥ c t , I t , i = 1 ∑ j = 1 3 I t , j , i ≥ 1 , 0 ∑ j = 1 3 I t , j , i = 0 .

(Stratified) Matched Win Ratio  The logic here is similar to the Algorithm 1 except for some modification, especially the way to define the winner in every matched pair comparison. We stratified patients into different strata based on their baseline covariates, and then form matched pairs on the study drug and the control. For each matched pair, we determine that the patient in the study drug is a winner or a loser by the following rule: 1.

Calculate the total number of successful improvements for each patient in placebo, i.e., calculate ∑ j = 1 3 I p , j , i , i = 1 , … , n 0 .

(Stratified) Unmatched Win Ratio  The procedure here is the same as the unmatched WR method for the composite endpoint with the prioritized survival components. However, like the above matched WR for continuous components, the rule to define the winner in every matched pair comparison is completely different and should follow the winning rule in the new matched WR.

Contingency Table  For evaluating the advantage of WR methods, we construct a conventional contingency table as in Table 1. We let n 11 = ∑ i = 1 n 1 . I t , i , n 10 = ∑ i = 1 n 1 . ( 1 − I t , i ), n 01 = ∑ i = 1 n 0 . I p , i , and n 00 = ∑ i = 1 n 0 . ( 1 − I p , i ).

Then we perform hypothesis test via odds ratio. The idea is, instead of calculating the total number of improvements in the treatment (placebo) group for ith patient, a success of treatment (placebo) is counted if the patient has at least one improved component after being allocated to the treatment (placebo) group. Therefore, the test statistic and its distribution is O R ˆ = n 11 n 00 n 10 n 01 , log ( O R ˆ ) ∼ N ( 0 , s e ˆ ) , s e ˆ = 1 n 11 + 1 n 10 + 1 n 01 + 1 n 00 .

We utilize ‘coxed’ package in R statistical software to generate survival time response [10, 11]. For simplicity, we illustrate our idea by only considering two components, death and hospitalization.

Time to the Component Improvements with Less Clinical Importance  E h o s = H 0 − 1 [ − log ( u ) exp ( − X β h o s ) ] , where X = ( x t , x c o v 1 , x c o v 2 ), β h o s = ( β t , β c o v 1 , β c o v 2 ). The x t is an indicator of whether the patient is in the treatment group. β t is the expected log hazard ratio (HR) that compares the risk of a patient in treatment to that in control for hospitalization. The drug is effective if β t > 0. β c o v 1 and β c o v 2 are coefficients of covariate x 1 and x 2 , respectively. The u is randomly drawn from a standard uniform distribution U [ 0 , 1 ]. H 0 = ∫ 0 t h 0 ( s ) d s is cumulative baseline hazard function, where h 0 ( t ) represents baseline hazard;

Time to the Component Improvements with More Clinical Importance  E d = H 0 − 1 [ − log ( u ) exp ( − X β d ) ] , where X = ( x t , x d h r a i t o , x c o v 1 , x c o v 2 ), β d = ( β t + β i n , β d h r a t i o , β c o v 1 , β c o v 2 ). β i n is expected log HR that describes the difference between the risk of a patient for death and hospitalization in treatment group. Therefore, β t ′ = β t + β i n is the expected log HR that compares the risk of a patient in the treatment to that in control for the death event. The x d h r a i t o is a standardized random variable that describes the strength of the relationship between risk of death and hospitalization for each patient without treatment effect. The β d h r a i t o describes the strength of the relationship between E d and x d h r a t i o . β d h r a i t o = 0 indicates that the patient’s risk of hospitalization is equal to their risk of death in the control group.

Time to Patient’s Three Component Improvements in the Placebo Group  y b a s e = β c o v 1 x 1 + β c o v 2 x 2 , y p j = β p j ( 1 − x k ) + y b a s e + ϵ p j , j = 1 , 2 , 3 .

The y b a s e is a baseline vector and β c o v 1 and β c o v 2 are coefficients of covariate vectors x 1 and x 2 , respectively. In addition, y p j is a vector that stores the time (or any continuous measurements) to the jth component improvement of patients who are in the placebo group. The x k is an indicator vector that shows whether patients are in the placebo group ( x k = 0) or treatment group ( x k = 1). The β p j is the placebo effect that may reduce a patient’s time to the jth component improvement in placebo to that in baseline. The placebo is effective if β p j < 0. The ϵ p j is the randomness that corresponds to the jth placebo response.

Time to Patient’s Three Component Improvements in the Treatment Group  y 1 = β t 1 x k + y b a s e + ϵ t 1 , y 2 = ( β t 1 + β i n 2 ) x k + y b a s e + ϵ t 2 , y 3 = ( β t 1 + β i n 3 ) x k + y b a s e + ϵ t 3 .

β t 1 is drug effect that reduces a patient’s time (or any continuous measurements) to the first component improvement in treatment to that in baseline, which is effective if β t < 0. The β i n 2 describes the difference of drug efficacy between the first and second components in treatment group, i.e., β t 2 = β t 1 + β i n 2 is the drug effect that reduces a patient’s time (or any continuous measurement) to the second component improvement in treatment to that in baseline. In addition, β i n 3 has a similar definition to β i n 2 , and ϵ t j for j = 1 , 2 , 3 is the randomness that corresponds to the ith treatment response.

We use a toy example here to show how the matched win-ratio method in Section 3.1.1 saves samples for the composite endpoint with two prioritized binary components. In our simulation, we set Type I error α = 0.05 and power β = 95 %. We use the same notation as in Section 3.1.1 and apply the closed-form sample size calculation formula (3.2). We let p t , the probability of death in the treatment group, vary among ( 0 , 0.3 ) and keep other probabilities of an event fixed.

In Figure 4, we set p c = 0.3, q t = q c = 0.5. It mimics the scenario that compared to a placebo, a drug does not improve the component of less importance. That is the drug is effective to death only and has no effect on hospitalization.

The blue line is always below the red line, showing a clear difference between the WR method and the conventional method which does not consider clinical importance and treats the two components equally. This smaller minimum sample size of WR method also matches Table 9, where WR has larger power than the conventional method (i.e. cox regression) when the treatment has effect on death only.

It can also be observed that the difference is small at the beginning, as it represents the true difference between p c and p t (i.e., the x-axis value) is large, and the more p t approach the p c = 0.3 the greater WR method can save the samples. This further demonstrates the advantage of WR method in detecting small treatment effect for prioritized composite endpoints, and its potential for small-size studies.

In Figure 5, we set p c = 0.3, q t = 0.45, q c = 0.5, a scenario that a drug is effective to both two components. In contrast, the WR doesn’t provide much benefit in power improvement, which aligns the Table 7. It can be observed that (1) although the blue line is below the red line when p t < 0.24, the minimum sample size differences between the two lines are very small; (2) the WR does not have advantage when p t > = 0.24. That is the benefit of utilizing a prioritized composite endpoint decreases as the p t approaches the p c .

We let β c o v 1 = − 0.5, β c o v 2 = 0.5, x c o v 1 , x c o v 2 ∼ B e r n o u l l i ( 0.5 ), x d h r a t i o ∼ U n i f o r m ( 0 , 1 ). Table 3 shows the distribution of patients in four generated strata.

We estimate the HR based on Cox regression and calculate the WR for our proposed SED and analyses. In addition, we calculate the corresponding confidence intervals and Type I error as well as power via the exact methods.

Type I Error  When under the null hypothesis, a drug has no effect such that every patient is equally likely to have hospitalization/death in the treatment and control groups time. We show that either HR or win ratios are close to 1 and the Type I errors are controlled for all examined methods. Our results are displayed in Table 4 and Table 5.

Power for the Same Effects on Both Components  Next, we examine the performance of WR methods by comparing it with other commonly used analyses for cases with either both two components have a similar effect or only one having an effect. Our results are shown below.

As seen in Table 7, it can be observed that the powers order is Cox regression > O’Brien’s > stratified unmatched ∼ unstratified unmatched > stratified matched when assuming the same effects on both components.

Power for Having Effect on Death Only (No Effect on Hospitalization)  As seen in Table 9, it can be observed that the powers order is stratified unmatched > unstratified unmatched ∼ stratified matched > O’Brien’s > Cox regression.

Table 9 thus demonstrates that WR methods can more greatly increase trial efficiency than traditional methods when treatment is effective on a prioritized component that occurs after a prioritized component, where the traditional methods that measure the first event cannot be detected thus. Specifically, in Table 9, when N = 60, the two unmatched WR methods increases around 30% more power than ‘Cox regression’ and ‘O’Brien’s rank sum-type test’ (i.e. the two traditional methods); when N = 200, the improvement is 20% for ‘Cox regression’ and is 10% for ‘O’Brien’s rank sum-type test’. The ‘stratified matched WR’ also shows the same trend. This shows the advantage of WR in trial efficiency enhancement: for small sized studies, considering a composite endpoint with win ratio can help increase study power.

Power for Having Effect on Death Only but Assuming Wrong Winning Criteria  As seen in Table 11, it can be observed that the powers order is O’Brien’s > Cox regression > stratified unmatched ∼ unstratified unmatched ∼ stratified matched when assuming that only effect exists on the death event, not the hospitalization event.

As highlighted in the introduction, two-stage enrichment designs such as sequential parallel comparison design, SED and sequential multiple assignment randomized trial have been proposed and used in clinical trials. After learning that the use of WR can increase the study power, we are interested in assessing whether the idea of WR can be implemented in two-stage design to further increase trial efficiency for rare disease clinical trials. We consider the SED and compare it with complete randomization (CR) in our evaluation in the followings.

Check Type I Error  Drug and placebo are equally effective in all the three components.

All Type I errors in Table 12 are preserved when sample size N is big. In addition, The Type I error under stratified matched WR is preserved more slowly than others.

Power Comparison 

Scenario 1  The drug is equally effective in improving all three components, and it’s more effective than placebo in all the three components. The results are in Table 13.

When ∑ j = 1 3 | β p j − β t j | = 1.5, SED always outperforms CR. The WR methods for composite components under both designs achieve higher power than other tests when sample size N is large. Stratified methods have higher power than nonstratified methods.

Scenario 2  The drug is much more effective than placebo in the first component, but it’s equally effective as placebo in the 2nd and 3rd components. We decrease the drug’s overall efficacy to the three components. The results are in Table 14.

When ∑ j = 1 3 | β p j − β t j | = 0.5, although powers decrease, SED still outperforms CR.

Scenario 3  We keep assuming that a drug is equally effective in improving the three components and more effective than placebo. However, we adjust the distribution of patients by decreasing the proportion of the target patient p 3 . The results are in Table 15. When ∑ j = 1 3 | β p j − β t j | = 1.5 and target population is low, the SED even more outperforms the CR than the scenario when p 3 = 0.8 when the sample size N is small.

In summary, given the same sample size N, the power of SED is at least approximately equal to or greater than the one under CR, especially for smaller N. That is, two-stage enrichment designs can further enhance trial efficiency, especially for a small-size clinical trial. Let us take the ‘stratified unmatched WR’ as an example. In Table 14 (scenario 2), when N = 100 the ‘stratified unmatched WR’ under SED increases 14% power than the ‘Contingency Table’ (i.e. the traditional method) but increases 4% under CR; when N = 500 the ‘stratified unmatched WR’ under SED continues to increase 14% power and increases 12% under CR. The ‘unstratified unmatched WR’ has the same trend. Table 15 (scenario 3) further confirms the benefit of SED in improving power for win-ratio methods.