Early stopping adaptive trial designs: Bayesian or frequentist?

The August talk of the South Australian branch was held in the Napier building at Adelaide University on Wednesday 22nd. The presenter was Dr Andrew Vincent, a biostatistician at the University of Adelaide.

Andrew first introduced himself to a full and mixed audience comprised of statisticians, students and clinicians, with a brief overview of his background. This included degrees in mathematics and statistics from Adelaide and Sydney University and eight years’ experience in the field in the Netherlands where he helped design Phase III clinical cancer trials before returning to work in Adelaide.

Andrew’s special interest in study design led him to consider the limitations of standard RCTs and the possibilities of using Bayesian approaches to overcome some of these limitations. In particular, Andrew discussed how Phase III drug trial designs were perhaps not always appropriate given the underlying research questions and hypotheses. On the plus side they are typically straightforward and simple in design (such as 2-arm trials), preferably have hard endpoints such as overall survival, and they tightly control for the probability of Type 1 error (alpha). On the down side however, they also typically require quite large sample sizes in order to detect the minimal clinically interesting difference (MCID) which can sometimes be quite small.

In practice, the limited ability to recruit sufficient patients can mean RCTs lasting the standard 3 to 5 years involve soft endpoints such as Progression Free Survival (PFS) rather than Overall Survival (OS). They can therefore essentially be thought of as underpowered Phase III trials which offer a compromise between the idealised hard endpoint trial that might take 10 years to complete and therefore be outdated when finished, and the smaller hard endpoint trial that finishes within a suitable timeframe but which is underpowered. Such trials can also often end up providing unsatisfactory answers. For example, how should we interpret a trial in which the primary endpoint of PFS gave p=0.06 and the chosen secondary (but more meaningful) endpoint of OS gave p=0.01?

Andrew described several alternative study design approaches including more relaxed Discovery trials with less stringent control of alpha (e.g. consider 1 or 2 different treatment interventions), Group sequential designs with alpha-spending functions and early stopping for futility, and Phase IIb Cohort Refinement Platform Trials (CARA). This finished with a brief history of Bayesian RCT designs that were described by David Spiegelhalter in J R Statist. Soc A (1994):157 (3):357-416. This attracted much debate at the time amongst well known statisticians in the field such as Pocock, DeMets,  Altman and Harrell.

Andrew discussed a number of key decisions to be made in implementing the Bayesian approach including the balance between sceptical priors and enthusiastic priors. Also, a normal prior might not always be appropriate and therefore mixed Gaussian priors are typically used. A user’s guide as to how to implement the approach in practice was provided by Frank Harrell in his book on “Regression Modelling Strategies”. The remainder of the talk then focussed on how Andrew has applied this to his own trial data.

Unlike a standard RCT, in which the approach is to simply accrue a predefined number of patients and then stop, the overarching goal with Bayes is to best determine when to stop for efficacy and also when to stop for futility. One might decide to stop for efficacy, for example, when P(D>0 | x) >0.95 and P(D<0.05 | x) >0.90. Crucially also, rather than having a restricted number of interim analyses, the Bayesian approach allows the researcher to continue to look at the data as often as required (after every patient if desired) and to continue this approach until it is deemed that the trial should be stopped due to evidence of either efficacy or futility – whose posterior distributions would be continuously updated.

Andrew described Bayesian analysis of a current example of a trial for weight loss with the treatments of either one or both of energy restriction and intermittent fasting. The aim was to stop the trial only when both endpoints were significant. The Harrell Bayesian approach was applied to the data to calculate the probabilities of stopping for futility and efficacy.

Andrew concluded the evening’s talk by noting the advantages of the Bayesian approach in regards to the RCT’s objectives. The approach allows one to be optimistic about the eventual effect size, it is flexible, and it allows for early stopping. In particular it also allows for continuation of a trial which seems promising but which may not yet have reached stati

Andrew Vincent

stical significance after the original sample size has been reached.

Following questions and discussion amongst the audience, a number of members as well as Andrew and the President Shahid Ullah adjourned for dinner at the Lemongrass Thai Bistro on Rundle Street.

By Richard Woodman


Get the latest posts delivered to your mailbox:

Show Buttons
Hide Buttons