What Clinicians Should Know About Adaptive Clinical Trials

hi my name is Roger Lewis and I'm a senior medical scientist at berry consultants today I'm going to talk about what clinicians should know about adaptive clinical trials when one designs a clinical trial there's always substantial uncertainty at the beginning regarding how best to treat subjects in the experimental arm uncertainty in the best dose of the drug the duration of treatment or exactly what the target population should be that uncertainty creates uncertainty regarding what the optimal trial design would be yet within a traditional approach to clinical trial design all the key clinical trial parameters that are determined at the beginning of the trial are held constant during the trial execution essentially the trial design you start with is the trial design that you carry through to the end this leads to increase risk of a negative or a failed trial even if the treatment is inherently affected and in fact perhaps a failed trial one that fails to give a clearly positive or a negative response is the worst possible outcome after the investment in a clinical trial however once patients are enrolled in a trial and their outcomes start to become known information is accumulating that reduces uncertainty regarding the optimal treatment approaches adaptive clinical trials are designed to take advantage of this accumulating information by allowing modification to the trial parameters in response to accumulating information and during the course of the trial according to predefined rules by taking advantage of this partial information this allows us to create a smarter trial and lower the chance of a failed trial this is a cartoon that's taken from JAMA back in 2006 that simply shows that an adaptive trial differs from a traditional trial in that there are multiple paths that MIT can be taken from the beginning of the trial shown by the oval at the top to the end of the trial and it is in fact the data that accumulates within the trial itself that determines which path is taken so when we talk about an adaptive clinical we mean a trial in which we make planned well-defined changes in key clinical trial design parameters during trial execution and based on data from that trial itself to achieve goals of validity scientific efficiency and safety by planned we mean that all the changes that we might make are defined a priori by well defined we mean that the criteria for making those changes are planned and by key parameters we're talking about changing big things like the number of treatment arms the dose that's used or the timing of interim analyses not minor inclusion or exclusion criteria not the sorts of things that would fall under routine amendments and we want to do this in a way in so that we achieve scientific validity and can make reliable statistical inferences regarding the treatment effects observed within the trial so this diagram shows the basic structure of an adaptive trial we begin the data collection with an initial set of allocation and sampling rules an allocation rule determines how the incoming patients are allocated to the various available treatment arms and the term sampling rules refers to how many patients will enroll before we take a first look at the data once we have completed that initial period where we're using these initial rules sometimes this is called the burnin period we analyze the available data knowing full well that those data will be incomplete some of the patients for example may not have made their last study visit or in fact some of those data have not been queried yet and there may be a small rate of errors within those data we ask ourselves based on the available information whether a stopping rule has been met and if no reason for stopping the trial for example because of evidence of harm or overwhelming benefit has been met then we take the available information and we revise those allocation sampling rules using whatever is the specific adaptive algorithm for the trial we then continue that data collection and continue this circular process until one of the pre-specified stopping rules is met one of the pre-specified stopping rules is always that we have reached the maximum allowable sample size for the trial and the stopping rule may dictate that the trial stops or it may refer simply stopping a phase or stage of the trial for example in a seamless 2:3 design the process of designing a clinical trial using an adaptive approach has a number of characteristics that distinguishes from the design process for a traditional design because one has to consider the kinds of changes that might be made we often obtain greater clarity of goals for the trial for example we need to determine whether we're trying to simply show proof of concept meaning that there is some treatment effect or we're trying to identify the dose of a drug during a phase 2 trial that we're going to carry forward into phase 3 versus trying to confirm a benefit that has been preliminary shown in learn phase trials a statistically significant p-value is never the goal of an adaptive trial because it is so easy to generate a trial design that yields a statistically significant p-value but fails to answer the question of what the next stage is ought to be in drug or device development these designs typically take frequent looks at the data because the goal is to make data-driven modifications to the trial and they are adaptive by design we are pre specifying the adaptations and the criteria for those adaptations with specific goals in mind we are not making post hoc or ad hoc changes to the design and because these trials are inherently more complex than a traditional trial we use computer simulation to understand the performance characteristics of the trial and to adjust the characteristics of the trial design to meet our goals for type 1 error control power or other operating characteristics when comparing the characteristics of a traditional approach to clinical trial design to a flexible approach there's a number of characteristics that generally help us distinguish the two for example in a flexible or adaptive approach there's generally a greater number of interim analyses we may use a variable randomization ratio as opposed to a fixed one to one or two to one randomization we generally will have a larger number of experimental arms available to the trial so that we can explore the treatment space with greater completion we have a plan for using incomplete data that occurs during the intro analyses to the best of our ability and we often use either a Bayesian or frequentist approach but we have a tendency to use a Bayesian approach to make the best use of partial data and then as mentioned earlier we're going to use computer simulation to understand the error rates of the trial so why do we do go to this additional work to do an adaptive and fundamentally more complex trial design we do this to avoid getting the wrong answer to avoid drawing a qualitatively incorrect conclusion such as concluding that a drug doesn't work when in fact it has clinically important efficacy or drawing the conclusion that it doesn't work because we didn't apply it in the right patient population we also want to avoid taking too long to draw the right conclusion too long in terms of the investment of time the number of human subjects that are put at risk or in the spending of scarce resources when thinking about what are the areas of a trial in which there is an advantage to using adaptive approach it's very useful for clinicians to think about a concept we term anticipated regret anticipated regret is a mental exercise where you put yourself in the position of considering what you wish you had done differently if you knew your current approach had led to a failed trial meaning a trial that failed to definitively answer the question that you are trying to ask so a substantial fraction of all confirmatory trials fail despite promising learn phase results investigators and clinician experts can often anticipate the design decisions they wish they could take over after the trial fails and those are the areas in which we want to design the adaptive trial to mitigate that risk now there is a wide variety of adaptive strategies that we can bring to bear to create a more efficient trial with a greater chance of success these include frequent interim analyses longitudinal modeling so we learn about the relationship between early and late endpoints response adaptive randomization where we change randomization ratios to preferentially randomize future patients to treatment arms that are either the most promising or about which we need the most Meishan explicit decision rules based on Bayesian predictive probabilities at each interim analysis so we continue or stop a trial based on the predicted probability of an event in the future such as the trial ultimately showing benefit dose-response modeling so we use the relationship between the treatment responses at different doses to reduce our uncertainty regarding the dose response or enrichment designs where we focus on patient populations based on the data coming into the trial so we enroll the patients most likely to benefit and all of these approaches however they're put together in a particular trial need to be evaluated through extensive numerical simulation so we truly understand the trial strengths and its weaknesses so coming back to this picture of the overall adaptive trial how do we understand what's a good trial and what is a trial that is not yet ready to be implemented or has substantial risks we do that through a process called trial simulation in trial simulation we test drive the clinical trial design thousands of times by making assumptions about what the truth is regarding the patient population that's going to be enrolled the Kuril rates the efficacy and safety profiles of the drugs so we conduct the trial thousands of times shown by the stack of pictures and we look at the average performance how often does it get the right answer how often does it get the wrong answer and we look at the sink examples of single trials to see if at interim analyses the decisions made by the adaptive algorithm see meth eclis clinically and scientifically appropriate it is by inspection of the output of extensive trial simulation that both statisticians and clinicians can understand the strengths and the weaknesses of the trial and understand what its performance should look like during the actual conduct of the trial because after all the goal of this is to kill as few people as possible during the conduct of the trial and at the same time yield scientifically valid reliable estimates of treatment effects so in conclusion not all trials need or should have an adaptive design but when used appropriately adaptive designs may improve efficiency and reduce cost maximize the information obtained from the enrolled patient population minimize risk to both the subjects and the sponsor but to achieve this the design decisions should be based on objective performance rather than habit or tradition and that objective performance is evaluated by a simulation an adaptive design will not save a poorly planned trial or an effect ineffective treatment but it will help you more efficiently identify those treatments with to true promise and minimize the efforts expended in testing treatments that are ineffective thank you very much you

Leave a Reply

(*) Required, Your email will not be published