Adaptive Clinical Trials

Don Berry

At the recent NCCN conference, I was introduced to the idea of an "adaptive clinical trial."  There are a number of ideas that get amalgamated into this general description. One example is that the weighted random assignment into each arm of the trial changes over time in response to trial results.  

For example, the trial may begin by assigning all patients to each of the two arms with probability one half.  However, after say the first year the data suggests patients in one of the two arms have greater survival, at that point the weight increases.  It may be that if probability of surviving 6 months is 5 percentage points higher for the new treatment, then the proportion of new patients assigned to the new treatment arm increases to 60-40.  If at a year, the difference remains, then the percentage may increase to 70-30.  However, if the difference reduces, the percentage of patients assigned to the new treatment arm may fall to 50-50.  On the other hand, if patients in the new treatment arm have one year survival that is 10 percentage points higher, then the percentage of patients assigned to the new treatment may increase to 80-20.

Before talking about the advantages or disadvantages of such a trial design, it is important to remember that current trials accrue patients over time and large trials may take years before they have enough patients.  During this time, information on the value of the new treatment is being carefully accumulated, stored and kept away from prying eyes.  The FDA strongly prefers that data not be available to researchers so as not to lead to biased trial results.  That said, trials often have a data committee who are responsible for looking at the data and deciding whether or not there are severe enough safety problems such that the trial should be stopped.

From a statistical point of view, the trial design has no obvious disadvantages and may have some advantages.  Assignment is still random, and as long as no information on the value of the various treatments leaks to patients or doctors, the trial results should remain unconfounded and unbiased.  One advantage of the trial design is that it may reduce the selection-into-study bias.  Potential patients may be much more willing to sign up for a trial where they know that the probability of being assigned the better arm is increasing in the difference between the trial arms.

Two possible problems are that the power of the trial may be reduced because one of the arms is getting a smaller proportion of the patients.  Another problem is associated with so called "bandit problems."  It is theoretically possible that through statistical coincidence, patients on one of the trial arms start doing very poorly.  This will cause the weights to move away from that arm to the other trial arm.  It may be that the initial poor results would be overwhelmed by later data, had the assignment remained at 50-50 but because of the adaptive design there isn't enough patients being assigned to the trial arm for overwhelming to occur.  While this is not a bias problem, it may mean a greater probability of false-positive trial results.

The FDA raises a number of concerns with such trial designs in this report.