Saving EMILIA

In a recent post I raised a concern about the large randomized clinical trial conducted to test the value of the drug Kadcyla.  The trial was called EMILIA and it was written up in the New England Journal of Medicine.  

The concern is that a number of people left the trial and so we do not observe if and when they passed away.  Moreover, these patients may left from the two trail arms at different rates, biasing our results.  The authors seem to be aware of the issue when they appeal to an "intent-to-treat" analysis.  The problem is that even though the patients were randomly assigned to the two treatment arms, we don't know who left the trial and why they left.  Moreover, the trial was open label.  That is, patients knew exactly which arm they were in when they made the decision to leave.

In the post I suggested that it may not be possible to determine causality with intent-to-treat analysis.

My colleague, Matthew Chesnes, suggested that it may be still possible to determine causality even if not everyone accepted the random assignment that they were given.  His intuition is that if almost everyone accepted their assignment wouldn't we get pretty close to showing causality?

The intuition is correct.

If it was the case that everyone in the EMILIA trial accepted their random assignment then we can use the observed probabilities to determine causality.  We see that 35% of women in TDM-1 (Kadcyla) trial arm passed away within the first two years, while 48% of women in the X+L arm (the alternative treatment) pass away in the first two years (see chart).  From these numbers we can determine that TDM-1 causes women to have greater survival.  In fact, we can determine that for at least 12% of women in the study would have lived less than 2 years on X+L but survived over 2 years on TDM-1.

The problem is that not everyone did accept their random assignment.  From clinicaltrials.gov we learn that 38 women in the TDM-1 arm left the trial and 52 women in the X+L arm left the trial.  No information is provided about when these people left the trial.  And because they left, we don't know what happened.  Still, there are only two possibilities.  They may have passed away within 2 years or they may have lived longer than 2 years.

We can use an idea developed independently by the econometrian, Charles Manski, and the epidemiologist, James Robins.  The insight of Manski and Robins was that when we don't observe probabilities we may still be able to bound the probabilities using information about the proportion who leave the trial and the fact that probabilities lie between 0 and 100%.

For the 495 women assigned to TDM-1, 457 stayed with their assignment and had a 35% probability of passing away within two years.  For the 38 women who left the trial the lowest probability is 0% and the highest probability is 100%.  From the law of total probability we can determine that the lowest probability of passing away within two years given the assignment to TDM-1 is (457/495)*35 = 32% and the highest probability is (457/495)*35 + (38/495)*100 = 40%.

For women originally assigned to the TDM-1 arm, their probability of surviving at least two years lies between 60 and 68%.

We can do the same thing for the X+L arm.  The lowest probability of passing away within two years is (444/496)*48 = 43% and the highest probability is (444/496)*48 + (52/496)*100 = 54%.

For women originally assigned to the X+L arm, their probability of surviving at least two years lies between 46 and 57%.

Note that these two bounds do not overlap.  That is, it must be the case that women assigned to the X+L have a lower probability of surviving two years than women assigned to the TDM-1 arm.  As there is no other explanation for the difference, we can assign the difference to the drug treatment.

As, at least 60% of women in TDM-1 survive more than 2 years and at most 57% of women on X+L survive more than 2 years, it means that at least 3% of women live longer on TDM-1 than X+L.

Certainly, 3% is not huge, but it is positive.

Kadcyla causes at least some women to survive longer than they would have if they had taken the combination of Lapatinib and Capecitabine.

Despite the potential for bias from attrition, EMILIA may still provide proof of causality.