Using Mixture Models to Determine Treatments for Colon Cancer

Finite mixture models seem to hold much promise.  They are currently used for everything from determining what movie you would like to watch to analyzing gene-expression (see Geoffrey McLachlan's page).  I am interested in how they can be used to determine variation in treatment effects (see previous post).

The question is whether there are different types of stage 3 colon cancer patients.  Eventually the question is whether the therapy should be different for the different types of patients, but here it is just whether the patients are different.

The two charts are created from the Moertel et al (1990) data on use of adjuvant chemotherapy among stage 3 colon cancer patients.  See more on this data here.  The method is the "npEM" kernel density "EM-like" algorithm in the "mixtools" package.  I added the bounds calculator.

The first chart shows the probability distribution over days of survival for two latent types of patients, the Type Green patients and the Type Red patients.  The separation in the lines is due to the fact that the estimation method is unable to exactly determine the distribution.  The second chart presents the distribution over positive lymph nodes for Type Green and Type Red patients.

These charts are based on a finite mixture model in which it is assumed that there are at least two types of patients (fewer is allowed).  It is also assumed that there are two "signals" of the patient's latent type.  The first signal is the number of days of survival after entering the trial.  The second signal is the number of positive lymph nodes the patient has.  

It is assumed that while there is a statistical relationship between survival and the number of positive lymph nodes, there is not a causal relationship.  It is the unobserved type that determines the relationship between survival and the number of positive lymph nodes.  See American Cancer Society on lymph nodes and cancer.  This assumption is potentially testable using this data because we can estimate the same model separately in the two other treatment arms and by random assignment, the mixture of latent types should be the same across treatment arms.

The charts use the "observational" arm of the trial.  The data in this arm is not able to perfectly distinguish the latent types even with the assumptions I made above.  Still we can see that there do seem to be at least two types of stage 3 colon cancer patients.

The Type Green patients tend to live longer (actually the trial ends before death for most of these patients) and these patients generally have less than 5 positive lymph nodes.  Type Red patients generally have lower survival probabilities and tend to have a much higher number of positive lymph nodes.

While this analysis is very rough, it does raise an important question.  Should all stage 3 colon cancer patients receives adjuvant chemotherapy?  We know from the trial results that the "average" patient lives longer with adjuvant chemotherapy.  But is that average patient Red or Green or some mixture of both?

Solving the Mystery of the Swedish Mammography Trial cont.

Ja. Jeg fandt det!  

Dicte would be proud. I had been looking for causality to the mammography trial in the two Swedish counties of Dalarna and Ostergotland, but the culprit turned out to be Malmo, across the way from Denmak.  Ironically, Malmo was fingered by lead author of the Two-County trial, Laszlo Tabar.  

Tabar argued in the BMJ that the Malmo trial shouldn't be used to determine the value of mammography screening because between 70% and 74% of women invited to the trial actually had a mammogram and some 24% of women who were not invited had a mammogram.

Fik dig!

The researchers who conducted the Malmo trial actually determined both the compliance rate for the women who were invited to the trial and separately for the women who were not invited.  For the second group the researchers surveyed 500 women and determine the number of mammograms that they had.

The Malmo study, randomized women between 45 and 69 living in the city of Malmo between an invitation to have a mammogram and a control group.  The study found that after about 9 years of follow up 63 of 21,088 in the invitation group and 66 of 21,195 in the control group passed away from breast cancer.  Again, these are very small differences and we have the issue that it is an intent-to-treat analysis. But now have estimates of the mammography rates in the two groups.

If we assume that the take up rate of mammography of those receive the letter is 70% and the take up rate for those who didn’t is 24% and we also make the "exclusion restriction" that the letter has no impact on the probability of dying from breast cancer, we can calculate the average effect of mammography on survival from breast cancer for women in Malmo. 

We can write down the following system of equations, where A is the probability of dying from breast cancer conditional on getting mammography screening and B is the probability of dying from breast cancer without mammography screening. 

0.7*A + 0.3*B = 63/21088

0.24*A + 0.74*B = 66/21195

Plugging these equations into Mathematica we get.

A = 0.00291

B = 0.00317

While the intent-to-treat analysis suggest a 4% reduction in death from breast cancer associated with mammographic screening.  Accounting for the actual take up rate suggests an 8% reduction. Note that none of this accounts for sampling variation and is only presented for illustrative purposes.

Now this analysis also assumes that everyone who took up mammography screening is the same as those that didn't take up the screening, excepting for the fact that they took up mammography screening.  This is plainly not true. 

To be continued....

Solving the Mystery of Swedish Mammography Trials

Kenneth Branagh as Kurt Wallander

I felt like Kurt Wallander on the trail of a suspect in a small Swedish town with the murder rate of Cabot Cove.  The quarry was causality in the Swedish Two County randomized controlled trial.  

The trial, conducted some thirty years ago, randomized women in two Swedish counties (thus the name) to receive either an invitation to receive a mammogram or no invitation.  These invitations were given over the next several years and the trial participants were followed for the next twenty plus years.

The results were that 339 women of 77,080 passed away from breast cancer in the invitation group and 339 women of 55,985 passed away in the control group.  The claim is that this result proves that general mammography screening reduces death from breast cancer.  The issue is that this doesn't tell us that mammography reduces death from breast cancer.  

This is an intent-to-treat analysis.  

We know that these women were randomized into two groups and we know the average death rate from breast cancer in the two groups, but we don't know how many people actually got a mammogram in each of the two groups.  It was this number that I was searching for.  How many people actually got mammograms?

After trudging through the many papers written on the study I found one of the two numbers I needed.  Approximately, 85% of women in the invitation group accepted the invitation and received a mammography.  As far as I can tell the researchers never determined the take up rate of mammography in the group that didn't receive mammography.

So what can we say from this trial?

As long as the mammography rate in the invitation group was higher than in the control group then the analysis implies that mammography causes a reduction in death rates from breast cancer.  If we are willing to make a behavioral assumption - that Swedish women are motivated to get mammography by receiving invitations, then this large randomized controlled trial can be used to infer causality.  Our prime suspect must have had an accomplice.

We can't really say much more about the value of mammography without knowing how many women in the control group received a mammogram. 

Saving EMILIA

In a recent post I raised a concern about the large randomized clinical trial conducted to test the value of the drug Kadcyla.  The trial was called EMILIA and it was written up in the New England Journal of Medicine.  

The concern is that a number of people left the trial and so we do not observe if and when they passed away.  Moreover, these patients may left from the two trail arms at different rates, biasing our results.  The authors seem to be aware of the issue when they appeal to an "intent-to-treat" analysis.  The problem is that even though the patients were randomly assigned to the two treatment arms, we don't know who left the trial and why they left.  Moreover, the trial was open label.  That is, patients knew exactly which arm they were in when they made the decision to leave.

In the post I suggested that it may not be possible to determine causality with intent-to-treat analysis.

My colleague, Matthew Chesnes, suggested that it may be still possible to determine causality even if not everyone accepted the random assignment that they were given.  His intuition is that if almost everyone accepted their assignment wouldn't we get pretty close to showing causality?

The intuition is correct.

If it was the case that everyone in the EMILIA trial accepted their random assignment then we can use the observed probabilities to determine causality.  We see that 35% of women in TDM-1 (Kadcyla) trial arm passed away within the first two years, while 48% of women in the X+L arm (the alternative treatment) pass away in the first two years (see chart).  From these numbers we can determine that TDM-1 causes women to have greater survival.  In fact, we can determine that for at least 12% of women in the study would have lived less than 2 years on X+L but survived over 2 years on TDM-1.

The problem is that not everyone did accept their random assignment.  From clinicaltrials.gov we learn that 38 women in the TDM-1 arm left the trial and 52 women in the X+L arm left the trial.  No information is provided about when these people left the trial.  And because they left, we don't know what happened.  Still, there are only two possibilities.  They may have passed away within 2 years or they may have lived longer than 2 years.

We can use an idea developed independently by the econometrian, Charles Manski, and the epidemiologist, James Robins.  The insight of Manski and Robins was that when we don't observe probabilities we may still be able to bound the probabilities using information about the proportion who leave the trial and the fact that probabilities lie between 0 and 100%.

For the 495 women assigned to TDM-1, 457 stayed with their assignment and had a 35% probability of passing away within two years.  For the 38 women who left the trial the lowest probability is 0% and the highest probability is 100%.  From the law of total probability we can determine that the lowest probability of passing away within two years given the assignment to TDM-1 is (457/495)*35 = 32% and the highest probability is (457/495)*35 + (38/495)*100 = 40%.

For women originally assigned to the TDM-1 arm, their probability of surviving at least two years lies between 60 and 68%.

We can do the same thing for the X+L arm.  The lowest probability of passing away within two years is (444/496)*48 = 43% and the highest probability is (444/496)*48 + (52/496)*100 = 54%.

For women originally assigned to the X+L arm, their probability of surviving at least two years lies between 46 and 57%.

Note that these two bounds do not overlap.  That is, it must be the case that women assigned to the X+L have a lower probability of surviving two years than women assigned to the TDM-1 arm.  As there is no other explanation for the difference, we can assign the difference to the drug treatment.

As, at least 60% of women in TDM-1 survive more than 2 years and at most 57% of women on X+L survive more than 2 years, it means that at least 3% of women live longer on TDM-1 than X+L.

Certainly, 3% is not huge, but it is positive.

Kadcyla causes at least some women to survive longer than they would have if they had taken the combination of Lapatinib and Capecitabine.

Despite the potential for bias from attrition, EMILIA may still provide proof of causality.

Intent-To-Treat is the Last Refuge of the Cancer Resercher

In the large Swedish study on the effectiveness of mammograms the researchers couldn't force people to get mammograms but they could force them to receive letters.  Swedish women in two counties were randomized into two groups.  One group received an invitation to get a mammogram and the other group did not.  The EMILIA study of the effectiveness of Kadcyla on advanced breast cancer suffered from biased attrition.  In both studies, the researchers resorted to "intent-to-treat" to save their study, get published and claim a causal relationship.

Intent-to-treat refers to the idea that while the patients were not randomly assigned to the treatment groups they were randomly assigned an observed characteristic (they received a letter or not) and that observed characteristic MAY be associated with the treatment assignment.  It is like doing one stage of a two-stage instrumental variables analysis.

The problem with relying on intent-to-treat is that it may not provide evidence of causality.

To see this, think about what happens if we just observed two groups, one group received regular mammography and the other group did not.  We also observe their breast cancer rates and survival rates.  In fact, assume that we observe higher survival rates among the women who received regular mammography.  From this information, and only this information, can we determine the causal effect of mammography on survival from breast cancer?

We cannot.  

The problem is that we don't know anything about how the two groups were selected.  Even if we are able to account for differences in the observable characteristics like age, there still may be  differences in unobserved characteristics such as the women's genetic profile.

Now what if I told you that the group who received a mammogram was much more likely to have received an invitation to get the mammography than the group who did not receive a mammography?  Moreover, the invitation was randomly assigned.  Can this information determine the causal effect of mammography on survival from breast cancer?

It cannot.

The problem is the same.  Despite the random assignment of the invitation we still do not know the make up of the two groups.  In particular, we do not know things about the unobserved characteristics of the women such as genetics or a family history of breast cancer that would make them more likely to get a mammography (with or without the invitation).

The same problem occurs in the EMILIA trial.  Women left the trial at different rates depending on the treatment arm that they were assigned.  Because they left we do not know when or if they passed away.  The women that remained in the trial may be different across the two arms, so we can no longer assign the difference in the treatment outcomes to the different treatments.  We can no longer remove the possibility that the different outcomes were due to other differences between the women in the two trial arms.  We cannot use the trial to determine the causal effect of Kadcyla on breast cancer survival even though the women were randomly assigned to treatments.

A Fools Gold Standard

Austin Frakt at The Incidental Economist has a review on Angus Deaton's critique of randomized control trials as the "gold standard" of science.  Frakt suggests that one major advantage of RCTs is that they are "conceptually simple" requiring less mathematical or statistical training in order to understand the results.

Ideal randomized control trials provide two pieces of information:

1.  An unbiased estimate of the average treatment effect, and

2.  An unbiased estimate of the minimum proportion of the population who benefit from the treatment over its tested alternative.

I argue here that learning the average treatment effect is not terribly useful.  The question is whether we can learn that there exist people who benefit from the treatment from settings as conceptually simple as RCTs.  Here I suggest two.

The first is the case where a treatment becomes available at a certain point in time and we can look at what happened before and after.  The chart below shows the survival rate of HIV-infected patients before and after the introduction of various drugs that became the AIDS cocktail or HAART regime.  This chart was published in the New England Journal of Medicine in 1998 and was one of the first pieces of evidence that HAART was enormously effective in reducing deaths from AIDS.

The second setting is perhaps more controversial.  It is the case where we are willing to assume that the people in our study are selected into the treatments that are generally going to make them better off.  Technically, we need people to be selected into the treatment that "first order stochastically dominates."  Some writers call this assumption the minimal requirement for rational decision making (Hadar and Russell, 1969).

The chart below is from the BLS and reports the familiar result that people who attend college earn more money.  If we are willing to assume that people choose the education level that is more likely to give them the higher earnings then the result presented in the chart shows that for at least some people a college education increases wages.

But haven't we been told over and over again that this chart suffers from "selection bias"?  People who attend college may simply be those who would have earned more money anyway.  If college doesn't do anything to people's future income then these people are certainly spending a lot of money to play beer pong and attend college football games.  If they are spending money for nothing, then claims that college attendees are smarter than the average Joe are pretty suspect.

To be clear the claim is that college increases incomes for some people not necessarily for all people or even many people.  The chart and the assumption that college students are not idiots suggest that college has a causal effect on income.  No RCT needed.

RCTs are Necessary to Determine Causality. Right? Wrong.

It seems to be a widely held belief that randomized control trials are preferable if one is interested in determining causality.  Causality is something of a slippery concept, but let us use the following definition:  Drug A causes an increase in survival if drug A increases survival for at least ONE person relative to drug B.

I think this is a minimal requirement.  If it is not true that drug A increases survival for at least one person relative to drug B, then drug A certainly does not cause an increase in survival.

If we observe the proportion of patients who survive on drug A and the proportion of patients who survive on drug B, can we determine if drug A causes an increase in survival?  

No.  We don't have enough information.

If more patients who take drug A live longer than patients who take drug B, we cannot (without more information) determine why they lived longer and whether it had anything to do with drug A.

What is the minimum amount of information we would need to determine if drug A causes greater survival?  The answer is somewhat technical, but there are two cases where we could make the determination. 

The first case is where we observe patient survival for a representative group of patients who only had drug B available.  For example, prior to 2004 stage III colon cancer patients received 5FU as adjuvant therapy as oxaliplatin had yet to be approved by the FDA.

The second case is where we are willing to assume that patients are assigned to the drug for which their survival probability is higher.  This is what is called a "behavioral" assumption.  Such assumptions are relatively common in economics, but generally frowned upon outside of economics.

In both of these cases it is possible to determine whether drug A causes greater survival relative to drug B by simply looking at the probability of survival between those patients that take drug A and those patients that take drug B.

It is not necessary to have some ideal randomized control trial if we are able to observe survival for a group of representative patients who had no access to drug A or if we are willing to assume that patients are not assigned to the drug that is more likely to kill them.