Drug Increases Survival in Averaged Cancer Patient!

Lake Wobegon where all the children are above average.

You will never see the above heading in a newspaper article on a new break through drug.  This may be unfortunate, because it one of the more accurate headlines you will read.

In a comment to this post, Bill provides an example of a drug that decreases survival for 99% of patients by 1 month and increases survival for 1% of patients by 200 months.

If you punch that into Google you get that the average treatment effect (Rubin's typical causal effect) is an increase in survival of 1 month (approximately).  So while 99% of patients are made worse off by the drug, the 1% of patients do so well that they pull up the average so that Bill's "crappy" drug comes out smelling like roses.

While Bill's example is an extreme, the point is more general.  The average treatment effect does not provide evidence on how the treatment will effect individual patients.  If a particular treatment increases average survival by 30 percentage points for half the patients and decreases survival by 10 percentage points for the other half of patients we will find that the average increases in survival is 20 percentage points.  A treatment that increases survival by 20 percentage points is a major breakthrough in cancer research.

In their analysis of 5-Fu as an adjuvant treatment for Stage III colon cancer, Moertel et al (1990) find that 5-Fu is associated with a 20 percentage point increase in the probability of survival at 4 years.  The authors recommend that the treatment be provided to all patients.  We see that in this post, the treatment effect for older patients is vastly different than for younger patients.  Older patients are associated with a large treatment effect, while younger patients see a small or non-existent treatment effect.