 |
| Mutt and Jeff (June 3 1942) |
Rubin (1974) argues that we should prefer randomized control trials (RCT) to observational data because the "casual effect" of the treatment is measured from the RCT data under mild assumptions.
Rubin defines the "causal effect" of a new treatment as the difference between the outcome of the patient when she is given the new treatment and the outcome of the patient if she had received the current standard of care. Rubin acknowledges that the difference is not observed and is in fact unobservable. A patient can only ever receive one treatment and so it is not possible to observe the outcome in two alternative treatments.
Like the man in the top hat, Rubin suggests looking for the information in the light. In Rubin's case the "light" is provided by the RCT which measures the average treatment effect under mild assumptions. Rubin argues that average treatment effect is a measure of the difference in outcomes for the "typical" patient. If we take "typical" to mean that it is true for some reasonable sized group of patients, then there is no reason to believe that the "typical treatment effect" will even have the same sign as the average treatment effect. The average treatment effect averages over the difference in outcomes for each of the patients. If some patients benefit from the treatment and some patients are harmed by the treatment then the average treatment effect may be positive or negative depending on the relative sizes of the two patient groups and the relative sizes of the benefits or harms.
If the average treatment effect is positive then we know for certain that there exists one patient for whom the new treatment was better than the existing treatment. That is it. That is all we know for certain. It may be that all patients are better off with new treatment or it may be that (almost) all patients are worse off with the new treatment. The average treatment effect is observed by the light of the RCT but it tells us very little about what we are looking for.
There is an assumption that as long as the two groups are the same in important parameters and the sample size is sufficiently large that differences between groups will reflect reality. Unfortunately those in trials are not a representative sample of the patient population. Experience after an agent is marketed, if approved, probably will not be the same as in the trial. What is needed is a way to better identify who might and might not respond so that many who can not respond will not be exposed to toxicity without any chance of benefit. It is critical that markers be found that identify potential responders and clearly eliminate nonresponders.
ReplyDelete