Confounded by Confoundedness

Data based on the Big Mac index by
The Economist -magazine January 2012.

Over the last few weeks I have been confounded by a couple of articles on the health effects of diet.  

One article presented the results of a large diet survey where the health outcomes such as death were matched to the participants who filled out the diet survey.  The article suggested that various healthy diet choices led to major positive health outcomes like not dying.  The article didn't discuss the obvious concerns and skepticism we should have about such a study, on the contrary, the article interviewed various people who said the results bolstered policy recommendations such as food pyramids.

The second article was from a New York Times blogger, George Johnson.  Johnson was attending the American Association of Cancer Research conference and noted that there was little work in which large diet surveys were matched with health outcomes at the conference.  Johnson stated that the lack of interest was due to the problems of interpreting such work.

Observational studies such as large diet surveys with matched health outcomes data suffer from obvious confounding issues.  However, I'm confounded by both the unthinking acceptance of such statistical analysis and the absolute dismissal of such statistical analysis.  Isn't there some middle ground?

For anyone trained in economics the middle ground solution is to find "instruments."  Instruments are not bagpipes, rather they are observable variables for which it is plausible to make the following claim: there is some observational characteristic associated with survey participants that is associated with diet choice but not the health outcome of interest.  For example, some participants may live in France.  

When one lives in France, one tends to have access to French food and French food prices.  For example a French Big Mac is 510 calories, while a US Big Mac is 550 calories.  The US Big Mac is 8% bigger.  The US Big Mac costs $4.20, while the French Big Mac costs $4.43 according to the Big Mac Index.  So if you live in France you get to buy smaller Big Macs for more money than if you live in the US.

The French tend not to be as fat as Americans (see OECD slide here).  One reason is that they face different food choices and different food prices.  Another is that they have different preferences.  Apparently, there exist people in France who do not like Big Macs.

Economists, Aviv Nevo, Rachel Griffith and Pierre Dubois, analyzed differences in food consumption in France and the UK in a forthcoming paper in the American Economic Review.  The authors find that part of the explanation for different food consumption patterns is prices, but part is preferences.  

To the extent diet is determined by the prices faced by the survey participant, then it is likely to be unrelated to the participants preferences and other lifestyle choices like not smoking.  

If we know the geographical location of a survey participant, we can start to unravel whether diet causes cancer or whether diet and cancer are related through other lifestyle choices like smoking.