Bad Data Science and Woody Allen

"Life imitates art far more than art imitates life." - Oscar Wilde

In Woody Allen's 1973 iconoclastic movie "Sleeper" a man (health food store owner) wakes up two hundred years in the future. For breakfast he requests wheat germ, organic honey and tiger's milk - food in 1973 thought to be healthy. The futuristic doctors reply: "You mean there was no deep fat? No steak or cream pies or... hot fudge?" and "Those were thought to be unhealthy... precisely the opposite of what we now know to be true."

A recent article in the Wall Street Journal entitled "The Questionable Link Between Saturated Fat and Heart Disease" details scientific malpractice in research about what food is healthy or not. For over fifty years the scientific consensus was that fat - both saturated or not - is a "cause" of obesity, heart disease, and other chronic diseases. Note the key word "cause".

This purported scientific consensus regarding the health and nutritional value of eating fat was used to justify government and medical guidelines regarding optimal healthy diet. Medical authorities advised avoidance of saturated fats in butter, cheese and red meat because they allegedly clog our arteries and cause heart disease - and this belief set government nutrition policy.

A recent study published in Annals of Internal Medicine entitled "Association of Dietary, Circulating, and Supplement Fatty Acids With Coronary Risk: A Systematic Review and Meta-analysis" summarized evidence about associations between fatty acids and heart disease - and concluded that "...evidence does not clearly support cardiovascular guidelines that encourage high consumption of polyunsaturated fatty acids and low consumption of total saturated fats." In other words, saturated fat is not associated with an increased risk of heart disease.

In fact, there never was strong evidence that eating fats cause disease. There may have been correlations among small sample sizes in certain populations but the evidence did not show causation. Then why did a consensus emerge against eating fat? Simply put, bad data science mixed with egos and politics.

First, confirmation bias skewed the original studies and nutritional analysis. Heart disease - previously almost nonexistent - had become the leading killer in the United States by the 1950's and society put pressure on the medical establishment to find out why and to remedy. The leading theory at the time was that high cholesterol caused heart attacks (recent research has refuted this simple theory as well - the reality is more complex and the cholesterol cure may be worse than the disease). Some experiments showed a correlation between eating saturated fats and high cholesterol (without distinguishing between different types of cholesterol measurements). Ergo, eating fats cause heart attacks. Confusing correlation with causation and confirmation bias made the results unreliable.

Second, the scientific method was violated. The study failed to select countries randomly. Countries selected included Yugoslavia, Finland and Italy - those likely to confirm prior beliefs about fat, high cholesterol and heart disease. Countries excluded included France, Switzerland, Sweden and West Germany - where folks eat significant amounts of fat yet do not suffer from high rates of heart disease. This skewed selection and confirmation bias rendered the conclusions unreliable.

Third, the study suffered from small sample size - it over-relied chiefly on data from about 35 men - much smaller than the original representative sample of 655. Small sample size made the results unreliable.

Other studies purported to confirm the original study yet also had serious methodological problems. Some didn't control for other health variables like smoking, frequency of exercise, other foods or poor nutrition. Others allowed test subjects to wander in and out of the research group during the experiment.

The above should have raised red flags among the scientific and medical communities, the American Heart Association and various federal government agencies. That it did not is evidence of confirmation bias and politics in addition to gross negligence.

Real damage ensued from this scientific malpractice: we still have a serious heart disease problem - further research into other potential causes was not done until recently; folks who excluded fats from diet may have secondary adverse consequences like alzheimers and other health problems; obesity is a serious problem - not because of fats but over-consumption of carbohydrates and sugar; and the medical establishment and government authorities have lost credibility.

In addition, the concept of "scientific consensus" has lost credibility. This is further evidence that simply having a consensus in the scientific community about something does not make it true. Honoring the scientific method means always remaining skeptical about results and remaining open to debate and dissent.