When Population Health Science Goes Wrong, Part 2.

I started this Dean’s Note two weeks ago aiming to comment on five misdirections in population health science that contribute, at least in part, to the perception of population health science as unhelpful in its contributions to an improved health landscape. I commented on our mistaken focus on individual prediction and our mistaking what might matter most in population health. I comment here on three other misdirections.

Fundamental Attribution Error

Economists consider fundamental attribution error to be “our tendency to ignore context and attribute an individual’s success or failure solely to inherent qualities.” This manifests in many ways in our daily lives. When my son’s team wins at soccer, he comments how this was because he played very well and scored two goals. That may of course has contributed, but had he scored two goals in isolation his team would not have won. He won because they won, and “they” is harder to understand than “I.” I use this parent-friendly example to illustrate an error that we make in all aspects of our life, including our efforts to understand population health.

Take, by way of illustration, popular health books. If you have a few minutes, browse any online bookseller for books on health. You will find a plethora of books, all directed to how you can improve your health, exercise more, and eat less and better in order to be healthier. All these books are essentially predicated on the notion that if you do better, you will be healthier. It is perhaps easy to poke fun at these books (even though some are very good), but we need to look no further than a massively visible program, the First Lady’s “Let’s Move!” project, to understand the national discourse on childhood obesity. This program, informed by our population health science, aims to get everyone moving, to encourage all of us to exercise more in order to reduce obesity levels.

But would this work at the population level? Or, put another way, will increasing the prevalence of exercise in the population reduce the risk of obesity in the population? The perhaps counterintuitive answer is, “no.” Why? The reason, tied to our discussion in Part 1 of this Dean’s Note, is that understanding the prevalence of exercise tells us very little about obesity without understanding environment. Therefore, increasing the prevalence of exercise in the population is not particularly relevant to reducing the risk of obesity; it is impossible to predict who is going to develop obesity without understanding the environmental drivers of obesity. This underlies our core health fundamental attribution error. Your health is a function of your behavior and your environment, and, appreciating that the latter can shift the health of populations while the former may not, a “what matters most” perspective suggests that we are erring in attributing the production of health to individuals. We are likely to be much more successful attributing health to environments and focusing population health thinking on those factors that shape health in populations. A mathematical illustration of this concept is found here.

Compelling Ideas, Doubt, and Certitude

Population health is a pragmatic science, and what we do is motivated by a desire to intervene to improve the health of populations. It is therefore not surprising that we work hard to identify opportunities for action that emerge from our work. This is all appropriate and in keeping with our aspirations. The challenge, however, comes when we are swept away with compelling ideas for which there is limited evidence, when we adopt those ideas without due skepticism, and when we invest them with a certitude that transcends the evidence.

As I first discussed in Part 1, Nina Teicholz’s core argument is with policy decisions that rest on data that does not stand up to scrutiny. There are, unfortunately, many other examples where we can observe this phenomenon in population health—where compelling ideas sweep us forward beyond the evidence and compel us to take positions that do not stand the test of time. Unfortunately, as Teicholz’s very public rebuke of the field shows, taking these positions is not cost-free.

My colleagues and I have been engaged around issues similar to the ones raised by Teicholz, principally around the controversy surrounding salt. As we wrote then, “For more than four decades, starting in the late 1960s, a sometimes furious battle has raged among scientists over the extent to which elevated salt consumption has adverse implications for population health and contributes to deaths from stroke and cardiovascular disease. Various studies and trials have produced conflicting results. Despite this scientific controversy over the quality of the evidence implicating dietary salt in disease, public health leaders at local, national, and international levels have pressed the case for salt reduction at the population level.” The Director of the Centers for Disease Control asserted that 100,000 deaths a year could be attributed to excess sodium, and many public health agencies launched efforts to reduce sodium intake in populations. In an article in Health Affairs, my colleagues and I explored the development of this controversy. We found that science has invested substantial energy in trying to come to an “answer” in this case, when in fact there seems to be very little genuine data-driven consensus. A subsequent Institute of Medicine report affirmed this observation.

Why did this happen? I would suggest that this case adds to the body of cases where we have based policy on compelling ideas we would like to think work because they suggest concrete action. The Oxford American Dictionary defines “compelling” as “evoking interest, attention, or admiration in a powerfully irresistible way; not able to be refuted; inspiring conviction,” and it is this conviction that gets us in trouble. It would be good to believe that by reducing cholesterol in populations we can reduce cardiovascular disease in populations. But we probably cannot, because individual dietary cholesterol is a terrible predictor of the likelihood of that individual’s having heart disease, as amply demonstrated in Framingham data and in Figure 1 in Part 1 of this Dean’s Note.

Hence our current conundrum about dietary recommendations. We can imagine how we can lower salt consumption, therefore we invest public health energy in it. But if salt consumption in populations is not associated with better population health (or in fact is potentially associated with worse health), we are here tying ourselves to ideas that are not based on data. In the process, we are compromising both our standing as population health scientists and our very core purpose: improving the health of populations. Christopher Martyn notes, “No matter how hard you try to guard against it, there is always a tendency to require a higher standard for evidence that challenges your prejudices than for evidence that supports them.” That is unfortunately always the case and suggests that we need to hold ourselves to a higher standard—a standard of skepticism about our data to make sure that we are not propelled by compelling ideas that mislead us and the public, chipping away at our credibility and utility.

Attributing Causes

The fifth misdirection, building on the previous four, rests on our attribution of the production of health to causes we understand and think we can act on. This is perhaps a challenge as old as public health itself. In a fascinating paper, Christopher Hamlin tells the tale of a mid-19th century argument between two of the founders of modern public health—Edwin Chadwick and William Farr—about whether starvation was a “cause” of death in England at the time. Farr argued that it was, Chadwick that it was not. Chadwick “won,” largely because the politics of the time did not allow for the notion that an “advanced” society such as mid-19th century England could countenance starvation killing its citizens. At heart, Chadwick and Farr’s arguments were arguments about causal architecture—about the network of causes that influences health and about which of these causes one attributes death to. While in the rest of this Dean’s Note I have talked about causal attribution as a mathematical, perhaps near abstract concern, attribution of causes has real political import. Insofar as politics are about the allocation of resources, we as a society allocate resources to what we think kills us. Therefore, if we think that the cause of death is heart disease, we may put money into the study of heart disease, which may lead to endless studies looking for genes that cause heart disease. If we think that the cause of heart disease is smoking, we may put money into the study of smoking. If we think that the cause of heart disease is poverty, we may devote resources to studying and acting on poverty. Clearly a sophisticated reader recognizes that these causes are all interrelated and are all component causes of heart disease. True, but policy decisions are made on simple and focused approaches, not on complex causal diagrams, suggesting that those of us whose business is population health science need to focus on clear attribution of causes to the factors that matter most. A concrete example of this rests in the agenda-setting work of J. Michael McGinnis and William Foege in the mid-1990s, suggesting that the “actual causes of death” in the US were behavioral causes such as smoking and dietary intake (this work was replicated in the mid-2000s by Ali Mokdad and colleagues). Our research group’s work used these same methods to show that through similar logic that approximately 245,000 deaths in the US were attributable to low education, 176,000 deaths were attributable to racial segregation, 162,000 deaths were attributable to low social support, 133,000 deaths were attributable to individual-level poverty, 119,000 deaths were attributable to income inequality, and 39,000 deaths were attributable to area-level poverty in the year 2000. These numbers are roughly equivalent to the deaths attributed to acute myocardial infarction, stroke, lung cancer, chronic lower respiratory disease, unintentional injury, and renal failure, respectively.

Does it matter if we think that death is caused by myocardial infarction, smoking, or low education? It certainly does. The Estimates of Funding for Various Research, Condition, and Disease Categories (RCDC) report shows that approximately $320 million was spent by the federal government on “smoking and health” in FY 2014. Approximately $3.6 billion was spent in the same period on research in the “behavioral and social sciences” category. None of the funded work in the database has the word “income” in the title. It matters, therefore, that we point to a causal attribution that shifts resources in the direction of evidence that stands to improve the health of populations and away from areas that may be compelling but built on shaky data, and that focus on misattribution of causes. In other words, away from what matters.

Conclusion

Let’s come back to the piece by Nina Teicholz. Teicholz does little, unfortunately, to rise above the fray on the particular issue she engages, concluding essentially that we should go back to “what worked better for previous generations”—suggesting, of course, that it may be best to simply embrace practices that are based on no data at all. While this prescription is somewhat simplistic, it bespeaks a frustration among the general public with the state of population health scholarship. The reasons above suggest plenty of motivation for this frustration and amply show that our continuing indulgence in these misdirections is not cost-free. Rather, these misdirections have us going down a wrong path and losing the confidence of the public, a trust that will be hard to earn back. Doing so will require a thoughtful reexamination of our methods and approaches in population health science. McGinnis and Foege argue that “one of [our] most difficult challenges is to ensure that the urgent does not crowd out the important. In health the challenge is especially difficult because urgent matters can be so riveting.” I agree. I would suggest that dealing with these misdirections is one of our most important challenges in the field.

I hope everyone has a terrific week. Until next week.

Warm regards,

Sandro

Sandro Galea, MD, DrPH
Dean and Professor, Boston University School of Public Health
@sandrogalea

Acknowledgement: Parts 1 and 2 of this Dean’s Note were based in part on a body of work developed in partnership with Katherine Keyes.

Previous Dean’s Notes are archived at: https://www.bu.edu/sph/category/news/deans-notes/