{"id":66435,"date":"2015-04-12T06:00:13","date_gmt":"2015-04-12T10:00:13","guid":{"rendered":"https:\/\/www.bu.edu\/sph\/?p=66435"},"modified":"2021-02-08T16:37:35","modified_gmt":"2021-02-08T21:37:35","slug":"when-population-health-science-goes-wrong-part-1","status":"publish","type":"bu-article","link":"https:\/\/www.bu.edu\/sph\/news\/articles\/2015\/when-population-health-science-goes-wrong-part-1\/","title":{"rendered":"When Population Health Science Goes Wrong, Part 1"},"content":{"rendered":"\n
\n\t
\n\t\t\t\t\t
April 12, 2015<\/div>\n\t\t\n\t\t\t\t
\n\t\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\tTwitter<\/span><\/span>\n\t\t\tFacebook<\/span><\/span>\n\t\t\t<\/span>\n\t\t<\/div>\n\t<\/div>\n<\/div>\n\n\n

\"deansnote1\"<\/h3>\n

I read with interest recently the New York Times<\/em> editorial by Nina Teicholz, titled \u201cThe Government\u2019s Bad Diet Advice<\/a>.\u201d The Times<\/em> also published some comments on the Teicholz piece<\/a>, including my brief thoughts. In the piece, Teicholz takes issue with dietary guidelines, many of which are based on nutritional epidemiology studies that have been reconsidered in the 2015 Dietary Guidelines Advisory Committee<\/a> recommendations (the committee chair is Barbara Millen, a longtime Boston University faculty member). In her piece, Teicholz suggests that \u201cthe primary problem is that nutrition policy has long relied on a very weak kind of science: epidemiological, or \u2018observational,\u2019 studies in which researchers follow large groups of people over many years.\u201d I am on Teicholz\u2019s side in lamenting the challenges posed by policies that turn out to be wrong. In my assessment, however, her general diagnosis\u2014and her fingering of observational epidemiology as the particular culprit\u2014is wrong and a rather narrow diagnosis for a set of complex problems.<\/p>\n

However, what is most striking is the impunity with which Teicholz fingers our quantitative methods in an op-ed page in the \u201cnewspaper of record.\u201d That suggests, of course, that the opinion expressed is, to some extent, received wisdom and the articulation of a view that the editors of the New York Times<\/em> find meritorious. What, then, do we make of the Times<\/em> allowing the expression of a core, foundational challenge to the quantitative methods of population health science? Is Teicholz right, or is she wildly off the mark? And what has brought us to a place where this perspective has such high-profile currency?<\/p>\n

I think there are multiple ways to approach these questions. The easiest might be to dismiss Teicholz\u2019s comments. Harder, but perhaps more productive, would be to see them as a real and very public challenge to the foundational approaches we take in population health science and to consider what it is we are doing that might be contributing to this perception of our approaches. I would suggest that there are five misdirections in population health that contribute, at least in part, to the perception of population health science as being unhelpful in its contributions to an improved health landscape.<\/p>\n

I will comment on the first two of these misdirections in this Dean\u2019s Note and the other three in a coming week in the interest of keeping the Dean\u2019s Notes short(er) and more readable.<\/p>\n

A Focus on Individual Prediction<\/h3>\n

Almost our entire investment in health as a society is focused on improving health in the individual\u2014on making sure we develop approaches and initiatives to improve the health of the individual. Although it is notoriously difficult to estimate this, by some calculations more than 90 percent of health spending in the United States is linked to individual health care, and less then 10 percent is devoted to efforts that aim to improve population health. In some respects, the challenge of spending priorities reflects a larger political challenge of resource allocation and of the identification of the best return on society\u2019s investments. But central to the issue at hand is that our fascination with individual health is accompanied by a growing confidence in our capacity to improve and in many ways predict it. That is, of course, the underlying principle behind the dietary recommendations that are targeted by Teicholz, suggesting that if I, the individual, can eat less of X and more of Y, I can improve my health by an appreciable Z increment. This is also the foundation of a plethora of risk prediction calculators<\/a>, starting with the Friedewald formula<\/a> that contributed substantially to putting dietary cholesterol \u201con the map.\u201d Perhaps the apotheosis of this approach can be found in the surge of interest in personalized\/precision medicine, which stands to be the foundational and organizing principle<\/a> behind much of federal health research in the coming decades. But forgotten in this approach is one simple mathematical observation: Our capacity to predict the health of individuals is not very good at all. The data that inform risk prediction models come from population-based studies, and these data are useful indeed for models fit to population-level inference. However, these data fall dramatically short when it comes to individual-level prediction. Although the guidelines are now catching up to this, it has long been recognized, for example, that while cholesterol intake at the population level is associated with higher cardiovascular disease, knowing someone\u2019s cholesterol level does very little to discriminate between whether s\/he will have heart disease or not<\/a>. This fact is perhaps most succinctly illustrated by Figure 1 below.<\/p>\n

\""Figure\"<\/p>\n

Why is this? In many ways, this is a mathematical extension of the observation that relative measures of association derived from population level data have very limited discriminant ability to identify which population\u2014one with disease, one without\u2014any individual with a particular characteristic belongs to. This point is simply illustrated in Figure 2 below, showing that only when one reaches population-derived relative measures of association in the order of 300 or so do we achieve population \u201cseparation\u201d and confidence that the person with such a characteristic is more likely than not to have disease.<\/p>\n

\"Figure
Figure 2<\/strong>
Pepe MS, Janes H, Longton G, Leisenring W, Newcomb P. Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker. Am. J. Epidemiol<\/em>. 2004; 159: 882-890.<\/span><\/figcaption><\/figure>\n

This, then, bodes terribly for individual prediction and directly results in our failure to provide the world with useful estimates of individual health linked to particular exposures\u2014the core challenge posed by Teicholz. We could argue that this suggests a need for a renewed focus on our methods in order to provide us with approaches that are indeed better at prediction than our current approaches. I endorse that. But, at core and absent such methods, we need to make sure that we are clear in our own minds about what we are able to predict, in which group, and that we are informed by suitable humility about the capacities of our methods to move us anywhere near individual prediction. In this context, I worry most about the ascent of precision medicine and its dominance of the biomedical research agenda, predicated on this focus on individual prediction. Absent an appreciation of our limits on this front, we are, societally, investing in an approach that is perhaps compelling but is not grounded in methods that can deliver on its promise.<\/p>\n

What Matters and Why
<\/strong><\/h3>\n

Over the past 30 years, population health science has been centrally concerned with understanding how particular determinants are associated with health, so that we may then work to minimize those determinants and their impact. That is ultimately what has been animating the corpus of nutritional epidemiology challenged by Teicholz, and also what has animated much of quantitative population health science over the past several decades. In defense of the approach, we can note that it has yielded public health successes<\/a>. However, this approach has also resulted in a baffling array of suggestions about what may or may not be harmful or helpful to health. Building on the referent article: Is butter good or bad for you? One can find evidence that points in both<\/a> directions.<\/a> Does green tea prevent stomach cancer, or does it not<\/a>? This list could go on and on. Why do we have such a proliferation of science that points, maddeningly, in different directions? Although part of this answer rests on the very nature of science itself\u2014we learn through trial and error, we move forward in one direction and then realize we are wrong and therefore correct ourselves\u2014I would suggest that much of the challenge lies in our methods and approaches.<\/p>\n

We owe this particular pickle in no small part to a \u201crisk factor\u201d approach to etiologic thinking in population health science. This approach has had us, for the past several decades, working ever harder to isolate causes that produce disease in an effort to intervene and change those causes. I commented on this in my Public Health Forum a few weeks ago. This effort has resulted in substantial and impressive attempts on our part to create ever better methods that control extraneous influences in an effort to identify and isolate single causes. This approach shifts us away from recognizing that rather than thinking about lone causes, we should be thinking about relative weights of health influencers so that we may intervene on that which matters most<\/em>.<\/p>\n

How may we think of what matters most? The core concept here is that the magnitude of the import of any given population health factor is dependent as much on that factor as it is on the prevalence of those other factors<\/em> that interact with the cause of interest. This might not matter very much if we believed that any one cause works in isolation. But, apart from a few rare genetic germline mutations (e.g. cystic fibrosis), vanishingly few causes do, suggesting that we must understand the prevalence of the other factors<\/em> in order to determine when intervening on a particular factor might be important. Recognizing that causation of complex diseases rests on a complex network of co-occurring causes necessitates an approach that considers these other factors if we are to determine the relative weights to give our actions.<\/p>\n

This is particularly resonant in population health science simply because we are by definition concerned with the determination of health of populations\u2014and, as I commented before, that determination is ineluctably linked to context<\/a>. Therefore, if we are to understand what matters most, we must understand how a particular factor matters in context.<\/em> This will require a dramatically different causal inferential approach as well as a mathematical appreciation of the role of context in the production of population health. This all suggests that there are mathematical limits to the extent to which any single individual cause can matter, explaining why, for example, the import of butter or fat to individual diet is inconsistent across studies\u2014 studies that vary across contexts\u2014and explaining why authors like Teicholz despair at the observations of population health science. If anyone is interested in a formal mathematical basis for this argument, send me a note and I can send a paper, currently on press at Annals of Epidemiology<\/em>, that presents this case more thoroughly.<\/p>\n

I shall discuss the three other misdirections in an upcoming Dean\u2019s Note.<\/p>\n

I hope everyone has a terrific week. Until next week.<\/p>\n

Warm regards,<\/p>\n

Sandro<\/p>\n

Sandro Galea, MD, DrPH
Dean and Professor, Boston University School of Public Health
@sandrogalea<\/a><\/p>\n

Acknowledgement. This Dean\u2019s Note and the second Dean\u2019s Note in this series are based in part on a body of work developed in partnership with Katherine Keyes.<\/em><\/p>\n

Previous Dean\u2019s Notes are archived at: https:\/\/www.bu.edu\/sph\/category\/news\/deans-notes\/<\/a><\/p>","protected":false},"excerpt":{"rendered":"

Are we pursuing misdirections in population health science that contribute to the perception that our work is not improving public health? Part 1 of 2.<\/p>\n","protected":false},"author":8472,"featured_media":59530,"comment_status":"closed","ping_status":"closed","template":"","meta":{"bu_prepress_billboard":"","_bu_prepress_primary_term":"","_bu_prepress_primary_term_manual":""},"tags":[1729],"bu-publication":[3516],"sphnews-article-category":[3519,3527,3531],"sphnews-topic":[],"bu_edition":[],"media_type":[],"profile_tax":[],"_links":{"self":[{"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/bu-article\/66435"}],"collection":[{"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/bu-article"}],"about":[{"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/types\/bu-article"}],"author":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/users\/8472"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/comments?post=66435"}],"version-history":[{"count":19,"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/bu-article\/66435\/revisions"}],"predecessor-version":[{"id":191222,"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/bu-article\/66435\/revisions\/191222"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/media\/59530"}],"wp:attachment":[{"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/media?parent=66435"}],"wp:term":[{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/tags?post=66435"},{"taxonomy":"bu-publication","embeddable":true,"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/bu-publication?post=66435"},{"taxonomy":"sphnews-article-category","embeddable":true,"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/sphnews-article-category?post=66435"},{"taxonomy":"sphnews-topic","embeddable":true,"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/sphnews-topic?post=66435"},{"taxonomy":"bu_edition","embeddable":true,"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/bu_edition?post=66435"},{"taxonomy":"media_type","embeddable":true,"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/media_type?post=66435"},{"taxonomy":"profile_tax","embeddable":true,"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/profile_tax?post=66435"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}