{"id":79200,"date":"2016-01-06T12:22:43","date_gmt":"2016-01-06T17:22:43","guid":{"rendered":"https:\/\/www.bu.edu\/sph\/?p=77704"},"modified":"2020-09-17T10:28:20","modified_gmt":"2020-09-17T14:28:20","slug":"differences-in-meta-analyses-conclusions-depend-on-methods","status":"publish","type":"bu-article","link":"https:\/\/www.bu.edu\/sph\/news\/articles\/2016\/differences-in-meta-analyses-conclusions-depend-on-methods\/","title":{"rendered":"Differences in Meta-Analyses\u2019 Conclusions Depend on Methods"},"content":{"rendered":"\n
\n\t
\n\t\t\t\t\t
January 6, 2016<\/div>\n\t\t\n\t\t\t\t
\n\t\t\t\t\t<\/div>\n\t\t
\n\t\t\tTwitter<\/span><\/span>\n\t\t\tFacebook<\/span><\/span>\n\t\t\t<\/span>\n\t\t<\/div>\n\t<\/div>\n<\/div>\n\n\n

\"stack-booklets\"Meta-analysis\u2014a research method that aggregates multiple studies on a given topic\u2014is a powerful tool in public health. But the way such analyses are conducted can influence their conclusions, a team of School of Public Health researchers has found.<\/p>\n

In an article published in\u00a0PLOS ONE<\/em><\/a>, researchers led by Christopher Gill<\/a>, associate professor of global health, found that meta-analyses conducted via the Cochrane Collaboration, which follow a standardized set of methods, produced different results than non-Cochrane meta-analyses on the same topics. In addition, the overlap among studies used in the two kinds of reviews was \u201csurprisingly low,\u201d raising questions about whether the two methods had different criteria for inclusion.<\/p>\n

\u201cOur results indicate a substantial divide between the Cochrane and non-Cochrane literature,\u201d Gill said. \u201cWhile our analysis covers a very small fraction of the vast body of work that comprises the meta-analytic literature, readers should be aware that the two types of meta-analyses are not synonymous\u2014and that in some cases, the discrepancies could lead to fundamentally different conclusions about whether a given intervention is effective or not.\u201d<\/p>\n

The study found that non-Cochrane reviews reported significantly higher effect sizes of interventions, with lower precision\u2014a systemic difference that indicates that such reviews \u201cmay be overstating the evidence,\u201d Gill said. Meta-analyses conducted via the Cochrane Collaboration adhere to strict methodological and reporting standards that aim to minimize bias, maximize transparency, and improve the accuracy of summarized data, the authors said.<\/p>\n

Gill and colleagues conducted a \u201cmatched-pair\u201d analysis of 40 meta-analyses related to cardiovascular health, identifying pairs that addressed the same interventions and outcomes. The two sets were similar in terms of publication dates, how many studies were included, and average sample size. But of the 344 studies included in the paired reviews, only 129 (37.5 percent) were included in both kinds of analyses\u2014and only 2 of the 40 matched pairs included exactly the same studies.<\/p>\n

Overall, 37.5 percent of the pairs had discrepant results\u2014a finding that the authors called troubling.<\/p>\n

\u201cA number of reviews reported markedly different effect sizes, or reported effect sizes that contradicted each other (as from a protective effect to a null or harmful one, or vice versa). These are all instances where the bottom line interpretation by a reader of a review could differ qualitatively,\u201d they wrote.<\/p>\n

Gill said that, because meta-analyses are important tools in clinical research, there is a need for such analyses conducted outside of the rigorous Cochrane Collaboration. Still, \u201cit is concerning when two meta-analyses addressing the same question, within a similar time frame, reach different conclusions,\u201d he said. \u201cHow should the average doctor or health policy maker react when two \u2018gold standards\u2019 disagree with each other? That is the question we have to think about.\u201d<\/p>\n

Co-authors on the study included Michael LaValley<\/a>, professor of biostatistics; doctoral students Alana Brennan and Omid Ameli; and recent MPH graduates Johanna Useem, Michelle Vickery, and Nichole Reinen.<\/p>\n

\u2014Lisa Chedekel<\/a><\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

“Substantial divide” between two methods produces different results.<\/p>\n","protected":false},"author":8472,"featured_media":77719,"comment_status":"closed","ping_status":"open","template":"","meta":{"bu_prepress_billboard":"","_bu_prepress_primary_term":"","_bu_prepress_primary_term_manual":""},"tags":[2428],"bu-publication":[3516],"sphnews-article-category":[3519,3521,3531,3532,3540],"sphnews-topic":[],"bu_edition":[],"media_type":[],"profile_tax":[467],"_links":{"self":[{"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/bu-article\/79200"}],"collection":[{"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/bu-article"}],"about":[{"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/types\/bu-article"}],"author":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/users\/8472"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/comments?post=79200"}],"version-history":[{"count":4,"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/bu-article\/79200\/revisions"}],"predecessor-version":[{"id":174340,"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/bu-article\/79200\/revisions\/174340"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/media\/77719"}],"wp:attachment":[{"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/media?parent=79200"}],"wp:term":[{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/tags?post=79200"},{"taxonomy":"bu-publication","embeddable":true,"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/bu-publication?post=79200"},{"taxonomy":"sphnews-article-category","embeddable":true,"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/sphnews-article-category?post=79200"},{"taxonomy":"sphnews-topic","embeddable":true,"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/sphnews-topic?post=79200"},{"taxonomy":"bu_edition","embeddable":true,"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/bu_edition?post=79200"},{"taxonomy":"media_type","embeddable":true,"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/media_type?post=79200"},{"taxonomy":"profile_tax","embeddable":true,"href":"https:\/\/www.bu.edu\/sph\/wp-json\/wp\/v2\/profile_tax?post=79200"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}