Bias Favoring Report of Positive Alcohol Brief Intervention Trials: Time to Get the Whole Truth

Brief intervention for unhealthy alcohol use has recently received a great deal of scrutiny, particularly from funding agencies that rely on the literature to shape medical decision-making. Investigators examined evidence of publication bias (i.e., publication of research findings is related to the direction and magnitude of effect) and dissemination bias (i.e., selective reporting, selective publication, and/or selective inclusion of scientific evidence in systematic reviews resulting in inaccurate conclusions) in a meta-analysis of 179 randomized clinical trials of brief intervention for unhealthy alcohol use in adolescents and young adults.

• Effect sizes were 0.14 standard deviations (SD) higher in funded studies, 0.03 SD higher in studies that had a shorter lag time between study completion and publication, and 0.01 SD higher in studies that were cited more frequently.
• Studies that were cited more frequently were more likely to have reported positive effects (odds ratio, 1.10).
• Studies with larger and positive effect sizes were published more promptly. There was no evidence that the magnitude or direction of effects were associated with location source, language, or journal impact factor.

Comments:

Despite the retrospective study design and lack of detailed examination of other potential biases, this analysis suggests that studies of brief intervention for unhealthy alcohol use do show evidence of dissemination and publication bias—biases favoring reporting of studies with bigger and more positive effects. These results underscore the importance of reporting all data—including negative studies—to most accurately inform evidence-based practice.

Jeanette M. Tetrault, MD

Reference:

Tanner-Smith EE, Polanin JR. A retrospective analysis of dissemination bias in the brief alcohol intervention literature. Psychol Addict Behav. 2014 [Epub ahead of print]. doi: 0.1037/adb0000014.