Leveraging Multiple Outcomes in Matched Observational Studies (Colin B. Fogarty -- University of Pennsylvania)

Abstract: In order to bridge the gap between association and causation in observational studies, Fisher advocated for the testing of “elaborate theories.” One manner in which a causal theory can be made elaborate is through the prediction of a particular direction of effect for multiple outcome variables. When testing hypotheses on multiple outcomes, multiple comparisons must be taken into account. This is true not only when assuming no unmeasured confounding, but also when assessing how robust a study's findings are to unmeasured confounding in the subsequent sensitivity analysis. Concerns over a loss in power may lead practitioners to instead investigate the outcome variable they believe *a priori* will be most affected by the intervention, thus reducing the extent to which Fisher's advice is followed in practice. We demonstrate that when performing multiple comparisons in a sensitivity analysis, the loss in power from controlling the familywise error rate can be attenuated. This is because unmeasured confounding cannot have a different impact on the probability of assignment to treatment for a given individual depending on the outcome being analyzed. Existing methods for testing the overall truth of multiple hypotheses allow this to occur by combining the results of sensitivity analyses performed on individual outcomes. By solving a quadratically constrained linear program, we are able to perform a sensitivity analysis while avoiding this logical inconsistency. We show that this allows for uniform improvements in the power of a sensitivity analysis when compared to combining individual sensitivity analyses. This is true not only for testing the overall null across outcomes, but also for testing null hypotheses on specific outcome variables when using certain sequential rejection procedures. We illustrate our method through an example examining the impact of smoking on naphthalene levels in the body.