Building a Better Dataset
Kathryn Zeiler conducts a large-scale empirical study of medical malpractice insurance premiums with a grant from George Mason University’s Law & Economics Center.
Nancy Barton Scholar and Professor of Law Kathryn Zeiler is no stranger to empirical studies. With a JD from the University of Southern California Law School and a PhD in economics from the California Institute of Technology, Zeiler regularly applies economic theories and empirical methods to legal issues.
As a professor in BU Law’s highly ranked health law concentration, Zeiler is interested in how legal rules impact health-related outcomes, like the price of physician services or how many people are able to obtain health insurance. Her current project is an empirical analysis of state statutory tort reforms using her own data, collected with the help of a $105,000 grant from George Mason University’s Law & Economics Center.
States that experience high medical malpractice insurance premiums will often enact statutes that limit the amount in damages claimants may seek, with the goal of stabilizing the medical malpractice insurance market. “The conventional wisdom is that the liability system is out of control,” Zeiler says. “Politicians regularly claim that juries hand down outsized awards and that the liability system is plagued by uncertainty and errors. They point to these factors as the cause of high prices in medical malpractice insurance markets.” The assertions don’t end there. Zeiler notes claims made about physicians leaving states with unstable markets, putting the physician supply at risk. The trend is especially troubling in obstetrics and neurosurgery, areas that are high risk and high cost no matter what the insurance market dictates.
“A number of studies have been published that attempt to estimate the impacts of damages caps on medical malpractice insurance premiums to determine if the caps are working in the way state legislatures expect them to,” Zeiler says. In a 2013 review of the existing literature entitled “Do Damages Caps Reduce Medical Malpractice Insurance Premiums?: A Systematic Review of Estimates and the Methods Used to Produce Them,” published in 2013 in the Research Handbook on the Economics of Torts, Zeiler found that the published research relied on studies with severe limitations. “Many datasets are based on aggregate premiums collected by state, which makes it impossible to separate changes in the quantity of insurance policies sold from changes in price. Other datasets include pricing for only one policy size, typically policies that cover $1 million per occurrence and $3 million annually.”
Zeiler’s study will join the conversation as the 17th study of the impact of damages caps on medical malpractice premiums. The GMU grant allows her to build an original dataset, collecting medical malpractice insurance pricing data from state filings by county, policy size, year, insurer, and provider type.
Once collected, Zeiler and her coauthor, Michael Frakes of Northwestern University Law School, will use those data to employ state-of-the-art empirical methods to estimate the impacts of caps on malpractice premiums. “Once the dataset is complete, we will use it to generate the most comprehensive and accurate estimate to date of the impact of medical malpractice damages caps on insurance prices,” Zeiler says.
Improvements to empirical methods are concentrated on minimizing the effects of studying data in the field rather than in a controlled laboratory environment. “In a perfect world, we could randomize which states impose damages caps and which don’t,” Zeiler says. “Our inability to do that leads to problems with drawing causal inferences from observed outcomes, like the correlation between insurance prices and tort reform. One of the things we are trying to improve with this study is the mechanism to control for the possibility of selection bias in the study sample. Observed spikes in medical malpractice premiums lead us to believe that states that opt to impose caps are not random, as you would want them to be in a lab experiment.”
In her study, Zeiler is using a common statistical estimator called “difference-in-differences,” in which medical malpractice insurance premiums in states that impose caps, the treatment states, are measured at specific times before and after caps are imposed. Premiums are also measured at those same times in control states, in which caps are not imposed. Difference-in-differences then estimates the average change in premiums in treatment states had those states not imposed caps. The difference between the expected premium change and the observed premium change is the difference-in-differences that Zeiler is after.
“This is one method that researchers use to mimic a laboratory experiment using observational field data,” Zeiler says. “An important assumption missed in previous studies is that the control states are actually useful controls. This requires pre-treatment trends in control states to follow those in treatment states. Only when this assumption is met can we feel confident that the control states help us estimate necessary counterfactuals.” In her 2013 review of the existing studies, Zeiler noted that not a single study had reported the results of this check, leading her to question the results of those studies and the analyses based on them.
With data collection ongoing, the timeline for Zeiler and Frakes’s analysis is undetermined, although they hope to begin work-shopping preliminary drafts in spring 2016. The importance of the study, however, and its potential to impact health policy, is not lost on Zeiler. “At a time when the country is focused intensely on health care costs and access, our ability to accurately estimate the direct and indirect impacts of the liability system is essential,” she says. “A more comprehensive dataset of medical malpractice insurance premiums and the use of state-of-the-art empirical methods will allow us to take a big step forward in better understanding the actual impacts of the liability system.”