Quality Improvement in Addiction Medicine: When Is It Research?

Mary-Tara Roth, RN, MSN, MPH

Director, Clinical Research Resources Office, Boston University Medical Center, Boston, MA, USA

Patients with substance use disorder (SUD) can benefit from research advances,¹ but there is often a delay between the establishment of evidence of benefit and the implementation of an improved practice.^2,3 These advances can lead to novel therapies and treatments as well as changes at a programmatic level, such as improving processes to increase the number of patients who can be treated and retained in treatment. Once new treatments and practices are implemented, important questions remain: Are they beneficial, in this program, within this patient population? Have the changes led to the desired and expected outcomes? Are there any undesirable associated outcomes? Further, implementation of certain practices may need to be adjusted to conform to a specific treatment program or population. Thus, evidence-based new treatments, programs, and practices should be assessed to ensure that they deliver the expected benefits. This process of implementation and assessment of evidence-based practices to improve health care and delivery of care is known as Quality Improvement (QI).

Since QI involves the initiation and assessment of a change, and because results may be applicable to settings outside the local organization (and even published), it is common for those performing these initiatives to be concerned as to whether their QI projects also qualify as human subjects research. It can be difficult to illuminate the gray space between QI and research,^4,5,6 as evidenced by a well-publicized and controversial Office of Human Research Protections (OHRP) ruling that a multicenter project involving implementation of a simple 5-item checklist to ensure proper infection control when inserting a central venous line met the regulatory definition of human subjects research.^7,8,9,10 This article will review the distinctions between QI and research, including differentiating factors, and what steps must be taken to ensure that a project is carried out ethically and in compliance with applicable regulations.

The Quality Improvement/Research Distinction

The difference between QI and research is not mundane, and it is significant for both ethical and practical reasons. From an ethical and regulatory perspective, QI that is also considered to be research involving human participants must follow the Department of Health and Human Services (DHHS) Human Subjects Protection regulations under the OHRP.^11,12 This means that it must receive prospective approval from an Institutional Review Board (IRB) prior to starting, and obtain informed consent from the study participants (or obtain approval for a waiver of consent, which applies in some cases of minimal-risk research). Practically speaking, if one goal of the project is to publish the results, it is also important to know that the International Committee of Medical Journal Editors (ICMJE) requires research involving human subjects to describe IRB review and consent.¹³ So, if a QI project is also considered to be research and it hasn’t met these requirements, it cannot be published in journals following ICMJE standards.

What is Quality Improvement?

A recent expert working-group defined QI in health care as “systematic, data-guided activities designed to bring about immediate improvements in health care delivery in particular settings.”¹⁴ Interventions that are known to be efficacious are put in place to benefit the patients who receive them. This is unlike research, where interventions tested are not necessarily known to have benefits (this is what the research is testing), and although the participant may have some prospect of direct benefit from participation, the main goal of research is not to benefit the research participants, but to attain new knowledge and to generalize it to help future patients.

Multiple reports have underscored the importance of QI within a health care system.^15,16 Unlike research, QI is a necessary part of the provision of good medical care,^5,14,17 and part of the hospital’s ethical duty to provide patients with the best care. Some claim that institutions have a “moral imperative” to perform QI,¹⁸ and ongoing QI efforts are expected by certifying agencies such as the Joint Commission on Accreditation of Healthcare Organizations and the National Committee for Quality Assurance.¹⁷

What is Research?

One key to understanding whether a QI project is also research is to determine if the project meets the definition of research, under the DHHS Protection of Human Subjects Regulations (45 CFR 46).¹⁹ In order to determine whether a project is considered to be research in need of IRB approval, one must first ask: Is it research? If so, does it involve human subjects?

The DHHS Protection of Human Subjects regulations define research as: “[…] a systematic investigation […] designed to contribute to generalizable knowledge […]” (45 CFR 46.102 [d]).¹¹ Both parts of this regulatory definition must be in place for a given activity to be considered research. First, research involves systematic processes (such as data collection based on pre-defined datapoints). Second, the main purpose, or the intent of the investigation (what it is designed to do), is to generalize that knowledge. Most QI projects involve techniques and processes that satisfy the first part of this definition, such as systematic implementation/intervention, data collection, and analysis. Thus, this part of the regulatory definition of research is usually not helpful in making a distinction between QI and research.

The second part of the regulatory definition of research can be more instructive. True QI-only projects have as their goal to improve systems of care for current patients at the local institution. The changes are made to benefit these patients; they are not put in place with the objective of testing them to determine whether they might benefit patients elsewhere. Although some argue that the intent (i.e., designing the project this way) to generalize the knowledge from a project should not be the primary criterion to distinguish QI and research,²⁰ without a change in the current regulatory requirements, individuals doing QI and research will have to make judgments based on the best interpretation of the available and applicable regulations and guidance on human subjects research. It should be noted that true QI-only projects may not be designed to contribute to generalizable knowledge, but can contribute nonetheless. The key is what the project is designed to do.

Further Clarification of Generalizable Knowledge

Intent at the onset and extent of what is known (intent to sustain improvements)

To clarify whether a project is QI-only or QI plus research, it is helpful to appraise the intent of a project, as well as the extent of what is known about the intervention. In regards to intent, if a project is put in place to improve care for current patients, there must be some agreement by those with the authority to mandate changes within the organization that the changes are a) being made to improve care; b) feasible (though this is in part what might be assessed in the QI efforts); and c) sustainable, assuming that there are positive results from the QI efforts.¹⁷ For example, suppose a medical resident undertakes a project as QI and initiates an intervention and data collection to improve physician adherence to state guidelines for prescribing opioids for chronic pain and assessing for risk of opioid use disorder and aberrant behaviors. Once that project is done, and the medical resident moves on to a new rotation, will that effort continue to be in place? If not, and if the intervention was not something that was vetted and authorized by those with the authority to make sustained changes, then one must question whether the primary intent of implementing the change was really to improve care at the local level.

Another aspect to consider is the extent of what is known about the intervention or improvement. The implementation of a new health practice to improve care should be supported by sufficient evidence that it will be successful. Implementing a new health practice that has little available evidence to support its success should be considered to be research and implemented only after obtaining the necessary approval from the IRB.

Does publishing mean that the findings are generalized?

Often the first question people ask when considering whether they should submit their project to the IRB is whether they plan to publish the results in a scientific journal. After all, publishing is a way of generalizing research results. OHRP Frequently Asked Questions (FAQ) guidance²¹ notes that publishing “an account of a quality improvement project does not necessarily mean that the project fits the definition of research; people seek to publish descriptions of nonresearch activities for a variety of reasons, if they believe others may be interested in learning about those activities […]” In fact, some experts advocate that publishing QI is essential as it provides documentation of evidence, allows scrutiny, and facilitates dissemination of improvements.²² However, when planning to publish a QI activity, one should be careful to ensure that the manuscript is clear about the intent of the project. It should not be described as research, and results of the QI project should not be generalized beyond the local setting.

Further illuminating the gray area

Lynn et al¹⁴ advise that if a QI effort is “designed to produce both local improvement and new, enduring knowledge,” (p. 671) then it should be considered an “overlap project” with human subjects research. They list five characteristics of QI that overlaps with human subjects research:

1. Testing of issues that go beyond current knowledge
2. Random allocation of patients to enhance confidence in differences (rather than for equitable allocation of a scarce resource)
3. Deliberately delayed or ineffective feedback of data from monitoring the implementation to avoided bias in data interpretation
4. Involvement of researchers who have no ongoing commitment to improve care at the local level
5. Funding by parties outside the clinical setting

Reinhardt and Ray⁵ list 4 additional criteria that can be used to determine whether a project is QI-only, QI plus research, or research only. Meeting one of these criteria means that the project should be considered research, requiring prospective submission to the IRB:

1. Intervention: Is the intervention an accepted practice that is a new implementation within an organization, or is it a new, untried practice, or one with a little evidence to support its effectiveness and safety?
2. Risk: Is there absence of risk beyond the standard of practice or is there presence of any risk beyond the standard of practice?
3. Audience: Is the primary audience the organization or is it the scientific community and consumers?
4. Data source: Does the project include data from one single organization or does it include data from multiple organizations?

And what about randomization? Does randomization push the project into a research realm? Lynn et al¹⁴ touch on this topic in their list, detailed above. Programs with limited resources may need to demonstrate that QI initiatives are cost-effective before implementing on a program/institution-wide scale. In this case, the QI effort could be introduced on a limited basis, possibly through random assignment (individual or group level) of the improvement. In this way, feasibility and effectiveness can be assessed by comparing outcomes of groups who received and did not receive the improvement to determine if the improvement should be implemented throughout the organization. Lynn et al make a distinction between randomization to enhance confidence in differences between groups and randomization for equitable allocation of scarce resources, with the former more likely to be considered in the research realm. Individuals initiating QI efforts should follow their institutional policies but can inform their thinking using the guidance criteria described above from Lynn et al and Reinhardt and Roy.⁵

Oversight

Any QI project that contains a research component will need to be submitted to the IRB and approved prior to starting. As noted earlier, the Protection of Human Subjects regulations (45 CFR 46 subpart A)¹⁹ provide some flexibility and allow for waiver of consent as well as exemption determination for some categories of minimal-risk research. Compliance with Human Subjects Protection regulations is an important step in assuring the rights and safety of research participants.

Although QI projects that are not also research do not have to be submitted to the IRB, this does not mean that they can be conducted without adherence to appropriate best practice standards to ensure that the patients are protected and that the results can be counted on to inform the local practice. That is, QI projects must adhere to quality, safety, and ethical standards similar to research, even though the review and oversight pathways will likely function differently at different institutions. Lynn et al¹⁴ suggest requirements derived from Emmanuel et al, “What makes research ethical?”²³ A QI project must have scientific value, scientific validity, fair participant selection, favorable risk-benefit ratio, respect for participants, informed consent, and independent review. Although similar to ethical standards guiding human subjects research, some differ in how they are implemented. For example, the authors do not suggest that the independent review be conducted by the IRB. In QI, review will be integrated into the same procedures that ensure accountability for clinical care. Also, consent will be different from what is required for research. According to Lynn et al, consent for QI should be integrated within the consent for clinical care, and patients should understand that their consent to care is also consent to participate in minimal-risk QI activities. On the other hand, it should be noted that multiple authors support the systematic ethical oversight of QI projects—akin to but separate from the IRB.¹⁷ Individuals involved in QI efforts should check the applicable policies of their institutions.

Conclusion

It is important for those involved in carrying out QI within addiction medicine to know that there is often an overlap of QI and research, and that the primary intent of the project can determine whether the project is considered to be research or not. Some QI projects also involve research and will require submission and oversight from an IRB. However, all QI projects should be carried out in an ethical manner.

References

1. Hunter SB, Ober AJ, Paddock SM, Hunt, PE and Levan D. Continuous quality improvement (CQI) in addiction treatment settings: design and intervention protocol of a group randomized pilot study. Addiction Science & Clinical Practice. 2014;9:1-11.
2. Power EJ, Nishimi RY and Kizer KW, eds. Evidence-based treatment practices for substance use disorders: Workshop proceedings. National Quality Forum. 2005. Available at: http://www.apa.org/divisions/div50/doc/Evidence_Based_Treatment_Practices_for_Substance_Use_Disorders.pdf
3. Curran CR, Totten MK. Governing for improved quality and patient safety. Nursing Economics. 2011;29(1):38-41.
4. Grady C. Quality Improvement and Ethical Oversight. Annals of internal Medicine. 2007;146:680-681.
5. Reinhardt AC and Ray LN. Differentiating Quality Improvement From Research. Applied Nursing Research. 2003;16(1):2-8.
6. Casarett D, Karlawish JHT, Sugarman J. Determining when quality improvement initiatives should be considered research: proposed criteria and potential implications. JAMA. 2000;283:2275-2280.
7. Kuehn BM. DHHS Halts quality improvement study: Policy may hamper tests of methods to improve care. JAMA;299(9):1005-1006.
8. Pronovost P, Needham D, Berenholtz S, et al. An intervention to decrease catheter-related bloodstream infections in the ICU. NEJM 2006;255;26 2725-2732.
9. Gawande A. The Checklist. The New Yorker. 2007;Dec 10. Accessed at http://www.newyorker.com/magazine/2007/12/10/the-checklist May 4, 2010.
10. Office of Human Research Protections (OHRP) Determination letter Nov 6, 2007. Available at: http://www.hhs.gov/ohrp/detrm_letrs/YR07/nov07c.pdf
11. Office of Human Research Protections. 45 CFR 46.102(d).
12. Office of Human Research Protections. 45 CFR 46.102(f).
13. ICJME Protection of Research Participants; available at: http://www.icmje.org/recommendations/browse/roles-and-responsibilities/protection-of-research-participants.html
14. Lynn J, Baily MA, Botrell M et al. The Ethics of Using Quality Improvement Methods in Health Care. Annals of Internal Medicine. 2007;146:666-673.
15. Kohn LT, Corrigan JM, Donaldson MS, eds. To err is human: Building a safer health system. Committee on Quality of Health Care in America, Institute of Medicine. Available at: http://www.nap.edu/catalog.php?record_id=9728#toc
16. Committee on Quality of Health Care in America, Institute of Medicine. Crossing the quality chasm: A new health system for the 21st Century. 2001. Available at http://books.nap.edu/openbook.php?record_id=10027&page=1
17. Bellin E, Dubler, NN. The quality improvement – research divide and the need for external oversight. American Journal of Public Health. 2001;91(9):1512-1517.
18. Secretary’s Advisory Committee on Human Research Protections (SACHRP), March 27 and 28, 2008 Meeting Minutes. Available at: www.hhs.gov/ohrp/sachrp/mtgings/mtg03-08/minutes.html
19. US Department of Health and Human Services: Code of Federal Regulations no. 45 CFR 46, Subpart A.
20. Kass NE, Faden RR, Goodman SN, Provonost P, Tuns S, Beauchamp TL. The Research-Treatment Distinction: A Problematic Approach for Determining Which Activities should have ethical oversight. Ethical Oversight of Learning Health Care Systems: A Hastings Center Report, January-February 2013.
21. Office of Human Research Protection (OHRP) Frequently Asked Questions (FAQ) guidance. Available at http://www.hhs.gov/ohrp/policy/faq/quality-improvement-activities/index.html
22. Davidoff F, Batalden P. Toward stronger evidence on quality improvement. Draft publication guidelines: the beginning of a consensus project. Quality & Safety in Health Care. 2005;14: 319-325.
23. Emanuel EJ, Wendler D, Grady C. What makes clinical research ethical? JAMA. 2000;283: 2701-2711.