AHRQ 1/12/09 memo

AHRQ Health Care Innovations Exchange

Response to OMB Questions of January 12, 2009

1. Can AHRQ narrow the scope of this effort so that it creates a certain niche for itself and provides a service that is not already provided elsewhere? What is the focus here?

AHRQ does not believe the scope of this effort will be broad. Having inclusion criteria like “innovations will relate directly or indirectly to patient care” and “innovations will intend to improve one or more domains of health care quality” sets boundaries and when applying all of the inclusion criteria will actually help AHRQ limit the scope. If AHRQ provided more specificity to its criteria, it runs the risk of finding a very small number of innovations and this would mitigate the purpose of the resource. AHRQ’s experience in using inclusion criteria in other resources, like the National Guideline Clearinghouse™ (NGC), forms the basis of this belief.

AHRQ believes it is creating a niche for itself in providing a venue that captures creative solutions to health care quality improvement problems and presents them in a standardized format; some of the creative solutions show success and some do not. AHRQ knows that attempts at innovation are just as, if not more, important in accelerating quality improvement than successes at innovation. These attempts at innovation provide fascinating opportunities to learn about what doesn’t work and why while the innovation successes permit learning about what does works and how others could replicate it. No other resource captures both health care quality improvement successes and failures.

In addition to the innovative quality improvement strategies and associated tools that will be captured in the AHRQ Health Care Innovations Exchange, our focus will include learning and networking opportunities about innovation and adoption. We will be offering educational articles and events intended to extend health care professionals’ body of knowledge beyond patient care and quality improvement to innovation and adoption theory and practice. No other resource focuses on this type of education.

To mitigate any hint at duplication, AHRQ has been in discussions with representatives of public and private organizations that may offer information about quality improvement successes. Examples are provided:

• The Institute for Healthcare Improvement—Donald Goldmann, MD, Senior Vice President and Andrea Kabcenell, MPH, RN, Vice President;

• Health Resources and Services Administration--Rick Wilk, Director, Chicago Regional Division- Region 5;

• National Institute of Health’s Center on Minority Health and Health Disparities--Kyu Rhee, MD, MPP, Director, Office of Innovation and Program Coordination.

In addition, the AHRQ Health Care Innovations Exchange Technical Expert Panel and Editorial Board, each of which is comprised of experts representing large health care organizations, private regulatory organizations, and governmental agencies, have not expressed concern that AHRQ’s efforts will be duplicative and appreciate AHRQ’s efforts to prevent duplication by collaborating with others.

2. It seems like AHRQ already functions as a “one stop shop: for information about innovative practices….. If one of the requirements to participate is that the information about the innovation is publicly available, couldn’t AHRQ continue to do what it seems to be doing now (i.e., compiling a list of past and ongoing research, based on AHRQ grants and what AHRQ has found in the published literature)? This seems like the least burdensome way to accomplish this goal.

The AHRQ Health Care Innovation Exchange site will focus on innovative processes that improve health care rather than on research that generates new scientific information. After careful examination of a number of sites and other sources, like journals, that describe research and quality improvement information, AHRQ determined that none provided the level of background, contextual and implementation detail that potential adopters need to make an informed decision about best fit when searching for innovative solutions to care delivery problems. The design of the IE profile captures the following detail, some of which must be collected from the innovator:

• thorough statement of the problem

• a detailed description of the innovative activity

• references and related articles

• results or evidence of effectiveness

• context for the innovation

• descriptions of the planning and development process

• lists of the resources and skills required to implement

• funding sources

• adoption considerations (including how to get started, how to sustain, and other lessons learned.

For AHRQ to be effective in accelerating change and transformation in health care quality, we believe the innovators must be willing to make information about their problem and their innovative solution to that problem, along with lessons learned, publicly available. AHRQ does not want the Health Care Innovations Exchange to be a marketplace of proprietary solutions; thus, the emphasis on public availability of information.

Also, the Innovations Exchange invites innovations that are self-funded through the innovator’s organization and those that have not been published in peer-reviewed journals, e.g. those that are presented at professional conference. Sourcing these innovations and attempts will occur through many mechanisms to enrich the collection, including those that innovators themselves submit.

2. What is AHRQ’s definition for what constitutes an “activity that is truly innovative?”

For the purposes of the AHRQ Health Care Innovations Exchange, innovation refers to the creative implementation of new or altered products, services, processes, systems, organizational structures, or business models as a means of improving one or more domains of health care quality. In this context, the Health Care Innovations Exchange sees innovations as activities that are generally perceived as new in a particular setting or for a particular population relative to the usual care delivery processes. Applying that view to this work, the Health Care Innovations Exchange team will assess the novelty of quality improvement initiatives/efforts against usual care delivery (or care processes). The context and conditions of the novel approach to improving quality are important elements in making that assessment.

In addition to brand new ideas, the Health Care Innovations Exchange will welcome activities adapted from other industries to health care, transferred from one health care setting or market segment to another, drawn from settings in other countries, or applied to a new or different patient population.

Innovators must be able to communicate to the Health Care Innovations Exchange team how their innovations differ from what is regarded as standard practice in their organizations and among similar organizations. The Health Care Innovations Exchange team will couple their working knowledge of the industry (and the knowledge of experts) with the innovators’ perspective in making a final decision about novelty.

2. One of the requirements is that “there is reason to believe that the innovation will be effective.” Prior to providing the public with information about an innovation, wouldn’t AHRQ want to require that evidence for its effectiveness already be established? Otherwise, it is unclear what the utility of the information is.

This question is one that AHRQ gave great consideration in its decision making to move ahead with the Health Care Innovations Exchange. Currently existing in the health care community is the tension between quality improvement projects (applied quality improvement initiatives) and quality improvement science (formal research with proper controls to minimize bias, oversight to protect subjects/researchers/funders, and skilled researchers and statisticians). One might naturally think that AHRQ would focus solely on the science of quality improvement but AHRQ has learned that waiting for scientific findings can impede progress and speedy improvements. While AHRQ funds quality improvement scientific studies, the Agency has decided to also capture applied quality improvement initiatives that are novel as one effort to accelerate change and transformation across the nation. Experts working with AHRQ on the Health Care Innovations Exchange have described this step by AHRQ as innovative.

AHRQ recognizes that many of the innovations it will include in the Health Care Innovations Exchange will not be conceived nor implemented in conventional research settings and that the quality of evidence the innovators can collect may be limited by resources and other contextual factors. AHRQ believes that what these innovations will offer to the health care community has a benefit greater than the risk of evidence strength. Because of a focus on applied quality improvement rather than formal scientific health services research, the Health Care Innovations Exchange will accept a range of supporting evidence for effectiveness. Knowing that AHRQ’s audience has come to expect an assessment of the evidence of effectiveness, the Agency is rating and describing the evidence associated with the innovations included in the Health Care Innovations Exchange in this way:

• Strong: The evidence is based on one or more rigorous evaluations using experimental designs that minimized bias and were based on random allocation of patients to comparison groups. The results of the evaluation(s) show consistent direct evidence of the effectiveness of the innovation in improving the targeted health care outcomes and/or processes.

• Moderate: While there are no randomized, controlled experiments, the evidence includes at least one systematic evaluation of the impact of the innovation using a quasi-experimental design, which could include the non-random assignment of individuals to comparison groups, before-and-after comparisons in one group, and/or comparisons with a historical baseline or control. The results of the evaluation(s) show consistent direct or indirect evidence of the effectiveness of the innovation in improving targeted health care outcomes and/or processes. However, the strength of the evidence is limited by the size, quality, or generalizability of the evaluations, and thus alternative explanations cannot be ruled out.

• Suggestive: While there are no systematic experimental or quasi-experimental evaluations, the evidence includes non-experimental or qualitative support for an association between the innovation and targeted health care outcomes or processes. This evidence may include non-comparative case studies, correlation analysis, or anecdotal reports. As with the category above, alternative explanations for the results achieved cannot be ruled out.

• Unproven: There is no quantitative or qualitative evidence supporting an association between the innovation and targeted health care outcomes or processes. However, the design of the innovation itself provides a theoretical basis for this link. For example, the innovation may be modeled after similar, successful innovations that were developed in other contexts or industries.

Each innovation profiled in the AHRQ Health Care Innovations Exchange will have a rating and a description of the evidence associated with it. The rating and description will be developed by the Health Care Innovations Exchange project team, not the innovator.

5