Supporting Statement Part B 12-17-15

identify existing quality improvement measures and gather proposals for additional measures (not generated from patient survey data) that health care organizations could use to monitor progress related to enhancing patient understanding, navigation, engagement, and self-management; and
refine and cull the measures identified to develop a set of quality measures that reflects patient priorities, has expert support, and for which more formal measure development and testing is warranted.

The application of statistical methods in achieving these objectives is limited because of the lack of information on the existence and usefulness of quality measures related to enhancing patient understanding, navigation, engagement, and self-management. The project design is driven by this current lack of information and includes:

Convening a Technical Expert Panel to obtain information about relevant measures and other expert input,
Publishing a Federal Register Request for Information (RFI) requesting the public to provide information about existing measures and to propose new measures,
Conducting a Literature Review to identify existing measures and document their measurement specifications and properties,
Convening two Patient Focus Groups to obtain patient input on what things are most important for organizations to address and measure,
Conducting Environmental Scan Interviews with 25 purposively selected health care organizations that are engaged in activities to improve patient understanding, navigation, engagement or self-management with the goal of obtaining information about measures used by these organizations to guide their relevant quality improvement work, and
Conducting a Delphi Panel Review of Measures, whereby a panel of 9 to 12 experts and other stakeholders will use a modified Delphi method to evaluate the measures identified, providing data to support refinement and culling of the measures and establishment of a set of measures recommended for further development and testing.

Of these activities, only one involves the systematic collection of information from 10 or more respondents: the Environmental Scan Interviews.

1. Respondent universe and sampling methods

The Environmental Scan Interviews will be conducted with representatives from 25 health care organizations that are identified as undertaking activities related to improving patient understanding, navigation, engagement and self-care. The delivery of health care is localized. Even within a health care system, different locations may be undertaking different activities related to improving patient understanding, navigation, engagement and self-care. As such, the potential respondent universe for the Environmental Scan Interviews will be the number of establishments in the health care sector. The Quarterly Census of Employment and Wages (QCEW) program managed by the US Department of Labor, Bureau of Labor Statistics provides estimates of the number of establishments in each North American Industry Classification System (NAICS) industry sector. For the health care industry sectors, the preliminary estimates of the number of establishments in the first quarter of 2015 for Ambulatory Health Care Services (NAICS 621) is 575,877 establishments, for Hospitals (NAICS 622) is 8,979 establishments, and for Nursing and Residential Care Facilities (NAICS 623) is 76,109 establishments. These facility types provide an estimate of the potential respondent universe for the Environmental Scan Interviews.

A very small percentage of all health care establishments are likely to be engaged in relevant activities and a statistical sampling of all potential respondents would not be an efficient approach to meeting project objectives. A purposively selected sample of health care organizations that are actively engaged in relevant quality improvement work will be identified through six project activities:

The Technical Expert Panel will identify organizations that are actively engaged in quality improvement work aimed at improving patient understanding of health information, navigation of the health care system, engagement in decision-making, and self-management.
The literature review is expected to identify organizations involved in relevant quality improvement work.
Information on potential organizations will be requested from the American Board of Pediatrics and the American Board of Family Medicine, which both support maintenance of certification modules guiding individual providers and groups of providers in conducting relevant quality improvement work.
We will request nominations on a health literacy listserv maintained by the Institute for Healthcare Advancement and on AHRQ’s GovDelivery list of 64,000 subscribers of health literacy and cultural competence updates.
We will contact state and regional health literacy-focused organizations to obtain information about organizations in their areas that are engaged in relevant quality improvement work.
Representatives of health care organizations participating in the Environmental Scan Interviews will identify additional organizations they know to be engaged in relevant quality improvement work.

We anticipate that these six activities will identify approximately 75 organizations that are engaged in relevant activities and that are potential respondents for the Environmental Scan Interviews. The organizations identified will be prioritized into three categories based on the extent of relevant activities in each organization. Priority ratings will be based on the degree to which an organization appears to be engaged in a comprehensive effort to address patient understanding, navigation, engagement, and self-care and the likelihood that they are using quality improvement measures to monitor their work. Data collection efforts will focus on organizations with more comprehensive quality improvement programs so that we may identify a larger number and breadth of relevant measures. The prioritized listing of organizations will be used to recruit 25 organizations to participate in the Environmental Scan Interviews beginning with the highest rated organizations and recruiting in prioritized order until 25 organizations have provided information on their relevant activities and measures. Because the identified organizations are engaged in relevant activities demonstrating an interest in and commitment to this area of work, it is expected that the majority of targeted organizations (at least 60%) will agree to participate. We therefore anticipate contacting 40 organizations in order to reach our goal of recruiting 25 organizations.

Interviews will be conducted with representatives of the organization who are most knowledgeable about quality improvement efforts aimed at improving patient understanding, navigation, engagement, and self-care, and about the measures used to monitor the success of these efforts. We anticipate that a single individual is unlikely to provide all of the requested information, and therefore plan to interview two representatives who are particularly informed about relevant quality improvement work.

Exhibit 1 summarizes the information on the respondent universe and the sampling methods for this data collection activity.

Exhibit 1. Respondent Universe and Sampling Methods

Data Collection Activity	Respondent Universe	Sample Description	Sampling Method	Sample Size	Projected Response Rate	Projected Number of Respondents
Environmental Scan Interviews	660,965	Health care organizations engaged in activities to improve patient understanding, navigation, engagement, and self-care	Purposive Sample	40	62.5%	25

2. Information Collection Procedures

Environmental Scan Interviews will collect information about measures that organizations are using to monitor quality improvement activities designed to enhance patient understanding, navigation, engagement, and self-management. These telephone-based interviews will follow a semi-structured interview protocol. We will ask representatives of health care organizations to describe their relevant quality improvement activities and to provide detailed information about the measures used to monitor progress. For each measure identified, the respondent will be asked to provide information about how data are collected, how each measure is computed, the settings where each measure has been used, and information about measure reliability and validity. We will ask respondents to describe how they use these measures to monitor quality improvement activities and to identify supplemental measures they believe would be useful. Any written documentation about the measures identified will be requested. Environmental Scan Interviews will be recorded and transcribed verbatim. After completion of the interviews, all measure information reported during the call will be combined with any additional information available in written documentation provided by the organizations.

As described in the previous section, this information collection activity uses non-probability based samples. The objectives of the study are most efficiently achieved through the use of non-probability based samples to identify potential quality improvement measures and to refine and cull these measures based on patient priorities and expert support. As such, statistical inferences related to the underlying population will not be made using the information collected.

3. Methods to Maximize Response Rates

Because organizations showing great commitment to improving patient understanding, navigation, engagement, and self-management will be recruited, it is expected that the majority of organizations contacted will be willing to provide the requested information. We anticipate that we will have to contact 40 health care organizations to yield interviews with knowledgeable staff from 25 health care organizations. Several methods will be used to enhance participation rates in these interviews, including the following:

Having the AHRQ project officer or TEP members known to the organization send the first invitation to participate.
Acknowledging each organization’s existing activities and the contributions of the organization representatives to the improvement of patient understanding, navigation, engagement, and self-management.
Preparing and distributing informational material to the identified organizations and representatives that describes the objectives of the project, the types of information requested, the expected amount of time required for the interview, and the benefits of the information obtained to the organization and its patients.
Ensuring the semi-structured interview protocol is limited to information that is required to achieve project objectives, is not excessively lengthy, and minimizes burden on participants.
Involving knowledgeable staff in the conduct of the interviews.
Limiting the telephone interviews to no more than two hours.
Thanking respondents for their time and reinforcing the benefits of the project results to the health care organization, the respondent, and patients.

4. Tests of Procedures

No formal pre-testing of data collection procedures is planned. The Environmental Scan Interview Guide has undergone repeated rounds of review and revision by multiple experts on the project team. After the first two interviews, the project team will evaluate whether any adjustments should be made.

5. Statistical Consultants

Exhibit 2 provides information on the individuals consulted in the design and data analysis plans for the project. The exhibit provides information on the name, contact information, organizational affiliation, area of expertise, and role of each individual.

Exhibit 2. Expert Consultants

Name	Contact Information	Organization	Area of Expertise	Role in Collection and Analysis of Information
Cindy Brach, MPP	301-427-1444 cindy.brach@ahrq.hhs.gov	Agency for Healthcare Research and Quality	Qualitative methods, Health literacy	Reviewer
Mark Gritz, PhD	303-724-8359 mark.gritz@ucdenver.edu	University of Colorado	Econometrics, Survey research methods	Instrument review, Analysis plan
Angela Brega, PhD	303-724-1470 angela.brega@ucdenver.edu	University of Colorado	Quantitative methods, Qualitative methods, Health literacy	Instrument design, Analysis plan, Analysis
Karen Albright, PhD	303-724-3535 karen.albright@ucdenver.edu	University of Denver	Qualitative methods, Health literacy	Instrument design, Analysis plan, Patient Focus Group moderator, Interviewer, Qualitative analysis
Ulfat Shaikh, MD, MPH, MS	916-734-3690 ushaikh@ucdavis.edu	University of California Davis	Health literacy, Quality improvement	Instrument design, Analysis plan
Debra Saliba, MD, MPH	310-825-8253 saliba@rand.org	University of California Los Angeles	Delphi panels, RAND/UCLA Appropriateness Method	Delphi panel moderator and consultant, Analysis

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	IHAP Proposal Template
Subject	IHAP Proposal Template
Author	Angela Brega
File Modified	0000-00-00
File Created	2021-01-23