SUPPORTING STATEMENT: PART B
OMB #
Understanding Health System Approaches to Chronic Pain Management
Version: December 10, 2021
Point of Contact:
Jaswinder K. Legha
Contact Information:
Centers for Disease Control and Prevention
National Center for Injury Prevention and Control
4770 Buford Highway NE MS S106-8
Atlanta, GA 30341-3724
E-mail: oms0@cdc.gov
CONTENTS
Section B
B.1. Respondent Universe and Sampling Methods 3
B.2. Procedures for the Collection of Information 5
Primary Care Clinician Survey 5
Interviews with System Leaders 6
B.3. Methods to Maximize Response Rates and Deal with Nonresponse 11
B.4. Test of Procedures or Methods to be Undertaken 12
B.5 Individuals Consulted on Statistical Aspects & Individuals Collecting and Analyzing Data 12
List of Attachments
Attachment A: Primary Care Clinician Survey
Attachment B: Patient Survey
Attachment C: Health System Leaders Interview Guide
Attachment D: Case Studies Interview Guide
Attachment E: Member Checking Sessions Interview Guide
This formative research study will include systematic collection and analysis of a range of primary and secondary data sources. To answer the research questions, we will employ qualitative synthesis and analytic approaches, quantitative analyses, and various mixed-methods approaches.
Regarding primary data collection from primary care clinician and patient surveys (Attachments A and B), primary data will be collected from a respondent universe composed of 5,940 total respondents, consisting of up to 1,182 (assuming 30% response rate of 3,940) primary care clinicians and up to 1,000 (assuming 50% response rate of 2,000) patients, across up to10 diverse AMGA (American Medical Group Association) member health systems.
These health systems will be selected from AMGA member health systems who have self-identified to participate in the formative research study.
Among those expressing interest in participating in the formative research, the final health systems in this study will be selected based on the following criteria:
AMGA members who have expressed interest in participating and/or participated in previous research projects
Broad range of specific guideline recommendations implemented
Greater than 3 years of guideline implementation, beginning in 2017 or earlier
Geographic representation across the United States
Mix of urban and rural population served, including racial, ethnic and socioeconomic variation
Presence of primary care clinicians who prescribe opioids for chronic pain management (as opposed to prescribing solely by pain management specialists)
Availability of structured data on physician practices and patient outcomes related to specific guidelines, from before and after implementation
We will exclude systems that implemented guidelines more recently (defined as implementation of guidelines in 2018 or later) and therefore had insufficient time to observe changes over time; those with no primary care clinicians who prescribe opioids; and those without structured data. Although not nationally representative due to nonprobability sampling specifically among health systems that meet these criteria, the selection of a broad array of up to 10 geographically dispersed health systems will enhance our ability to understand the effects of a wide variety of chronic pain and opioid prescribing guidelines and policies and the impact of their implementation on diverse populations of patients, clinicians, and health systems.
The feasibility and evaluability assessments preliminary phases to help identify health systems are described in more detail in the accompanying Supportive Statement Part A (SSA) document.
Within health systems, the populations to be studied include: 1) patients age 18 years or older with chronic pain, who currently or previously have been prescribed long-term opioid therapy (LTOT), in primary care, outpatient settings (excluding active cancer, palliative, and end of life care), and 2) primary care clinicians, staff, and health system leaders affiliated with a diverse sample of participating AMGA member health systems.
Regarding primary data collection from health system leader interviews (Attachment C), a system liaison from each system will help to identify 5 leaders and administrators from each health system who were involved in the project recruitment and approval processes.
Regarding primary data collection from case study interviews (Attachment D), the system liaison will assist with identifying at least 2 clinicians and care team members per health system, ideally staff described as critical to system pain management and opioid prescribing activities. The system liaison will also help to identify at least two patient and/or caregiver cases per participating system. Examples of potential case studies of interest are described below.
Member checking sessions will involve five representatives per health system, including health system administrators and leadership representatives involved in the project recruitment and approval process.
Data collection activities are summarized in Exhibit 1.
Exhibit 1. Summary of primary data collection activities
Data Collection Method |
Targeted Respondents |
Methods for Selection |
Primary Care Clinician Survey (Att. A) |
1,182 (upper bound is 3,940 primary care clinicians for all systems with a 30% response rate) |
Primary care clinicians who are able to prescribe opioids |
Patient Survey (Att. B) |
1,000 (100/system) |
Health system-selected sample of patients previously or currently taking long-term opioid therapy for chronic pain |
Group Interviews with health system leaders (Att. C) |
50 (5/system) |
Health system administrators and leaders involved in the project recruitment and approval processes |
Staff Case Study Interviews (Att. D) |
20 clinicians (2/system) 20 other health care staff (2/system) |
Clinicians and staff described as critical to system opioid prescribing activities, identified by system leadership |
Patient Case Study Interviews (Att. D) |
50 patients and/or caregivers (5/system) |
Health system liaisons will help identify |
Member Checking Sessions (Att. E) |
50 (5/system) |
Health system administrators and leadership representatives involved in the project recruitment and approval process |
Within participating health systems, we will conduct a survey of primary care clinicians who possess the necessary licensure and certification to prescribe opioids (or clinicians who work under the supervision of a physician with such licensure/certification). The principal aim of this survey is to obtain data from the clinician perspective, including questions regarding awareness of chronic pain management and opioid prescribing policies and guidelines, including those related to access to medications for opioid use disorder (MOUD) implemented in their health systems; prescribing behaviors both before and after guideline/policy implementation; confidence in caring for patients with chronic pain; the impact of guideline/policy implementation on day-to-day work; barriers to care for patients with chronic pain; and questions regarding prescribing or referring for MOUD. Note that for the purposes of this formative research study, “chronic pain management policies/guidelines” refers to policies/guidelines that may include prescribing of opioid medications, nonpharmacologic therapies, and/or non-opioid medications for chronic pain, as well as opioid use disorder (OUD) assessment and treatment.
Survey data from clinicians at participating health systems will be collected electronically. The Primary Care Clinician Survey is expected to take 10 minutes to complete based on pilot testing and experience with similar data collection efforts. All survey data collected from respondents will be entered directly into the study database through REDCap, the project data management system. REDCap can be implemented in a variety of environments for compliance with standards such as HIPAA, 21 CFR Part 11, and up to FISMA high authorization.
Clinicians will be recruited via email by on-site administrative staff at participating health systems (Attachment I). Clinicians will be recruited in person to participate during “grand rounds” or other focused events within the health system. As discussed in the Supporting Statement: Part A document, a Certificate of Confidentiality applies for this study. Therefore, protections (and limits to protections) provided by the Certificate of Confidentiality will be discussed with respondents by trained staff at recruitment, and in the data collection instruments. On-site staff will record contact information for consented clinicians in REDCap. Depending on the preferences of participating health system leadership, respondents will be invited to complete the online surveys via REDCap through a suite of available options, including a unique email link, text messaging interface, or the REDCap mobile application. The online, mobile application, text message, and REDCap data entry screens also provide quality assurance through the use of identity confirmation procedures, logic and range checks, and automated skip patterns. In addition, for specific needs, health system staff will be authorized to enter response data directly into the REDCap database if surveys are administered by telephone or by in person interviews. REDCap can also be programmed to follow-up with unresponsive respondents to increase survey completion rates.
Another advantage of the REDCap system is the ability to employ built-in data quality checks, including checks for out-of-range values and missing data. REDCap also allows for the easy creation of automated data dashboards that will allow Abt Associates and CDC staff to periodically check overall survey response rate and other desired progress metrics. REDCap facilitates different access rights to be assigned by user profiles, which limits access to database variables, such as PII (personally identifiable information)/PHI (protected health information), at the individual user level. Therefore, staff at participating health systems, Abt Associates staff, and CDC colleagues can all access the same REDCap project database, while still ensuring that each individual user only accesses the appropriate data categories.
Upon completion of all data collection efforts, REDCap will be used to export the database in the desired program format for preparation of the final analytic dataset and performing specified statistical analyses.
Surveys of patients will be critical to investigate guideline and policy implementation in health systems. To ensure we obtain the patient perspective and experience with health systems’ implementation of opioid prescribing policies and guidelines, we will field a survey of patients with chronic pain aged 18 years and older who were previously or are currently taking LTOT. All patients that meet these criteria will be eligible to participate, and health system liaisons will identify a sample of 200 patients per system from the eligible pool.
Eligible patients will be approached by on-site staff at participating health systems during a regularly scheduled clinic visit. If eligible patients agree to participate, on-site staff will document consent, and provide access to recruited patients to complete the online surveys via REDCap through a suite of available options, including a unique email link, text messaging interface, or the REDCap mobile application. Protections, and limits to protections, provided by the Certificate of Confidentiality will be discussed with respondents by trained staff at recruitment, and in the data collection instruments. The online, mobile application, text message, and REDCap data entry screens also provide some quality assurance using identity confirmation procedures, logic and range checks, and automated skip patterns. In addition, for specific needs, health system staff will be authorized to enter response data directly into the REDCap database if surveys are administered by telephone or in person interviews. REDCap can also be programmed to follow-up with unresponsive respondents to increase survey completion rates.
From among the recruited sample of 200 patients per health system, we estimate a 50% response rate, for a potential of up to 1,000 patient responses. The patient survey is expected to take 10 minutes to complete based on pilot testing and experience with similar data collection efforts. Survey data collection from patients at participating health systems will be collected electronically. All survey data collected from respondents will be entered directly into the study database through REDCap. REDCap can be implemented in a variety of environments for compliance with standards such as HIPAA, 21 CFR Part 11, and up to FISMA high authorization.
Upon completion of all data collection efforts, REDCap will be used to export the database in the desired program format for preparation of the final analytic dataset and performing specified statistical analyses. Patients completing the brief survey (25 or fewer questions) will receive a $5 renumeration.
The patient survey will ask patients about their chronic pain histories, awareness of health system’s implementation of pain management and opioid prescribing policies and guidelines, beneficial and harmful consequences of implementation, and patient-clinician communication.
Interviews with Health System Leaders (Attachment C)
We will interview health system leaders involved with chronic pain management and opioid prescribing guideline implementation efforts. We will work with the system liaison to identify the appropriate leaders from each health system to interview to understand the system’s implementation process across primary care clinics. Potential leader respondents may include: Chief Medical Officer (CMO), Vice President (VP) of Quality and Safety, VP for Primary Care, lead for the opioid improvement effort (if applicable), members of health system opioid committee (if applicable), information technology (IT) analytics lead who developed measures and/or electronic health record (EHR) tools, or other relevant leaders. Prior to interviews, a copy of health system policies and guidelines will be requested to facilitate discussion.
Interviews will be conducted by telephone and are expected to take 60 minutes, depending on the level of respondent involvement in the implementation process. All interviews will be conducted by telephone or video software (e.g., WebEx or Zoom), and are expected to take up to 60 minutes. All interviews will be recorded with respondents’ permission and transcribed. If permission to record the interview is not given, we will rely on the notes taken by a note taker. In addition, similar to other respondents, protections, and limits to protections, provided by the Certificate of Confidentiality will be discussed with respondents.
Leaders will be interviewed on how and why decisions were made to implement guidelines or policies and who made those decisions; benefits and unintended consequences to implementation (such as patients leaving the system due to stricter prescribing behaviors); whether these initiatives have been successful or not; facilitators and strategies to overcome barriers; and lessons learned. Questions about effects of COVID-19 and whether and how systems have addressed racial/ethnic disparities in chronic pain management, opioid prescribing, and treatment of OUD will also be asked.
Staff Case Study Interviews (Attachment D)
Case studies with clinicians (and patients, discussed in the next section) can provide an in-depth understanding of a single or small number of cases set in their real-world contexts. Examining the experiences of clinicians, care teams, patients, and caregivers can provide a deeper understanding of real-world behavior within a specific healthcare context to elucidate how or why change occurred for chronic pain management, opioid prescribing, and/or provision of MOUD.
The following are examples we would be interested in fully understanding, given the aims of this formative research study:
The care team’s approach to buy-in with safer prescribing or actively engaging in efforts to diagnose and treat patients with OUD
The care team’s approach of working together with planned care visits, huddles, or workflows to improve care of patients with chronic pain
A clinician’s or care team’s approach to improve diagnoses and treatment of OUD in racial or ethnic minority populations, or develop specialized clinics to treat chronic pain (e.g., sickle cell disease)
We will rely on the system liaison to identify clinicians and care team members. We will aim to interview at least two staff members per health system, up to two clinicians and two care team members for an upper limit of four staff members per system. Interviews will be conducted by telephone or video software (e.g., WebEx or Zoom) and are expected to take up to 30 minutes. All interviews will be recorded with respondents’ permission. If permission to record the interview is not given, we will rely on notes taken by a note taker. In addition, similar to other respondents, protections, and limits to protections, provided by the Certificate of Confidentiality will be discussed with respondents.
We will also seek to identify a select number of patients with chronic pain for case study interviews.
Given the aims of this study, we would be interested in better understanding patient and/or caregiver stories of taking LTOT and its effects on pain and functioning, and/or engaging in nonopioid therapies, and/or receiving an OUD diagnosis and starting on MOUD (Note that not all listed examples may be applicable to all patients.)
As for clinician case studies, we will work with the health system liaison to identify at least two patients and/or caregivers per participating system, with an upper limit of five patients and/or caregivers per system. All interviews will be conducted by telephone or video software (e.g., WebEx or Zoom) and are expected to take up to 30 minutes. All interviews will be recorded with respondents’ permission. If permission to record the interview is not given, we will rely on notes taken by a note taker. As discussed above, protections, and limits to protections, provided by the Certificate of Confidentiality will be discussed with respondents.
Member Checking (Validation) Sessions (Attachment E)
We will also conduct “member checking,” or validation, sessions with each health system. “Member checking” sessions are a qualitative data collection methodology to gather informants’ interpretations of their organizational realities, and to validate results.
In these sessions we will review and discuss findings from the study of each health system’s implementation of policies and guidelines,1,2 and we will engage the participants in validating our findings for their system and/or helping to explain unexpected changes in trends or add additional context and insights. Each session will include 5 participants, be conducted via WebEx or Zoom, and take 60 minutes to complete. Prior to each session, participants will be provided with a list of topics to be discussed in the sessions.
For the survey results, we will run descriptive and univariate statistics stratified by individual health system as well as across all participating health systems. We will compute means, medians and standard deviations. We will plot the distributions of continuous variables and create frequency tables and plots for categorical variables. We will conduct the following analyses:
Non-response: To the extent possible, if we have enough information on the respondents within the system who were asked to complete the survey, we will provide a description of sampling strategy and non-response patterns, and implications for survey report (and apply weights if we have sufficient detail).
Descriptive analysis of survey responses: Descriptive statistics (mean, standard deviation, and distribution for continuous variables; frequencies and percentages for categorical and binary variables) will be reported in tables and visualized in figures and graphics. We will also code responses to open-ended survey questions thematically and describe common themes, along with demonstrative examples of responses under each theme.
Subgroup analyses of survey responses: For the clinician surveys, we will perform stratified analyses by geographic region, clinician type, and system characteristics such as number of patients, specialty practices, and application of prescribing policies. For the patient surveys, we will stratify analyses by patient demographics, diagnosis for which opioids were prescribed, and characteristics of opioid use including daily dosage in MME, duration of opioid use, and concordance with system opioid prescribing policies (treatment agreements, urine drug tests, etc).
Statistical Analysis for Health System Leader Interviews, Case Studies, and Member Checking Sessions
All qualitative data obtained from the health system leader interviews, case studies, and member checking sessions will be coded and analyzed using Nvivo qualitative analytic software. Codebook development will be iterative and include deductive codes (established a priori from the research questions and Consolidated Framework for Implementation Research (CFIR)7 domains and inductive codes (emerging from the data), as described in the SSA document.
Exhibit 2 includes a subset of codes we intend to include, such as CFIR domains, implementation strategies, and key constructs from our research questions. The codebook development process will begin with the a priori codes listed in Exhibit 2 and will conduct thematic coding and iteratively expand the codebook– especially where we do not have an existing framework. Research team members, led by an experienced qualitative researcher, will independently read selected excerpts of data sources (interview notes, meeting minutes) to link to a priori codes and develop potential new codes. We will discuss codes, definitions, and inclusion and exclusion criteria to develop an initial codebook and use the initial codebook to code three new sources. We will continue to refine the codebook until the codebook is sufficiently detailed to capture meaningful detail and use the codebook to complete coding of all materials. We will document each type of data across data sources, synthesize themes, and identify clear findings. Some qualitative codes, such as the type of organization recruited to participate in the Cooperative, may be transformed into quantitative codes that can be used as covariates in the quantitative analyses.
Coding quality assurance procedures will include training coders, periodically checking inter-rater reliability, and frequent debriefs on findings and coding. The resulting data will be analyzed to create individual case studies evaluating each grantee, and to support program-wide analysis of collective impact.
Exhibit 2. Working Draft Codebook: a priori Codes
Code |
Definition |
Implementation Strategies |
Methods or techniques used to enhance the adoption, implementation, and sustainability of a clinical program or practice |
|
Provide ongoing consultation with one or more experts (“practice facilitators”) in the strategies used to support implementing the innovation |
|
Assess various aspects of an organization to determine its degree of readiness to implement, barriers that may impede implementation, and strengths that can be used in the implementation effort |
|
Identify and prepare individuals who dedicate themselves to supporting, marketing, and driving through an implementation, overcoming indifference or resistance that the intervention may provoke in an organization |
|
Develop a formal implementation blueprint that includes all goals and strategies. The blueprint should include: 1) aim/purpose of the implementation; 2) scope of the change (e.g., what organizational units are affected); 3) timeframe and milestones; and 4) appropriate performance/progress measures. Use/update this plan to guide the implementation effort. |
|
Collect and summarize clinical performance data over a specified time period and give it to clinicians and administrators to monitor, evaluate, and modify clinician behavior |
|
Access new or existing money to facilitate the implementation |
|
Develop/build and implement EHR tools such as clinical decision support, reminders, alerts, templates, etc. |
|
Hold meetings targeted toward different stakeholder groups (e.g., clinicians, administrators, other organizational stakeholders, and community, patient/consumer, and family stakeholders) to teach them about the clinical innovation |
Implementation Barriers |
Factors that are barriers to implementation |
Implementation Facilitators |
Factors that are facilitators to implementation |
Select CFIR Constructs |
|
Intervention Characteristics |
Key attributes of the interventions that influence the success of the implementation |
|
Attributes of the context outside of the primary implementation setting (practice) that influence success of implementation |
|
Attributes of the context inside the primary implementation setting (practice) that influence success of implementation |
|
Norms, values, and basic assumptions of a given organization. |
|
Individuals’ shared perception of the importance of the implementation within the organization |
|
Tangible and immediate indicators of organizational commitment to its decision to implement an intervention. |
|
Commitment, involvement, and accountability of leaders and managers with the implementation. |
|
Characteristics of the individuals involved in implementing the intervention |
|
Individuals’ attitudes toward and value placed on the intervention as well as familiarity with facts, truths, and principles related to the intervention |
|
The process of implementing the intervention, including planning, engaging, executing, and evaluating. |
Note: Definitions from Powell (2015) and www.cfirguide.org.
To complement the primary data collection efforts described above, a variety of secondary data sources will be used to quantitatively describe and evaluate the implementation of specific guidelines and policies and their effects on patient outcomes and clinicians’ practices.
A subset of AMGA members contributes EHR and limited deidentified adjudicated claims data to a common data repository through a partnership with Optum, AMGA’s data analytics partner. We will conduct a quantitative patient-level, longitudinal (pre-post) analysis within each of the (up to) 10 health systems of the secondary data in the Optum dataset. We will also consider clinician-level longitudinal analyses for these 10 health systems as an alternative to the patient-level analyses if data/sample size limitations make the patient-level analyses impractical. Clinician-level data, which aggregates prescribing practice data across a panel of patients, may provide greater statistical power to show differences in opioid prescribing behaviors as compared to individual patient-level data. For both clinician and patient-level data, we will analyze how implementation of system-specific policies and guidelines impacted opioid prescribing practices. For example, opioid prescribing measures such as overall dosage (in MME), days’ supply of opioid prescriptions, use of non-pharmacological therapies, and co-prescribing opioids and benzodiazepines will be considered.
Additionally, we will examine trends and pre-post changes in quality improvement (QI) measures at each health system, as several health systems have measures they have built and used to monitor implementation of policies or guidelines.
These secondary analyses will provide quantitative estimates of the association between implementation of the policies and guidelines and the patient outcomes of interest separately within each of the 10 health systems (although causal relationships cannot be definitively determined). We will analyze each system separately because each system implemented policies and guidelines in a different manner, and there are many unobserved system-specific factors that may impact patient outcomes. Results from these secondary analyses will add context to the discussions with health system leaders during the member checking sessions described above.
Health systems selected to participate will have already expressed a high level of interest and administrative support to participate in the formative research study. In addition, the evaluability assessment will have already identified health systems capable of accessing robust EHR data in a timely manner to effectively complete all research objectives. For these reasons, we expect a high level of participation in the health system leader interviews and member checking sessions.
While response rates in surveys of clinicians have declined in recent years,3 we will use several approaches to improve the survey response rate. Given Abt Associates’ recent experience with web surveys for clinicians, and using e-mail addresses provided by systems, we believe a 30% response rate is achievable. Strategies to increase clinician participation include:
Brevity of survey questionnaire (25 or fewer items), expected to take 10 minutes
$25 incentive for participation (incentive may be reconsidered once systems are finalized)
Survey questionnaire with items that are relevant to clinicians, as informed by cognitive testing with primary care clinicians and discussions with experts
Encouragement and follow-up reminders from health systems leadership
Strategies to increase patient participation include:
Brevity of survey questionnaire, expected to take 10 minutes
$5 incentive for participation
To address non-response among patients, identified patients will receive a follow-up email one week after the initial recruitment email. After two weeks of non-response, clinical staff at participating health systems directly involved with a patient’s care will call the participant by telephone to again invite them to participate in the survey. Patients who do not complete the survey after this second attempt will not be contacted again and will be considered unresponsive to recruitment. Nonparticipation in the study or nonresponse after recruitment will not impact clinical care.
Liaisons at each health system will be identified to champion this formative research study system-wide to effectively engage administration, staff and patients to garner sufficient support and participation in all research activities. We will work closely with these liaisons to help identify clinicians and patients who are eager to share their experiences for the case study interviews. Only two patients and two clinicians will be required for case study interviews from each health system, which will assist with ease of recruitment. Interviews will be scheduled at the convenience of all participants. In addition, financial incentives will be given to respondents as outlined above.
The estimate for burden hours presented is based on the sum of the total time burden of the primary care clinician survey, patient survey, interviews with system leaders, case study interviews, and member checking sessions by mid-level researchers and clinician researchers (e.g., nurses). The Primary Care Clinician Survey is expected to take 10 minutes to complete based on pilot testing and experience with similar data collection efforts. The Patient Survey is expected to take 10 minutes to complete based on pilot testing and experience with similar data collection efforts. The burden estimate for the health system leader interviews, case study interviews, and member checking sessions are based on Abt Associates’ experience with similar data collection efforts in the past.
Abt Associates is the contractor who will develop data collection tools and perform all analysis on behalf of CDC. The professionals from Abt Associates have over 40 years of experience providing high quality, timely, and cost-effective data collection for federal agencies. Abt Associates employs many statisticians, health economists and experienced research methodologists. Sharmini Radakrishnan, Ph.D., from Abt Associates, reviewed the proposed statistical analyses. Dr. Radakrishnan has designed several rigorous, practice-based research studies for CDC and other federal agencies. She is available should any questions regarding the statistical analyses for this project arise. The key project contact at Abt Associates is Sarah Shoemaker-Hunt.
Key contact information for Sarah Shoemaker-Hunt is provided below:
Sarah Shoemaker Hunt, PhD, PharmD, Principal Associate
10 Fawcett St.
Cambridge, MA 02138
Office: 617-349-2472
Email: Sarah_Shoemaker-Hunt@abtassoc.com
1 Lincoln YS, Guba EG. But is it rigorous? Trustworthiness and authenticity in naturalistic evaluation. N Dir Eval. 1986; 1986 (30):73–84.
2 Cohen DJ, Crabtree BF. Evaluative criteria for qualitative research in health care: controversies and recommendations. The Annals of Family Medicine. 2008 Jul 1; 6(4):331-9.
3 McLeod CC, Klabunde CN, Willis GB, Stark D. Health care provider surveys in the United States, 2000–2010: a review. Evaluation & the health professions. 2013 Mar; 36(1):106-26.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Moreland, Briana (CDC/ONDIEH/NCIPC) |
| File Modified | 0000-00-00 |
| File Created | 2022-02-18 |