HHS/CDC/NCIPC
SUPPORTING STATEMENT FOR
OMB INFORMATION COLLECTION REQUEST
Part B
Evaluation of the SAMHSA PDMP Electronic Health Record (EHR) Integration and Interoperability Expansion Program
Supported by:
Department of Health and Human Services
Centers for Disease Control and Prevention
National Center for Injury Prevention and Control
Division of Unintentional Injury Prevention
Christopher M Jones, PharmD, MPH
Prescription Drug Overdose Team Lead
National Center for Injury Prevention and Control
Centers for Disease Control and Prevention
4770 Buford Highway Northeast, Mailstop F62
Atlanta, GA 30341-3724
Phone: 770-488-3944
Fax: 770-488-1317
Email: fjr0@cdc.gov
Submitted: TBD
B.1. Respondent Universe and Sampling Methods 1
B.2. Procedures for the Collection of Information 3
B.3. Methods to Maximize Response Rates and Deal with Non-Response 6
B.4. Tests of Procedures or Methods to be Undertaken 7
B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 8
The Centers for Disease Control and Prevention (CDC), National Center for Injury Prevention and Control, is requesting approval of a new information collection request for a period of 24 months to conduct a qualitative evaluation of the Prescription Drug Monitoring Program (PDMP) Electronic Health Record (EHR) Integration and Interoperability Expansion (PEHRIIE) program. This program is a collection of state initiatives funded by 2012 Prevention and Public Health Funds (PPHF-2012) cooperative agreements through the Department of Health and Human Services (DHHS), Substance Abuse and Mental Health Services Administration (SAMHSA). CDC is conducting a qualitative evaluation of this program in order to understand the processes, challenges, and successes in implementing and sustaining integration of PDMP data with Health Information Technology (HIT) systems and interoperability of PDMP systems across states. This information collection will also capture the experiences of clinical end users with the systems being upgraded under the PEHRIIE program as well as their recommendations for how the goals of the PEHRIIE program could have been better accomplished. To achieve these goals, CDC will conduct in person interviews with two non-overlapping groups of respondents: Key Project Staff/Stakeholders and Clinical End Users, as described below.
Key Project Staff/ Stakeholders: Key Project Staff/Stakeholders are defined as anyone involved in the planning and/or implementation of a state’s activities under the PEHRIIE program as well as those involved in the maintenance of the upgraded products resulting from the project. Interviewees in this respondent group will be identified by the primary project coordinators in each state. Based on preliminary conversations with the primary PEHRIIE project coordinators in each state, the individuals in this respondent category fall into one of seven groups based on their roles in the PEHRIIE projects:
PDMP Staff – Eight of the nine PEHRIIE states have identified at least one individual beyond the primary PEHRIIE project coordinator that is responsible for the management and operations of their PDMPs that is/was involved in their PEHRIIE project. This group of individuals will provide significant information pertaining to the process of implementing these upgrades to their PDMPs, the barriers encountered, and the anticipated benefits of these upgrades. Generally, the roles of the identified individuals in this category are either administrative, legal, or information technology (IT) support in nature, giving them specialized insight into the various aspects of the planned projects.
PDMP Software Vendors – Seven of the nine PEHRIIE states are using one of two PDMP software vendors as the backbone of their PDMP system, while the remaining two states are working with a PDMP software developer exclusively within their state. As the PEHRIIE projects will require upgrading these software systems, representatives of these vendors are critically important for the planned qualitative evaluation of the PEHRIIE projects and program.
PDMP Data Sharing Hub Vendors – Eight of the nine PEHRIIE states are using one of two PDMP data sharing hubs to establish interstate PDMP data sharing. Representatives of these vendors will provide information about the work required for each state to establish interoperability as part of the PEHRIIE projects.
State Government Agencies – Eight of the nine PEHRIIE states have identified at least one government agency within their state but outside of the PDMP that is/was involved in the planning, implementation, or maintenance of their PEHRIIE projects. Generally, these individuals have served in a consulting capacity, particularly with regards to legal and technical issues.
Health Information Exchange (HIE) Personnel and Technology Partners – Six of the nine PEHRIIE states have indicated that they are or will be establishing integration with EHRs and PDS systems by integrating their state’s PDMP databases into one or more pre-existing statewide or regional HIEs. Individuals in this respondent group include both the staff members responsible for overseeing the implementation of the PEHRIIE projects on behalf of the HIEs and the technology partners providing and maintaining the software underlying the HIEs.
Implementation Site Staff – In addition to the above described integration efforts involving HIEs, six of the nine PEHRIIE states have indicated that their PEHRIIE projects also involve implementation efforts directly at the clinical end user site, especially for integration with PDS systems at commercial pharmacy chains. Individuals in this group include the staff members at the implementation sites responsible for overseeing these efforts as well the IT support staff.
Other Stakeholders – Four of the nine PEHRIIE states have indicated that individuals that do not fit into any of the above described categories were also involved in their projects. These individuals include representatives of a software system being used by two of the PEHRIIE to establish EHR integration of PDMP data and members of external, non-governmental consulting groups.
Please note that because of the unique nature of each of the nine PEHRIIE projects, not every group of key project staff and stakeholders was involved in every PEHRIIE project. Based on these preliminary conversations with the primary PEHRIIE project coordinators in each state, the CDC evaluation team anticipates conducting 91 Key Project Staff/Stakeholder interviews with state-specific stakeholders (average of 10 per state, range 6 – 16) and 14 Project Staff/Stakeholder interviews with individuals at technology vendors that are involved in multiple PEHRIIE state projects (average 3 per vendor, range 1-5) for a total of 105 Key Project Staff/Stakeholder Interviews. This information is summarized in tabular form in Attachment 5 (Tables 6, 7, and 8). Because these preliminary discussions took place with nine or fewer respondents, OMB approval was not required nor sought. Because this work is an evaluation focused on identifying common challenges and facilitators of success across all projects, and because each of the individual projects funded by the PEHRIIE program is significantly different from the others, interviews are planned with the majority of the identified key project staff and stakeholders in each funded state. Thus, interviews will be conducted with almost the full population of staff and stakeholders engaged, eliminating the need for sophisticated sampling procedures. Under their cooperative agreements with SAMHSA, states funded through the PEHRIIE program are expected to collaborate with CDC in this evaluation; therefore, near 100% participation in the interviews is expected.
Clinical End User Group Interviews: Clinical End Users are defined as anyone responsible for prescribing or dispensing controlled substances as part of their medical practice who sees patients at a site where access to PDMP data has been directly integrated into their EHR or PDS system through one of the PEHRIIE projects (herein referred to as “implementation sites”) and who are 1) registered with their state’s PDMP; 2) have accessed the system since the implementation of the PEHRIIE project; and 3) were not personally involved in the development or implementation of the PEHRIIE project. Implementation sites include hospital emergency departments (EDs), primary care and/or ambulatory clinics, and pharmacies. Integration of access to the PDMP within EHRs and/or PDS systems may be achieved either directly or through a connection with an HIE. Because states are actively developing HIEs and adding new medical practice site connections; because the number of individuals practicing medicine at these sites is constantly changing; and because some of the PEHRIIE states have not yet determined their final sites for implementation of their PEHRIIE project at the time of this request, it is very difficult to estimate the total number of potential clinical end user interviewees for this information collection. However, based on preliminary conversations with the primary PEHRIIE project coordinators in each state, the CDC evaluation team has identified a total of 39 specific implementation sites (or when the states have not yet selected specific sites, the type of implementation sites) where clinical end users will be interviewed (average of four sites per state, range: 3-8). Interviews will be conducted with three clinical end users at each identified implementation site in order to capture a variety of experiences while not overburdening the employees at any one implementation site. Potential interviewees will be identified via purposive sampling from the whole population of clinical end users at a given implementation site following the protocol described in Section B2. A total of 117 clinical end user interviews will be conducted. Because of the large pool of potential respondents at each implementation site, and because there is significant interest among the medical provider community surrounding the prescription drug epidemic in general and PDMP improvement in particular, the CDC evaluation team expects close to a 100% clinical end user interview completion rate. This information is summarized in tabular form in Attachment 5 (Table 9).
Interviews will be conducted over a period of four days during a series of site visits within each of the PEHRIIE-funded states. Site visits will be conducted by at least two members of the evaluation team. Prior to the start of the information collection, the evaluation team will contact the primary PEHRIIE project coordinator in each of the nine funded states to discuss the ideal timing for the evaluation team’s visit to each of their states. Every attempt will be made to schedule state visits at a time at which the majority of critical key project staff and stakeholders will be available.
The following steps will be taken to schedule interviews with the key project staff/stakeholders in each state and to prepare the interviewees for their interviews with the evaluation team:
Each state’s primary PEHRIIE project coordinator will provide the evaluation team with a list of names, organizational affiliations, and contact information for the primary stakeholders involved in the planning, implementation, and/or maintenance of their project’s activities and end products.
The evaluation team will then contact each person on the list using the email template included as Attachment 7 in order to schedule their interview during the evaluation team’s visit to their state and site. A summary of the evaluation project and a description of the purpose of the interview will be sent to each potential interviewee at this time (Attachment 6), as well as copy of the interview guide to be used (Attachment 3).
The evaluation team will continue to contact the key project staff and stakeholders from each state to confirm availability and willingness to participate and to schedule interviews until each stakeholder, or when appropriate, stakeholder organization, is well represented during site visits. When multiple key project staff/stakeholders from the same organization were involved in the PEHRIIE project, they will be given the opportunity to schedule their interviews at the same time to reduce the burden on their organization.
In the event that critical stakeholders will not be available during the evaluation team’s visit to their state and site, a phone interview will be scheduled with the evaluation team at the interviewee’s earliest convenience. Interviewees that will be interviewed via conference call will be provided with the same materials as interviewees that will be interviewed in person. These materials will be disseminated prior to scheduling the interviews as described above.
The following steps will be taken to identify and recruit clinical end users at the identified PEHRIIE-project implementation sites, to schedule interviews with identified interviewees, and to prepare interviewees for their interviews with the evaluation team:
Each state’s primary PEHRIIE project coordinator will provide the evaluation team with a list of medical practice sites where prescribers and pharmacists will have access to the upgraded PDMP system via their EHRs or PDS systems as well as the name and contact information of an individual or individuals who can facilitate the recruitment of potential clinical end user interviewees at their site (herein referred to as “recruiters”). In states where integration will be accomplished via a Health Information Exchange (HIE) servicing multiple medical practice sites, states will be asked to identify and provide contact information for recruiters at least one each of the following practice types connected to the HIE: a hospital ED, a primary practice or ambulatory clinic, and a pharmacy.
The evaluation team will contact the identified recruiters at each implementation site via email to discuss the recruitment of clinical end users at their site. Recruiters will then be provided with a copy of the evaluation summary (Attachment 6) and a document describing the ideal interviewees for our purposes and the date for the intended interviews (Attachment 10). This document will then be disseminated to potential respondents via email or used as posters or handouts during recruiting. Ideal clinical end users will be anyone responsible for prescribing or dispensing controlled substances as part of their medical practice who are 1) registered with their state’s PDMP; 2) have accessed the system since the implementation of the PEHRIIE project; and 3) were not personally involved in the development or implementation of the PEHRIIE project.
Recruiters from each implementation site will provide the evaluation team with the names and contact information of potential interviewees from their site at least one month prior to the planned site visits. The evaluation team will then contact potential interviewees via email using the email template included as Attachment 7 in order to schedule an interview during the evaluation team’s visit to their state and site. A summary of the evaluation project and a description of the purpose of the interview will be sent to each potential interviewee at this time (Attachment 6), as well as copy of the interview guide to be used (Attachment 4).
The evaluation team will continue to contact potential clinical end user interviewees from each implementation site to confirm availability and willingness to participate and to schedule interviews until three clinical end user interviews have been scheduled at each implementation site.
In the event that the evaluation team is unable to schedule three clinical end user interviews at a given implementation site during the planned site visit, phone interviews will be scheduled with potential interviewees from those sites at the interviewee’s earliest convenience. Interviewees that will be interviewed via conference call will be provided with the same materials as interviewees that will be interviewed in person. These materials will be disseminated prior to scheduling the interviews as described above.
Interviews with key project staff/stakeholders will be conducted following the interview guide in Attachment 3 and interviews with clinical end users will be conducted following the interview guide in Attachment 4. Based on pilot testing, interviews with key project staff/stakeholders will take approximately 45 minutes each, while interviews with clinical end users will take approximately one hour. A more detailed description of the pilot test process is provided in section B4. Each interview will be conducted by at least two members of the evaluation team, with one main interviewer and one note-taker. Additional evaluation team members may also participate as note-takers via conference call. Prior to each interview, the interviewee will be read a brief summary of the evaluation and interview purpose and will be allowed to ask any questions that they may have based on the evaluation summary disseminated prior to the interview date (Attachment 6). Interviewees will also be advised that their participation in the interview is voluntary and that they do not have to answer any question that they deem sensitive. Interviewees will be asked to verbally acknowledge that they understand these statements and agree to continue with the interview. Finally, interviewees will be asked to give verbal approval allowing for both notetaking by the interviewers and audiotaping of the interview.
During the interviews, both evaluation team members will take extensive notes using standard note-taking sheets in order to capture consistent data. As noted above, with explicit permission from the interviewees, interviews will be audiotaped to use for reference and clarification in cases where the written notes are unclear. After each interview, all note takers will compare their notes and compile a summary for that interview.
Following the completion of all interviews pertaining to a given PEHRIIE grantee state, two analysts will collaboratively create a coding schema that will be used by each analyst independently to code each final interview notes summary. Coding schema will be applied as consistently as practicable across states while still allowing for important individual state findings to emerge. They will then conduct a thematic analysis of the collected qualitative data as well as across and within case analyses in order to address the following evaluation questions:
Process Questions
To what extent did grantees implement activities as planned?
To what extent did grantees improve the quality of their PDMP data?
To what extent did grantees improve PDMP access and usability?
To what extent did grantees establish or improve interoperability with other PDMPs?
To what extent did grantees establish or improve integration with HIT systems?
What were the costs and resources needed to implement and sustain PDMP interoperability and integration with other systems?
Outcome Questions
To what extent did the grant increase use of PDMP data?
What impact did the PEHRIIE project have on providers’ prescribing or dispensing behaviors?
Final results will be compiled into state-specific summary reports and a final comprehensive report that will be distributed to the primary stakeholders in the SAMHSA PEHRIIE program, including agency officials at SAMHSA and CDC, state PDMP administrators and health technology experts, colleagues at the PDMP Center of Excellence at Brandeis University, and other interested parties.
Key Project Staff/Stakeholders: This work is an evaluation designed to identify common challenges and facilitators of success across all projects. The design and aims of projects funded by the PEHRIIE program vary by state. Therefore, qualitative interviews are needed with a range of stakeholders in each funded state involved in the planning and/or implementation of the project's activities, as well as maintenance of the upgraded products resulting from the project. Because interviews will be conducted with stakeholders involved in all aspects of the project planning, implementation, and ongoing maintenance of project products, findings will reflect a comprehensive range of viewpoints on the successes and failures of the PEHRIIE program and its impact.
Clinical End Users: Interviews will be conducted with a range of clinical end users in each funded state, including individuals with different levels of experience, individuals working in different practice settings and seeing different types of patients, and individuals accessing their PDMP via different types of EHRs and PDS systems. Collecting qualitative information from numerous types of clinical end users will provide a wide range of experiences and opinions that are expected to cover a spectrum of outcomes. The evaluation team believes that three interviews per implementation site will capture this range of experiences while not overburdening any one implementation site. As this is an evaluation using purposive sampling to identify potential interviewees, findings from the clinical end user interviews will be representative only of the opinions of the interviewees themselves and will be noted as such in the final reports culminating from this information collection.
No unusual problems are anticipated for this information collection.
Key Project Staff/Stakeholders: As the potential interviewees received grant funding for their work on these projects through the PEHRIIE cooperative agreement program, there is an implicit personal benefit to those individuals and their employers in assessing the effectiveness of both the larger project in general and their work specifically. Moreover, states funded through the PEHRIIE program were informed that the CDC would be performing an evaluation of their planned project activities as part of the cooperative agreements; thus nearly 100% participation by stakeholders is anticipated. Every effort will be made to schedule site visits to maximize the availability of key project staff and stakeholder interviewees for in-person interviews. In cases where an interviewee is unavoidably not available during the site visit, the interviewee will be given the opportunity to participate in an interview with the CDC evaluation team via conference call. Therefore, this information collection surpasses the required 80% response rate required by OMB.
Clinical End Users: The primary PEHRIIE project coordinators in the funded states will identify individuals at each implementation site that will aid in recruiting participants for the clinical end user interviews at each site, as described in detail in section B2. The CDC evaluation team will provide materials to the recruiters at each implementation site to be used during the recruiting process (Attachment 10). These materials will describe the evaluation project and the ideal interviewee for this information collection while emphasizing the importance of increasing the use of the PDMP for combatting the prescription drug epidemic and the need for feedback from these particular communities of clinical end users. However, consultation with individuals who were involved in a series of smaller pilot-projects that preceded the PEHRIIE program and that were funded by the Office of the National Coordinator for Health Information Technology (ONC) and managed by the MITRE corporation, including CDR Lee from SAMHSA, Mr. Hammer from MITRE, and the project coordinators of some of those projects, indicates improved PDMP data availability and accessibility is of high concern for medical providers and pharmacists. Moreover, as detailed in section B1, the potential respondent pool is quite large relative to the planned number of interviews with clinical end users. Finally, in the event that interviews cannot be scheduled with three clinical end users during a given site visit, interviewees from that site will be given the option to participate in phone interviews with the evaluation team at their convenience instead. As such, difficulties recruiting clinical end user interviewees are expected to be minimal. The evaluation anticipates completing close to 100% of the planned clinical end user interviews, surpassing the required 80% response rate required by OMB.
The Key Project Staff/Stakeholder interview guide (Attachment 3) was pilot tested with three individuals to establish the time burden required to complete the interview and to identify any potentially problematic questions. The pilot-interviews were conducted with the primary project coordinators from three PEHRIIE states who were willing and able to be interviewed in person during an unrelated conference that members of the CDC evaluation team were also attending, thus minimizing the time and cost burden for conducting these pilot-interviews. Minor changes were made to the wording and order of the questions to reflect the feedback garnered from the pilot interviewees. As such, no further pre-testing of this information collection instrument is planned. If it becomes apparent that significant changes to the interview guide are needed during the information collection, the information collection will be temporarily suspended and a modified interview guide will be submitted for OMB approval.
The Clinical End User interview guide (Attachment 4) was pilot tested with three individuals to establish the time burden required to complete the interview and to identify any potentially problematic questions. Potential clinical end user pilot interviewees were primarily identified by Mr. Hammer from MITRE as individuals currently using the PDMP systems that were established through the ONC-funded pilot projects that preceded the PEHRIIE program. Additional potential clinical end user pilot interviewees were identified by Mr. Allain, the PDMP administrator in Indiana, a state that participated in the ONC-funded pilot program. Pilot-interviews were conducted via conference call with one pharmacist and two prescribers who are all registered, active users of their states’ PDMPs. Minor changes were made to the wording and order of the questions to reflect the feedback garnered from the pilot interviewees. As such no further pre-testing of this information collection instrument is planned. If it becomes apparent that significant changes to the interview guide are needed during the information collection, the information collection will be temporarily suspended and a modified interview guide will be submitted for OMB approval.
The CDC evaluation team was responsible for the design of the evaluation of the PEHRIIE program and the projects funded thereby. The evaluation team is led by Chris Jones, LCDR U.S. Public Health Service and CDC Prescription Drug Overdose Team Lead, assisted by two CDC ORISE Fellows, one CDC Evaluation Fellow, and two subject matter experts (SMEs) at Brandeis University working with CDC through Intergovernmental Personnel Act (IPA) agreements. In-person data collection will be conducted by at least two evaluation team members. Data analysis will be conducted primarily by the ORISE and Evaluation Fellows under the direction of the team lead and the SMEs. Below is a list of the names, roles, and contact information for the members of the CDC evaluation team:
Christopher Jones, PharmD, MPH - CDC Team Lead
LCDR, U.S. Public Health Service
Centers for Disease Control and Prevention
4770 Buford Highway, MS F-62
Atlanta, GA 30341
Phone: (770) 488-3944
Email: cjones@cdc.gov
Peter Kreiner, PhD - Subject Matter Expert
Institute for Behavioral Health and PMP Center of Excellence
Brandeis University
415 South Street, MS #035
Waltham, MA 02454
Phone: (781) 736-3945
Email: pkreiner@brandeis.edu
Cindy Thomas, PhD - Subject Matter Expert
Schneider Institutes for Health Policy
Brandeis University
415 South Street, MS #035
Waltham, MA 02454
Phone: (781) 736-3921
Email: cthomas@brandeis.edu
Kristen Cincotta, PhD - ORISE Fellow
Centers for Disease Control and Prevention
4770 Buford Highway, MS F-62
Atlanta, GA 30341
Phone: (770) 488-0565
Email: kcincotta@cdc.gov
Sausan El Burai Félix, MPH - ORISE Fellow
Centers for Disease Control and Prevention
4770 Buford Highway, MS F-62
Atlanta, GA 30341
Phone: (770) 488-3956
Email: SElBuraiFelix1@cdc.gov
Linda Vo, MPH, CHES - Evaluation Fellow
Centers for Disease Control and Prevention
4770 Buford Highway, MS F-62
Atlanta, GA 30341
Phone: (770) 488-3967
Email: LindaVo@cdc.gov
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | CDC User |
| File Modified | 0000-00-00 |
| File Created | 2021-01-28 |