Form B1 Appendix B1. HSA Interview Guide

Evaluation of Learning Health Systems K12 Training Program

Appendix B1. Health System Advisor (HSA) Discussion Guide

May 18, 2020

Health System Advisor Discussion Guide

Estimated Time: 60 minutes

Introduction

Thank you for taking the time to speak with us today as a part of the evaluation being conducted by 2M Research (2M) on behalf of the Agency for Healthcare Research and Quality (AHRQ). The purpose of this study is to learn more about your experience supporting the learning health system (LHS) K12 training program as an advisor or mentor to an LHS scholar. We are speaking with other health system advisors at the 11 grantee institutions. At each of the institutions, we will also be speaking with scholars about their experiences. AHRQ will use this information to ensure that program goals are met, to improve upon the program, and to understand the value of the LHS K12 training program to its stakeholders, like you.

During this interview, you will be asked a series of questions about your perspectives on the LHS K12 training program. Our discussion should last approximately 60 minutes. There are no right or wrong answers—we just want to know about your experiences with the LHS K12 training program. You can refuse to answer any questions you do not want to answer. Your participation in this study is voluntary, and you may stop participating at any time. There will be no negative consequences if you choose to stop or if you choose not to participate at all.

All interview data will be treated in a secure manner and will not be disclosed, unless otherwise compelled by law. The information 2M collects will be aggregated and summarized for reporting to AHRQ and the public. Information about your institution’s program and comments from other health system advisors and staff will be aggregated. All analyses will be conducted using a de-identified data file.

There are no risks to those who participate. The benefit of participating is that your experiences will help AHRQ understand how the training programs are designed, as well as the successes and barriers encountered while implementing the training program.

If you have questions about this study after this interview, please contact Dr. Amy Windham, the evaluation lead, at awindham@2mresearch.com or 703-214-1512.

Before we begin, we would like to have your permission to audio record our discussion so that we can ensure our notes are accurate and complete. The audio recordings will be deleted once the project is complete.

Can we audio record this conversation?

Do you have any other questions before we begin?

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is XXXX-XXXX. Public reporting burden for the collection of information is estimated to average 60 minutes per response. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: AHRQ Reports Clearance Officer, Attention: PRA, Paperwork Reduction Project (XXXX-XXXX), AHRQ, 540 Gaither Road, Room #5036, Rockville, MD 20850.

Respondent Background

Let’s go ahead and get started! These first few questions are about you.

1. Would you start by telling me a little bit about your role and position with (name of respondent’s organization)?

Probe: How long have you been in this role?

2. What is your role at (name of respondent’s organization) in influencing health system decisions on clinical service delivery, quality improvement activities, or operations?

3. What is your role in relation to the LHS K12 training program?

Research Carried Out in Health Systems

I would like to ask you a few questions about your opinions about the use of research in your health system. When we talk about research, this could include clinical research, health services research, health systems research, or operations-related research, and others.

3. How important is it for the research carried out in your health system to reflect system priorities?

3. Do you think other leaders in your health system share this view about research, or do you think they have a different view?
   
   
   

Probe: In what ways do you think their view differs?

3. To what extent are your health system’s priorities reflected in the research currently being carried out in your health system?

Next, I’m going to ask you some questions about how your health system uses research conducted by LHS K12 scholars.

3. Has the research carried out by LHS K12 scholars impacted patient care in your health system?

Probe: If yes, how was patient care impacted by the research?

3. Has the research carried out by LHS K12 scholars informed quality improvement efforts (in your health system)?

Probe: If yes, in what ways did the research inform quality improvement?

3. Has the research carried out by LHS K12 scholars informed operations (in your health system)?

Probe: If yes, in what ways did their research inform operations?

3. Has the LHS K12 training program produced research/evidence that reduced the cost of care (in your health system)?

Probe: If yes, what evidence was found, and how did it impact cost of care?

Patient/Family Engagement in Research

My next set of questions are related to patient and family participation in research.

3. In your opinion, how important is it that patients and their families participate in research carried out in your health system?

If the respondent asks, the following are examples of how patients and families may participate in research:

• Determining research topics or research questions

• Providing input on the execution of research projects

• Reviewing research results and providing input on their meaning or implications
  
  
  

3. How does your health system support or facilitate patient and family engagement in the research carried out in the health system?

3. Has the LHS K12 training program impacted the capacity of your health system to engage patients and families in research?

Probe: If yes, in what ways has capacity been impacted?

3. Has the LHS K12 training program impacted your health system leadership’s attitudes toward patient and family engagement in research?
   
   
   

Probe: If yes, how have those attitudes changed?

Other System Stakeholder Engagement

The next few questions are about other health system stakeholders who may be involved with the LHS K12 training program.

3. Are there other key stakeholders within your health system who are engaged in the research being carried out by LHS K12 scholars?

Probe: If yes, what are the roles of these key stakeholders within your health system? (e.g., clinical staff, nurses, technicians)

If no, skip to question 17

3. How are these other stakeholders engaged in research?

Probe:

• Do they provide input on research questions or help determine research topics?

• Do they provide input or facilitate the execution of research projects?

• Do they review research results and provide input on their meaning, implications, or implementation?
  
  
  

LHS K12 Training Program Impact on the Health System

The next questions are about the impacts of the LHS K12 training program.

3. How has your involvement with the LHS K12 training program shifted your view of how research should be conducted in your health system?

Probe: If no: why do you think your views have not shifted?

3. How has your involvement with the LHS K12 training program shifted your view of how research results should be used by your health system?

Probe: If no: why do you think your views have not shifted?

3. Do you think the LHS K12 training program has impacted how other leaders in your organization value research conducted in your health system?

Probe: If yes: In what ways do you believe their values have changed?

3. How has the LHS K12 training program impacted the extent to which system priorities are reflected in research conducted in the system?

Probe: What are some examples?

3. Has the LHS K12 training program promoted research focused on quality improvement?

Probe: If yes, in what ways did the training program promote research on quality improvement?

3. Has the LHS K12 training program impacted the design or execution of other research carried out in your health system—that is, research carried out by non-LHS K12 scholars?

Probe: If yes, how would you describe these changes?

3. What evidence have you seen that health system leaders are more engaged in the dissemination of research as a result of the LHS K12 training program?

3. What aspects of the LHS K12 training program contributed to changes in how research is conducted or how research results are used by your health system?

Sustainability of LHS K12 Training Program

We only have a few more questions. The last questions are about the LHS K12 training program moving forward.

3. Would you like to see the LHS K12 training program sustained or continued in your health system?

Probe: Why or why not?

3. What sources of funding would be required to sustain it?
   
   
   

4. Besides funding, what other resources would be required to sustain it?
   
   
   

5. Do you envision your health system funding or co-producing LHS researchers/scientists in the future?

Probe: Why or why not?

3. Do you envision your health system providing other resources or financial support for LHS research?

Probe: Why or why not?

3. What changes to the LHS K12 training program would make it more responsive to your health system’s needs or priorities?
   
   
   

4. What changes to your health system are needed to improve the potential value of the LHS K12 training program to the system?

Closing

Is there anything else you would like to share with us regarding your experience as a health system advisor in the LHS K12 training program?

We appreciate the time you have taken to answer our questions. Your input is very valuable. If you have additional questions following this interview or if you think of anything else you would like to add, please feel free to email us.