Form A1 Appendix A1. Scholar Interview Guide

EVALUATION of Learning Health Systems K12 Training Program

Appendix A1. Scholar Interview Guide

May 18, 2020

Scholar Discussion Guide

Estimated Time: 60 minutes

Introduction

Thank you for taking the time to speak with us today as part of the evaluation being conducted by 2M Research (2M) on behalf of the Agency for Healthcare Research and Quality (AHRQ). The purpose of this study is to learn more about your experience participating in the learning health system (LHS) K12 training program at your institution. We are speaking with other scholars from each of the 11 LHS K12 programs. This information will be used by AHRQ to ensure that program goals are met, to improve upon the program, and to understand the value of the LHS K12 training program to its stakeholders, like you.

During this interview, you will be asked a series of questions about your perspectives on the LHS K12 training program. Our discussion should last approximately 60 minutes. There are no right or wrong answers—we just want to know about your experiences with the program. You can refuse to answer any questions you do not want to answer. Your participation in this study is voluntary, and you may stop participating at any time. There will be no negative consequences if you choose to stop or if you choose not to participate at all.

All interview data will be treated in a secure manner and will not be disclosed, unless otherwise compelled by law. The information 2M collects will be aggregated and summarized for reporting to AHRQ. Information about your institution’s program and comments from other scholars and staff will be aggregated. All analyses will be conducted using a de-identified data file; your identity will not be shared.

There are no risks to those who participate. The benefit of participating in this study is that your institution’s experiences will help AHRQ understand the types of projects scholars were able to implement using the award funding, as well as the successes and barriers encountered while implementing the training program.

If you have questions about this study after this interview, please contact Dr. Amy Windham, the evaluation director, at awindham@2mresearch.com or 703-214-1512.

Before we begin, we would like to have your permission to audio record our discussion so that we can ensure our notes are accurate and complete. The audio recordings will be deleted once the project is complete.

Do we have your permission to audio record this conversation?

Okay, thank you. Do you have any questions before we begin?

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is XXXX-XXXX. Public reporting burden for the collection of information is estimated to average 60 minutes per response. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: AHRQ Reports Clearance Officer, Attention: PRA, Paperwork Reduction Project (XXXX-XXXX), AHRQ, 540 Gaither Road, Room #5036, Rockville, MD 20850.

Respondent Background

1. Would you please start by telling us a little bit about yourself as it relates to your involvement with the LHS K12 program? For example, your training experience as a clinician or research scientist and/or how long you’ve been engaged in research in general and patient-centered outcomes research in particular. You may also tell us any additional information you think would be helpful for us to know, as related to your LHS training.

2. To what extent did you function as part of your health system during your time as a scholar?

Embeddedness

Our first questions are about your experience working with health system stakeholders, including your health system advisor/mentor.

3. We know that the LHS K12 scholars may receive guidance and mentorship from a variety of people during their training and project. Please describe any health system advisors or other mentors (from the health system) who contributed substantially to your training program and/or research project.

Probes: How many health system advisors or mentors did you have?

If the respondent had multiple: What were their different roles?

(Note to interviewer: Ascertain whether there was one primary health system advisor or whether there were multiple advisors who were substantially engaged [i.e., met with regularly]. In most cases, there will be just one primary health system advisor, and the questions should refer to that person. In some cases, there may be multiple health system advisors with substantial involvement, and the respondent should answer in relation to all the advisors.)

For my next few questions, when I say “he/she” or “they/them,” I will be referring to the person/people we were just talking about. We are interested in how these advisors helped you with your training and research project.

4. Thinking about the person/people you just mentioned, on average, how often did you meet with him/her/them during your training program?

Probe: Was this sufficient for your needs? Why? Why not?

(Note to interviewer: We do not have a set metric in terms of number of times per week, month, etc., so let the respondent use their own interpretation.)

5. How did he/she/they (the individual[s] mentioned above) influence the development of your research topic?

6. How did he/she/they support the implementation of your research project?
   
   
   

7. How did he/she/they impact how your study results were disseminated?

8. How did he/she/they impact how your study results were used by the health system?

9. Were there any other ways he/she/they supported your experience with the LHS K12 training program or research project?

Probe: If yes, please describe.

10. Besides helping with your training and research project, in what other ways did he/she/they support your career development?

11. How did the LHS K12 program facilitate your engagement with your health system advisors/mentors or other relevant stakeholders (from within the health system)?

12. How did the LHS K12 program prepare you to engage your health system advisor(s) in a productive way? Please be specific if you can.

13. What challenges did you experience engaging your health system advisor(s) in your research project or training program activities?

Probe: How did you overcome those challenges?

14. What challenges did you face when trying to incorporate their input into your research?

Probe: How did you overcome those challenges?

Use of LHS K12 Research by the Health System

Next, I’m going to ask you some questions about how your research might have been used by your health system. We understand that the nature of scholars’ research projects and the timeline for your training program may not allow for direct use of research findings, but we would like to hear about any actual, planned, or potential use of your research by the health system.

15. Has your research impacted patient care in your health system?

Probe: If yes, how was patient care impacted?

Probe: If no, why not? Does it have the potential to be impacted?

16. Has your research informed quality improvement efforts (in your health system)?

Probe: If yes, in what ways did the research inform quality improvement?

Probe: If no, why not? Does it have the potential to inform quality improvement?

17. Has your research informed operations (in your health system)? (Operations includes activities related to hospital management, for example, admission or discharge procedures, case management and/or care coordination activities, or training activities.)

Probe: If yes, in what ways did the research inform operations?

Probe: If no, why not? Does it have the potential to inform operations?

18. Has your research reduced the cost of care (in your health system)?

Probe: If yes, how did it impact cost of care?

Probe: If no, why not? Does it have the potential to impact cost of care?

Patient Engagement

The next questions are about how patients and families may have been involved in your research.

19. How did your research project engage patients and families?

If it did not: Why were patients and families not engaged in your research?

Then skip to the General Experience/Career Preparation Section

If the research project DID engage patient and families, ask questions 19­–21.

20. How did the LHS K12 program facilitate patient and family engagement in your research?

Probe: How did the training program facilitate patient and family engagement?
Probe: How did the health system facilitate patient and family engagement?

21. What were the most significant barriers to engaging patients and families in your research?

22. How did the engagement of patients or families in your research impact the design or execution of your research project? This could be a positive or negative impact.

General Experience/Career Preparation

Next, I’d like to ask you some questions about your overall experience in the training program, as well as your successes and challenges.

23. What was your most significant accomplishment during the LHS K12 program?

Probe: Why does this stand out as being significant to you?

Probe: What facilitators helped you to be successful?

24. What was your biggest challenge during the LHS K12 program?

Probe: Why?

Probe: How did you overcome this challenge?

Probe: What resources were helpful in overcoming this challenge?

25. In what ways has the LHS K12 program contributed to advancing your career?

26. In what ways has the LHS K12 program prepared you for the next phase of your career?

27. How do you plan to integrate health systems research into your future work?

Probe: Do you plan to pursue/continue a health systems-related career path?

Probe: Why or why not?

Closing

Before we wrap up, is there anything else you would like to share with us regarding your experience as a scholar in the LHS K12 training program?

We appreciate the time you have taken to answer our questions. Your input is very valuable. If you have additional questions following this interview or if you think of anything else you would like to add, please feel free to email us.