6 Appendix F: Pilot Test Interview Protocol for Providers

Appendix F – Pilot Test Interview Protocol for Providers with TeamSTEPPS® Experience

AHRQ – Building Diagnostic Safety Capacity

Provider Interviews – TeamSTEPPS® Course Pilot Test Evaluation

MedStar Health Research Institute (MHRI) will conduct interviews and/or focus groups with providers with TeamSTEPPS® experience from up to ten (n=10) settings. Evaluation will be completed within 3-6 months after implementation of the TeamSTEPPS® Course.

• 60 cognitive interviews with providers (up to 6 providers per setting x 10 settings); each interview will last approximately 45 minutes

Recruitment Criteria

We will aim to recruit providers and staff in the following manner:

• Providers who have been exposed to the TeamSTEPPS® Course to improve communication among providers related to diagnosis

• Providers who have prior experience using TeamSTEPPS® strategies

• Diversity among practice staff and providers

MHRI staff will work with the practice coordinators to identify individuals to participate in the interviews.

Interview Goals

The goals of the focus groups/interviews will be to:

• Obtain feedback on the Course materials

• Obtain feedback on the barriers and facilitators encountered for the Course

• Obtain feedback on satisfaction with the Course

• Obtain feedback on receptivity and enhancements to the Course to improve adoption

Materials

• Copies of the TeamSTEPPS® Course materials

• Informed consent documents

• Digital recorder

Location

Interviews will take place at the information collection setting at a time convenient to the provider and/or staff members. Interviews may also take place over the phone or other videoconferencing software to enhance ability to recruit and retain clinicians.

Informed Consent Procedures

Participants will complete the informed consent process prior to starting the interview.

Participant Stipends

None.

Each interview will take no more than 45 minutes.

Form Approved
OMB No. xxxx-xxxx
Exp. Date xx/xx/20

AHRQ – Building Diagnostic Safety Capacity

Provider Interviews – TeamSTEPPS® Pilot Test Evaluation

WELCOME AND INTRODUCTION

• Thank you for agreeing to speak with me today.

• My name is [INSERT NAME OF INTERVIEWER] and I am here to ask you a few questions about the AHRQ’s TeamSTEPPS® Course to improve communication among providers related to diagnosis.

• With your permission we will be audio recording the session. This will help make sure that we don’t miss anything that you say and can share with other people who are working on this project. The recording will be deleted after we have the notes transcribed and we are sure that all of your comments are accurately captured.

• TODAY/TONIGHT I will be asking you questions about your experiences with the AHRQ’s TeamSTEPPS® Course to improve communication among providers related to diagnosis.

• Everything you say here will be kept confidential and included as part of our assessment of the feasibility of implementing the TeamSTEPPS® Course into practice. We will not share your name or attribute any of your words directly to you.

• Do you have any questions before we begin?

• Ok, great. Let’s get started.

This survey is authorized under 42 U.S.C. 299a. The confidentiality of your responses to this survey is protected by Sections 944(c) and 308(d) of the Public Health Service Act [42 U.S.C. 299c-3(c) and 42 U.S.C. 242m(d)].  Information that could identify you will not be disclosed unless you have consented to that disclosure. Public reporting burden for this collection of information is estimated to average 15 minutes per response, the estimated time required to complete the survey. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number.  Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: AHRQ Reports Clearance Officer Attention: PRA, Paperwork Reduction Project (0935-XXXX) AHRQ, 5600 Fishers Lane, Room #07W42, Rockville, MD 20857.

Experience with TeamSTEPPS®

1. Before I ask you about the specific materials for this course, please rate your experience using TeamSTEPPS®:

• Novice

• Intermediate

• Expert

• TeamSTEPPS® Master Trainer

General TeamSTEPPS® Course Questions

2. When you were first given the TeamSTEPPS® Course what did you think?

   1. How did you feel about using it?

   2. Can you describe your practice’s implementation process?

   3. How did you make decisions on which elements of the TeamSTEPPS® Course to use?

3. How easy or challenging did you find the TeamSTEPPS® Course implementation to be?

   1. Were any elements of the TeamSTEPPS® Course that were easier or harder to implement? If yes, can you describe which ones and what made them more challenging?

What can we do to make them more effective? Easier to implement?

4. What was the best thing about the TeamSTEPPS® Course from your perspective?

5. What was your least favorite thing about the TeamSTEPPS® Course?

6. What would you have changed about the TeamSTEPPS® Course to make it more user friendly?

7. What do you think your patients felt about their diagnostic teams following implementation of this Course?

   1. Can you give me an example of a patient experience that was positive?

   2. How about a negative one? Is there anything that could have been done to make it a more positive experience for that patient?

TeamSTEPPS® Course Modules

Thank you for your response(s). Next, I’d like to ask you some specific questions about each of the Course modules.

8. Which TeamSTEPPS® Course modules did you participate in? [Checklist]

• Module 1 – Introduction

• Module 2 – Team Structure

• Module 3 – Communications

• Module 4 – Leadership

• Module 5 – Situation Monitoring

• Module 6 – Mutual Support

• Module 7 – Pulling it All Together

Module 1: Introduction

9. From your perspective as a provider, what is the most important thing for providers to know when beginning a TeamSTEPPS® training?

   1. Did the TeamSTEPPS® Course meet your expectations for the introductory module?

   2. Is there anything about this module that you would improve/change?

10. What was your first impression after completing Module 1?

    1. Did you learn anything new about diagnostic errors from this module?

    2. Was there anything in the first module that stuck out to you as being particularly important? If yes, what?

    3. Was there something that we should have included or emphasized more? If yes, please describe.

Module 2: Team Structure

11. What is important for providers to understand about team structure?

    1. From your perspective, how can the TeamSTEPPS® course better address this?

12. How did this module impact the way you think about team structure related to diagnosis?

    1. What recommendations do you have that could make this module more impactful?

Module 3: Communications

13. From your perspective as a provider, what is the most important thing for diagnostic teams to understand about communication?

    1. What changes to the course might need to be made to better address this?

14. How did this course impact the way you apply communication skills with other diagnostic team members?

    1. With the patient?

    2. Outside your practice?

    3. What changes to the course would you make to better enhance communication among the diagnostic team?

Module 4: Leadership

15. From your perspective as a provider, what skills of effective leadership are most helpful in supporting communication among providers related to diagnosis?

    1. From your perspective, how can the TeamSTEPPS® course better emphasize these particular skills?

16. How effective was the module at meetings its learning objectives?

    1. How would you define “effective leadership related to diagnosis?

    2. What skills are most helpful to facilitating the improvement of diagnosis-related provider communication in your setting?

17. How did this module impact the leadership of your diagnostic team? Leadership in your health system related to diagnosis?

    1. What would you change about the module to make it more effective?

Module 5: Situation Monitoring

18. What do you think is the most important thing for providers to understand about situation monitoring?

    1. Situational awareness?

    2. Creating a shared mental model?

    3. How well do you feel that the course addressed this concept? What changes do you recommend we make to better highlight this topic?

Module 6: Mutual Support

19. What is most important for diagnostic teams to understand about mutual support?

    1. Can you describe how well the TeamSTEPPS® Course improved your understanding of mutual support within the diagnostic team?

20. How would you describe the overall impact of Module 6: Mutual Support on communication in the diagnostic process in your healthcare setting?

    1. With the patient?

    2. Outside your practice?

    3. How could we improve the module to make it more useful for other practices to adopt?

Module 7: Pulling it All Together

21. Please describe the overall impact of Module 7 on the implementation of this TeamSTEPPS® Course to improve communication among providers related to diagnosis.

    1. Please describe how your practice/leader used this module to support implementation.

TeamSTEPPS® Course Workbook

Now I would like you to consider the materials from the TeamSTEPPS® Course Participant Workbook that you have in front of you for this next series of questions.

22. How did your organization use the infographic during the training?

    1. What other materials would have been useful to help you or your leader introduce the topic of diagnosis, diagnostic error, and team communication?

       1. What are your thoughts about the format of the materials? Is there a better way for us to think about presenting them?

          1. To leaders? To clinicians? To administrators?

       2. Was there anything on the infographic that stuck out to you?

          1. How might we improve this to create a more powerful impact?

          2. Who should this be shared with to create the most lasting impact?

23. Can you describe how your practice used the team synchronization exercise? Was this helpful when defining your diagnostic team?

    1. Who did your practice end up including in their diagnostic team?

    2. How could we improve this tool to make it easier for other practices/organizations to adopt?

24. Which of the strategies for communication did your practice include in its training?

• Diagnostic-focused referral tool

• Tool to help communicate the diagnosis to the patient

• Call-outs

• SBAR

1. Can you describe how receptive you were to the tool(s)?

2. What could we do to improve the tool?

25. Which of the strategies introduced during the Leadership module were included in your practice/organizations implementation?

• Briefs

• Debriefs

• Huddles

1. How easy/challenging were these to learn from the materials provided?

2. Are there other materials that might be important to include (e.g., more case examples; scripts; etc.) to help facilitate learning?

26. Which of the following Module 5: Situation Monitoring materials did you find most helpful?

• Activity to develop a shared mental model

• STEP process

1. How could we improve the tools and training materials to better support the uptake of these strategies by clinicians? By the team?

27. Which of the strategies within the Mutual Support module did you receive training on? Find most helpful?

• Task assistance

• Feedback

• Two-challenge rule

• CUS

• Advocacy and assertion

• DESC script

1. How could we improve the tools and training materials to better support the uptake of these strategies by clinicians? By the team?

28. How important were the supplemental resources for your learning?

    1. Have you used any since you were trained?

    2. Would your training have benefited from including them?

       1. If yes, which ones should we include in the formal training?

29. How about the reflective practice exercise? Can you describe your experience with that?

    1. How well do you think that the reflective practice concept relates to diagnosis and diagnostic improvement?

    2. How did these concepts (Listen, Think, Act) impact the way you communicate with other providers?

       1. What about the way you communicate with staff? Patients?

30. Please describe how your practice used the Team Assessment .

    1. What could we do to improve how we explain the Team Assessment and how practices/organizations should use it?

31. What was your first reaction to the Case Story? Did this resonate with you? How important was it to have a case to help convey the opportunities for diagnostic improvement that are learned during this TeamSTEPPS® course?

TeamSTEPPS® Course Facilitator Guide [TEAM LEADERS ONLY]

32. How important was it to have the facilitators guide as part of the TeamSTEPPS® course?

    1. How could the facilitator guide be improved?

Those were all the questions I had today. Are there any questions that I should have asked that I did not?

Thank you for your time and participation in this interview. Your comments will be very helpful to this project and will help us to make important improvements to the TeamSTEPPS® Course!