1 Provider Focus Group

Form Approved

OMB No. 0935-XXXX

Exp. Date XX/XX/20XX

Provider Focus Group

Public reporting burden for this collection of information is estimated to average XX minutes per response, the estimated time required to complete the survey. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: AHRQ Reports Clearance Officer Attention: PRA, Paperwork Reduction Project (0935-XXXX) AHRQ, 5600 Fishers Lane, # 07W41A, Rockville, MD 20857.

Demographic Questions	Format
What is your age?	List: <25 25-34 35-44 45-54 55-64 65-74 >=75
How do you currently describe yourself? (Mark all that apply)	List: - Female - Male - Transgender - I use a different term (free text)
What is your race? (One or more categories may be selected)1	List: - American Indian or Alaska Native  - Asian  - Native Hawaiian or other Pacific Islander  - Black or African American  - White
Are you Hispanic or Latino/Latina?	Yes/No
What are your number of years in practice?	List: 0-4 5-9 10-19 >=20
What is your specialty?	Open Ended (format)
How familiar are you with shared care planning?	Likert

1.0 Requirements Gathering Scope 

Objective 

This testing guideline will focus on the clinician-facing application.   

This testing guideline will focus on managing patient data from multiple sources from the perspective of the treating clinician or team. Priority areas to focus on will be goals for presentation of information and issues of aggregating and prioritizing data.

This testing guideline will leverage focus group or other small group conversations with clinicians

2.0 0 Focus Group Assumptions and Constraints 

Assumptions

Participants have access to the current version of the patient-facing application at Oregon Health and Science University for review.  

Participants have access to wireframes (requirements gathering component early stages) and any early versions of a revised clinician-facing application (V 2(c)) for this evaluation.

Participants receive(d) a walkthrough of the concept of the SDS and are able to provide input on how this data can best be consumed for shared care planning.

For the usability testing component, navigation, number of clicks and time spent in application will be captured in log data. 

The questions and tasks listed under each discussion are not meant to be read/reviewed verbatim or answered by each participant. They are meant to guide the interaction.

The moderator/observer should encourage clinician participants (and dyads) to think aloud as they are discussing and/or navigating the wireframes and/or application(s).

The results of each focus group or usability testing session should be incorporated into an agile design process for clinician-facing app development  

EMI Advisors will plan to observe sessions as much as possible (for both components) and notes will be provided for incorporation into development sprints (prioritized and submitted via a ticketing or tracking system).

Constraints

Wireframes will be used initially, updated regularly and inform agile development.

Application development may coincide with this evaluation and facilitate further interaction.

Some data will not display, this guideline should try to address this constraint. 

Data that will be available in the final version of the application may not be available in the version of the application tested. 

The applications cannot write to the EHR. Clinicians and patients/caregivers are unable to edit the data contained in the EHR through the eCare Plan application.

There may be unexpected limitations on the data available from the SDS for review.  

3.0 0 Focus Group Risks
Description	Probability  High|Med|Low	Impact  High|Med|Low	Mitigation
User cannot access the applications in the production site or the live site.	Low	High	Confirm access and credentials prior to testing
User cannot access data in external system when such data is present.	Medium	Medium	Confirm access and credentials prior to testing

4a.0 0 Focus Group Team Roles & Responsibilities
Name	Roles	Responsibilities
Gina DeNoble	Moderator/Observer/Interviewer	Schedule and conduct evaluation sessions

4b.0 Participating Clinician Information
Name	Location of Testing	Clinician is comfortable using technology 1-5 with 1 [strongly disagree], 2 [disagree], 3 [neutral], 4 [agree], and 5 [strongly agree]	Clinician is experienced with using care planning apps 1-5 with 1 [strongly disagree], 2 [disagree], 3 [neutral], 4 [agree], and 5 [strongly agree]

5.0 0 Focus Group Introduction
ID
5.0	Introduction:  Before getting started, provide the participant with a brief overview:     (OBSERVER READ ALOUD) Thank you for taking the time to meet with us today. I am [OBSERVER NAME] and will be guiding you through your review of the apps and I will also be taking notes. We will begin with an overview of four major concepts: shared care planning, care planning, care coordination and shared decision-making, and discuss your initial thoughts. After we will get your initial impression of the eCare plan application system to help prioritize development of a new release of the clinician-facing application.  Concept Definitions: Shared care planning - process of collaboration among people at risk for or living with MCC, clinicians, and healthcare teams to proactively discuss and record: (1) roles and tasks among care team members, including the individual, their family and caregivers; (2) plans for coordinating care within and across organizations and settings; (3) strategies for supporting and empowering patients to manage their own health; (4) plans for engaging in shared decision making. Care planning - a collaborative process focused on discussing patient and clinical goals of care, conducting shared decision-making to identify strategies for clinical and self-management to achieve these goals based on evidence and patient preference, clarifying roles for different members of the care team, and empowering patients and caregivers  Care coordination - organization of patient care activities to facilitate appropriate and timely delivery of healthcare services by multiple clinicians in multiple care settings  Shared decision-making - when a health care provider and a patient work together to make a health care decision that is best for the patient  This is one of three expected sessions with gradually more specific goals: Stage 1: Introduction to the project, evaluation, app(s) and next steps NOTE: Clinician group, walkthrough of patient-facing app, introduction to wireframes, details of summary tab, concept of SDS, some test data Stage 2: Further discussion of data summarization and prioritization and next steps NOTE: Clinician group, advancing wireframes where possible, preview of new clinician-facing app where possible, use of synthetic data, discussion of clinician-patient dyad, agile development feedback process Stage 3: Review and prioritization of final recommendations for agile design NOTE: Expected use of preliminary V 2(c) app using realistic synthetic or de-identified data with some clinician-patient dyad engagement (low, medium and high) EXAMPLE: Today’s session will take about one hour. We will begin with an overview of the goals of the project, the goals of this evaluation and the requirements gathering process. We will review the patient-facing application with synthetic data, review some wireframes or examples of ways in which the clinician-facing application may present information and discuss the agile design process. As we move through each component, we welcome your questions and comments and encourage group interaction and problem-solving help us understand how to improve the user experience for the clinician-facing application.
5.1	Privacy:  Your participation is completely voluntary. Your name and title will not appear in any report. There are no right or wrong answers, and we are interested in your observations for how to make the applications better. Are there any questions before we begin? (You may also stop at any point in the testing to ask questions).
5.2	Introductory questions:  Let’s start with the purpose of the eCare plan app. Our goal is to facilitate person-centered (ie goal-centered) care planning for people living with MCC and improve shared decision-making between patients/caregivers and clinicians, regardless of treatment location.  We will leverage a hypothetical patient, Patricia Noelle, to foster a discussion for Stage 1.
5.2a	Next, we will demo the patient/caregiver-facing app, MyCarePlanner. Then we will review some wireframes (app mockups and/or app demos) with you and go over the evaluation, requirements gathering, and agile processes. This work will be informal and conversational. Your active participation will be encouraged wherever possible.
5.3	As we review each app and the context of the supplemental data store (SDS), please share with the group your existing healthcare visit workflow and care delivery process, suggested changes to workflow (that might include an eCare plan app) what kind of information summarization, prioritization, presentation and visualizations (ways of presenting information) would be ideal for assisting with shared care planning from your perspective.  Validate the clinician’s user journey reviewing what the clinician does before, during, and after a healthcare visit. This could be done for 2 types of appointments: regular general check-up and problem/acute illness appointment for a PCP and a specialist. This would include observing how clinicians look for information to support shared care planning, such as: Reviewing patient information Reviewing quantitative goals and associated lab values Reviewing new/updated medications Reviewing new/updated conditions/diagnoses Reviewing new encounters with other clinicians (i.e., hospitalizations) Reviewing any questionnaire responses Reading and patient personal goals Writing notes to prep for the encounter ALTERNATE: When the V 2(c) app is ready for review: Again, and to compare with first session, validate the clinician’s user journey through the app to include reviewing what the clinician does before, during, and after a care encounter with a patient who has MCC. This could be done for 2 types of appointments: regular general check-up and problem/acute illness appointment for a PCP and a specialist. This should include use of the app to: Launch clinician app from the EHR Review patient information Review quantitative goals and associated lab values Review new/updated medications Review new/updated conditions/diagnoses Review new encounters with other clinicians (i.e., hospitalizations) Review new questionnaire responses Read new patient personal goals Write notes to prep clinician for the encounter (confirm reason for visit/new reason for visit)
5.4	We are working toward an improved version of this application and hope to meet with you in Stages 1-3, would you be willing to participate throughout this process?

6.0 0 Usability Testing
ID
6.0	Usability Testing Sections:   Test 1. Load data from additional source (if applicable) Test 2. Adjudicate data from multiple sources (if applicable) Test 3. Review Goals, add a new goal Test 4. Review Activities Test 4. Review Care team Test 5. Review Medications Test 6. Review Vital signs and lab results Test 7. Review Health concerns   Workflows will also be tested, these details are TBD. Observer to Note Rough Start Time: _______________    Observer to Note Rough End Time: ________________