CMS-10871 HQR User Language Card Sort

Request for Approval under the “Generic Clearance for the Center for Clinical Standards and Quality IT Product and Support Teams” (OMB Control Number: 0938-1397)

TITLE OF INFORMATION COLLECTION: Hospital Quality Reporting User Language Card Sort

PURPOSE OF COLLECTION:

What are you hoping to learn / improve? How do you plan to use what you learn? Are there artifacts (user personas, journey maps, digital roadmaps, summary of customer insights to inform service improvements, performance dashboards) the data from this collection will inform? 

Users have expressed confusion over some of the language the HQR System uses when referencing hospitals, ASCs, and other HQR organization types. We’d like to find out how prevalent this confusion is, what alternative language might exist, and if any of those alternatives could replace our current terminology.

We’ve conducted interviews with stakeholders to establish potential alternatives and would like to test current language against these potential new terms.

Research Goals

• Determine whether “provider” makes sense to users as HQR uses it

• Discover what language users employ to refer to CAHs, ASCs, hospitals, etc.

• Determine whether “facilities” could work for users as an aggregate term

Main questions the study will research

• Is the word “provider” confusing to users when we use it to mean “hospital” on HQR?

• What does “provider” mean to our users?

• Is “facility” a good alternative catch-all phrase?

• How do users understand the language we’re currently using?

• Can we communicate more effectively with users in future designs?

TYPE OF COLLECTION: (Check one)

[X] Card Sorting [ ] Cognitive Testing

[ ] Field Studies [ ] First Click Tests

[ ] Focus Groups [ ] Participatory Design

[ ] Survey [ ] Tree Testing

[ ] User Interviews [ ] Usability Testing

[ ] Other: _______________________

CERTIFICATION:

I certify the following to be true:

1. The collection is voluntary.

2. The collection is low-burden for respondents and low-cost for the Federal Government.

3. The collection is non-controversial and does not raise issues of concern to other federal agencies.

4. The results are not intended to be disseminated to the public.

5. Information gathered will not be used for the purpose of substantially informing influential policy decisions.

6. The collection is targeted to the solicitation of opinions from respondents who have experience with the program or may have experience with the program in the future.

Name: Danita Patel (Product Lead for HQR HCD)

To assist review, please provide answers to the following question:

PERSONALLY IDENTIFIABLE INFORMATION

1. Is personally identifiable information (PII) collected? [ ] Yes [X] No

2. If Yes, will any information that is collected be included in records that are subject to the Privacy Act of 1974? [ ] Yes [] No

3. If Yes, has an up-to-date System of Records Notice (SORN) been published? [ ] Yes [ ] No

GIFTS OR PAYMENTS

Is an incentive (e.g., money or reimbursement of expenses, token of appreciation) provided to participants? [ ] Yes [X] No

If Yes, describe:

BURDEN HOURS

Category of Respondent	No. of Respondents	Participation Time	Burden
(2) Private Sector (HQR End Users)	75	0.25 hours	19 Hours
Totals			19 Hours

FEDERAL COST

The estimated annual cost to the Federal government is ____________

There is no additional cost for this form implementation, as it is part of the Hospital Quality Reporting contract scope that has already been awarded.

ACTIVITY DETAILS

1. How will you collect the information? (Check all that apply)

[X] Web-based or other forms of social media

[ ] Telephone

[ ] In-person

[ ] Mail

[ ] Other, Explain.

2. Will interviewers or facilitators be used? [ ] Yes [X] No

3. Who will you collect the information from?

Describe the people you will interact with or collecting information from and why the group is appropriate for the program / service to connect with. Please provide a description of how you plan to identify your potential group of respondents and if only a sample will be solicited for feedback, how you will select them (e.g., anyone who provided an email address to a call center representative, a representative sample of administrators who downloaded a report in May 2021, intercept interviews at a particular field office, a list of customers, e.g., a CRM database that has contact information, to reach out to that defines the universe of potential respondents and have a sampling plan for selecting from this universe). Attach a copy of your sampling plan if applicable.

We will collect this information from the end users of the HQR system. These users interact directly with HQR and consist of hospital quality directors and specialists who oversee HQR program requirements and third party vendors who submit data on behalf of these hospitals.

These users will have previously indicated willingness to participate in HQR user research.

4. How will you ask a respondent to provide this information?

For example, after an inquiry is submitted online, the final screen will present the opportunity to provide feedback by presenting a link to a feedback form / an actual feedback form.

We will use a web-based user research tool called Optimal Workshop. Users will answer a few general questions about their workplace, complete a card sort, then answer a handful of questions about language.

5. What will the activity look like?

Describe the information collection activity – e.g., what happens when a person agrees to participate? Will facilitators or interviewers be used? What is the format of the interview/focus group? If a survey, describe the overall survey layout/length/other details. If User Testing, what actions will you observe / how will you have respondents interact with a product you need feedback on. 

Clicking on the link to the study will open the Optimal Workshop study. The first three questions are all on one page. They will then select the “next” button and be brought to the card sort. Users will sort words like “doctor” and “facility” into groups of their own choosing. They will be able to name the groups. When they have finished this they will be asked individual questions, one per page, that are asked in a random order. They will then finish the study and be thanked.

6. Please provide your question list.

Paste here the questions or prompts presented to participants in your activity. If you have an interview / facilitator guide, that can be attached to the submission and referenced here.

1. What’s your job title?

2. How many beds does your workplace have?

   • 25 or less

   • 25 - 100

   • 100 - 250

   • 250 - 500

   • More than 500

3. Does your workplace qualify as any of the following?

   • Ambulatory Surgical Center (ASC)

   • Critical Access Hospital (CAH)

   • Rural hospital

   • Rural Emergency Hospital

   • Healthcare System

   • None of the above

4. What type of organization are you primarily associated with?

   • Single Hospital / Facility

   • Healthcare System / Network of Facilities

   • Vendor contracting with hospitals

   • Vendor contracting with CMS

   • Ambulatory surgical center

   • Quality Improvement Organization / Network

   • Other

5. How many facilities are part of your hospital system or facility network? (question only asked if participant selects option B for Question 4)

6. Just about finished

   • Please answer the following questions

[the following questions will be asked in an order randomize by Optimal Workshop]

7. HQR users submit measure data to CMS on behalf of _______________ with CCNs or NPIs.
   Fill in the blank

8. I use the following to describe my workplace:

   • Facility

   • Hospital

   • Provider

   • Organization

   • Healthcare facility

   • Other

9. To me, the word "provider" means __________
   Fill in the blank

10. The word "provider" means a hospital or other medical organization. Select one

    • Strongly agree

    • Agree

    • No opinion

    • Disagree

    • Strongly disagree

11. The phrase "healthcare facility" describes where I work.

    • Strongly agree

    • Agree

    • No opinion

    • Disagree

    • Strongly disagree

12. The phrase "facility" describes where I work.

    • Strongly agree

    • Agree

    • No opinion

    • Disagree

    • Strongly disagree

Please make sure that all instruments, instructions, and scripts are submitted with the request.

7. When will the activity happen?

Describe the time frame or number of events that will occur (e.g., We will conduct focus groups on May 13,14, 15; We plan to conduct customer intercept interviews over the course of the Summer at the field offices identified in response to #2 based on scheduling logistics concluding by Sept. 10; or This survey will remain on our website in alignment with the timing of the overall clearance.)

We will be implementing the survey in late Fall/Winter 2023. This timeline is contingent on PRA approval and developer capacity to complete the build of the web-based form and implementation in the HQR production environment. The survey will remain on our website in alignment with the overall clearance.

Instructions for completing Request for Approval under the “Generic Clearance for the Center for Clinical Standards and Quality IT Product and Support Teams”

TITLE OF INFORMATION COLLECTION: Provide the name of the collection that is the subject of the request (e.g. Comment card for soliciting feedback on xxxx).

PURPOSE: Provide a brief description of the purpose of this collection and how it will be used. If this is part of a larger study or effort, please include this in your explanation.

TYPE OF COLLECTION: Check one box. If you are requesting approval of other instruments under the generic, you must complete a form for each instrument.

CERTIFICATION: Please read the certification carefully. If you incorrectly certify, the collection will be returned as improperly submitted or it will be disapproved.

PERSONALLY IDENTIFIABLE INFORMATION: Provide answers to the questions. Note: Agencies should only collect PII to the extent necessary, and they should only retain PII for the period that is necessary to achieve a specific objective.

GIFTS OR PAYMENTS: If you answer yes to the question, please describe the incentive, and provide a justification for the amount.

BURDEN HOURS:

Category of Respondents: Identify who you expect the respondents to be in terms of the following categories: (1) Individuals or Households; (2) Private Sector; (3) State, local, or tribal governments; or (4) Federal Government. Only one type of respondent can be selected per row.

No. of Respondents: Provide an estimate of the Number of respondents.

Participation Time: Provide an estimate of the amount of time required for a respondent to participate (e.g., fill out a survey or participate in a focus group).

Burden: Provide the Annual burden hours: Multiply the number of responses and the participation time and divide by 60.

FEDERAL COST: Provide an estimate of the annual cost to the Federal government.

ACTIVITY DETAILS: Complete each section as described.

If you are conducting a focus group, survey, or plan to employ statistical methods, please provide a description of how you plan to identify your potential group of respondents and how you will select them.

Submit all instruments, instructions, and scripts are submitted with the request.

10