HCBS Database-OMB Statement B-FINAL_11.26

SUPPORTING STATEMENT

Part B

Agency for Healthcare Research and Quality’s (AHRQ)

Consumer Assessment of Healthcare Providers and Systems (CAHPS^®)

Home and Community Based Services (HCBS) Survey Database

November 2018

Agency for Healthcare Research and Quality (AHRQ)

Contents

1. Description of Respondent Universe 3

2. Information Collection Procedures 4

3. Methods to Maximize Response Rates 6

4. Tests of Procedures 7

5. Statistical Consultants 7

Supporting Statement B

Adult HCBS Survey Database

1. Description of Respondent Universe

The CMS and AHRQ Consumer Assessment of Healthcare Providers and Systems Adult Home and Community Based Services Survey (Adult HCBS) Database will serve as a central repository of survey results submitted by users of the Adult HCBS survey. Through its contractor, AHRQ will host the Adult HCBS database. The database will be comprised of data that are voluntarily submitted by state Medicaid programs that have administered the survey, and as such the database will not be representative of all state Medicaid programs using the Adult HCBS survey.

The CAHPS^® Home and Community-Based Services Survey is the first cross-disability survey of home and community-based service beneficiaries’ experience receiving long-term services and supports. It is designed to facilitate comparisons across state Medicaid HCBS programs throughout the country that target different adults with disabilities, e.g., including frail elderly, individuals with physical disabilities, persons with developmental or intellectual disabilities, those with acquired brain injury and persons with severe mental illness.

The HCBS CAHPS Survey was developed by CMS for voluntary use by state Medicaid programs, including both fee-for-service HCBS programs as well as managed long-term services and supports (MLTSS) programs. States with adequate sample sizes may consider using survey metrics in value-based purchasing initiatives.

The HCBS CAHPS Survey is a questionnaire with a maximum of 69 core items developed for measuring beneficiary experience with the Medicaid home and community-based services and supports delivered by providers. Core questions cover topics such as: getting needed services, communication with providers, case managers, choice of services, medical transportation, and personal safety, as well as community inclusion and empowerment. The survey also includes 3 cognitive screening questions, 9 questions to identify the relevant HCBS services to be asked about, and 15 demographic questions. A 21-item module on experience with employment services is offered as a separate supplement. The survey was developed to be administered by an interviewer in person or by telephone. English and Spanish versions of the instrument are available. Among other metrics, the survey items support composite measures. Composite measures provide beneficiary experience information on HCBS staff reliability, communication with HCBS staff, getting help from case managers, choice of services, personal safety, adequacy of medical transportation, and community inclusion and empowerment.

Case-Mix Adjustment. Case-mix refers to respondent health status and other socio-demographic characteristics that have been shown to affect composite and rating item scores of programs surveyed. Characteristics used to case-mix adjust Adult HCBS program scores, where applicable, will be determined by best practices using the Adult HCBS survey.

Testing for Statistical Differences. The individual private feedback reports will test for statistically significant differences between mean scores and ratings of individual programs and the mean of all program means in the Adult HCBS Database using the t-test. A significance level of 0.05 or less is considered statistically significant. The mean scores will be adjusted for case-mix differences before the statistical tests are applied. HCBS program mean scores will be denoted to show if they are significantly higher than the overall mean, significantly lower than the overall mean or not significantly different than the overall mean.

2. Information Collection Procedures

Information collection for the AHRQ Adult HCBS Database will occur in an annual data collection cycle beginning in 2019. Information collection procedures for submitting and processing data are shown in Figure 1.

Figure 1. Adult HCBS Database Data Submission

Step 1: Call for Data Submission. Announcements about the opening of data submission are disseminated through various sources. AHRQ’s electronic newsletter and communications target approximately 50,000 subscribers. In addition, the AHRQ CAHPS News and Events listserv targets approximately 40,000 subscribers. Reminder announcements will be sent two weeks after the initial call for data submission. In addition, the AHRQ CAHPS Web site posts public information about the yearly timeline and instructions for data submission. Through these efforts, HCBS programs will be made aware of and invited to submit their survey data voluntarily to the Database. As the administrator of the Database and under contract with AHRQ, the contractor provides free technical assistance to programs and their survey vendors through a dedicated email address (CAHPSDatabase@Westat.com that is routed to Westat) and via toll-free phone number (888-808-7108).

Step 2: Registration for Potential Participants. A secure data submission Web site will allow interested programs, to register and submit data. The interested program’s point-of-contact (POC), will complete a number of data submission steps and forms. First, the online one-page Registration Form will take approximately 5 minutes to complete (see Attachment A). After registering, and if deemed eligible to submit data, an email will be sent to authenticate the account and to update the user password (see Attachment E, Email #1).

Once users are registered and have an account password, they will be able to access the main page menu of the secure Web site. Information about eligibility requirements, data use agreements, and data file specifications regarding how to prepare their data for inclusion in the Adult HCBS database will be posted and can be reviewed.

Step 3: Enter HCBS Program Information. This step requires each program that wishes to voluntarily submit their Adult HCBS survey data, to submit the requested characteristics such as the name of the program, population served, number of enrollees, state, etc.

Step 4: Upload Data Use Agreement (DUA). To protect the privacy of all participating programs, a duly authorized representative from the program must sign a data use agreement (DUA) (see Attachment B). The DUA language was reviewed and approved by AHRQ’s general counsel and asserts that the program’s data will be handled in a secure manner using necessary administrative, technical, and physical safeguards to limit access to the data and maintain its confidentiality. In addition, the DUA explains that the data are used only for the purposes of the Database, that only aggregated results will be reported, and that the program will not be identified by name. Data are not included in the database without this signed DUA. Users can submit the DUA via fax, postal mail, or upload a copy of the signed agreement to an online portal.

Step 5: Upload Questionnaire. Each program must submit a copy of the questionnaire used. The CAHPS Database reviews the questionnaire to ensure that it meets Adult HCBS Survey standards (the survey instrument must include all core questions, not alter the wording or order of any core questions, and must not omit any of the survey items related to characteristics that are used for case mix adjustment). Following review by CAHPS Database staff, an email notification will be sent to the registrant within three business days indicating approval or rejection of the survey. Only those programs that receive questionnaire approval may subsequently submit data files (see Attachment E, Email #8).

Step 6: Secure Online Data Submission. To enable participants to transmit their Adult HCBS survey data to the contractor in a secure manner, an online data submission extranet will be developed. The online system will be designed to accept survey data collected by the Adult HCBS survey. Survey data are to be submitted in the required data file format. Data files must conform to the Data File Layout Specifications provided by the Adult HCBS Database.

Data File Approval. Once a data file is successfully uploaded, a program written in Visual Basic (VB) will read the submitted files and load them into a SQL database that stores the data. Upon submission, a data file status report will be produced and made available to the participant. This report will display a frequency distribution for each survey item and identify any out-of-range values or other data errors. If data file errors are detected, the submitter must review the Data File Status Report for further detail. Participants are expected to correct any errors and resubmit their data file(s) for processing. If the data have been properly received, a CAHPS Database staff member then reviews the report for additional data quality checks. If any data problems are discovered, users will be notified immediately along with a description of the problem. If there are no problems with the data file, or with other aspects of the submission, the CAHPS Database staff will grant final approval status and an email will be sent to the participant contact. This email message will indicate that their data will be included in the Adult HCBS Database (see Attachment E, Email #10).

Step 7: Account Status. Participants have the opportunity to check the status of their account at any time during the submission process. Only those data receiving final approval by the CAHPS Database staff will be included in the Adult HCBS Database.

3. Methods to Maximize Response Rates

AHRQ and CMS, via their websites, will make a number of guidance documents and other materials available to assist programs with the Adult HCBS. CMS provides a Technical Assistance Guide, which provides instructions and tips about constructing the sampling frame, choosing the sample, maintaining confidentiality, collecting the data, tracking returned questionnaires and calculating the response rate.

The CAHPS User Network is the principal source of CAHPS survey products, information about CAHPS-related products and services, technical assistance for survey users, and networking opportunities for users and researchers. The User Network is funded by the U.S. Agency for Healthcare Research and Quality

The CAHPS User Network promotes their databases to encourage data submission in a number of ways:

1. Send Govdelivery messages to participants who registered for CAHPS News and Events.

2. Organizational partners and stakeholders that have national reach to programs;

3. Users that have contacted the HCBS-CAHPS technical assistance helpline about the HCBS-CAHPS survey;

4. Other outlets such as national Webcasts and conferences.

As noted earlier in this document under Information Collection Procedures, Step 1 – Call for Data Submission, announcements about the opening of data submission are circulated through various sources as a mechanism to boost program participation in the Database. AHRQ’s electronic newsletter and communications efforts target approximately 50,000 subscribers. In addition, the AHRQ CAHPS News and Events listserv targets approximately 40,000 subscribers. AHRQ, through its contractor Westat, provides free technical assistance to users through a dedicated email box and toll-free phone number. In addition, email messages are sent to database registrants to remind them about data submission deadlines.

4. Tests of Procedures

The CAHPS Database staff will speak with submitters about their experience and use their feedback to improve the collection process.

5. Statistical Consultants

Joann Sorra, PhD

Westat

1600 Research Blvd.

Rockville, MD 20850

joannsorra@westat.com

Naomi Yount, PhD

Westat

1600 Research Blvd.

Rockville, MD 20850

naomiyount@westat.com

List of Attachments:

Attachment A: Adult HCBS Registration Form

Attachment B: Adult HCBS Database Data Use Agreement

Attachment E: Submission Emails

8