Supporting Statement B_2900-0712_SHEP Surveys_final Oct 2021

SUPPORTING STATEMENT B

2900-0712

SURVEY OF HEALTHCARE EXPERIENCES OF PATIENTS (SHEP)

NATION-WIDE CUSTOMER SATISFACTION SURVEYS

B. Collections of Information Employing Statistical Methods

1. Provide a numerical estimate of the potential respondent universe and describe any sampling or other respondent selection method to be used. Data on the number of entities (e.g., households or persons) in the universe and the corresponding sample are to be provided in tabular format for the universe as a whole and for each stratum. Indicate expected response rates. If this has been conducted previously include actual response rates achieved.

Enrolled Veteran patient population ~9,500,000

Potential respondent universe: ~1,938,000 patients/year

Expected response rate: ~34% (n=659,000)

Sampling methodology: Random selection based on unique patient ID and specific encounter date and type coordinated through a VA facility

2. Describe the procedures for the collection of information, including:

a. Statistical methodology for stratification and sample selection

b. Estimation procedure

c. Degree of accuracy needed

d. Unusual problems requiring specialized sampling procedures

e. Any use of less frequent than monthly data collection to reduce burden

FORM NUMBER	SAMPLE SIZE/YR	RESP RATE	# RESP	RESP FREQ	% ELEC	BURDEN (Minutes)	BURDEN (Hours)	NOTES
10-1465-1 IP Long Form	1200	34%	408	1	0%	20	136	Inactive
10-1465-2 IP Short Form	180,000	34%	61,200	1	0%	14	14,280	Active
10-1465-3 Amb. Care LF	1200	34%	408	1	0%	26	177	Inactive
10-1465-4 Amb. Care SF	1200	34%	408	1	0%	15	102	Inactive
10-1465-5 PCMH	720,000	34%	244,800	1	30%	12	48,960	Active
10-1465-6 PCMH LF	60,000	34%	20,400	1	0%	21	7,140	Inactive
10-1465-7 HHCAHPS LF	1,200	34%	408	1	30%	11	75	Active
10-1465-8 ICH-CAHPS LF	1,200	34%	408	1	30%	16	109	Active
10-1465-9 Spec Care SF	672,000	34%	228,480	1	30%	12	45,696	Active
10-1465-10 Community Care	120,000	34%	40,800	1	30%	11	7,480	Active
10-1465-11 CAHPS Your Recent Visit 4.0: Telehealth LF	180,000	34%	61,200	1	30%	13	13,260	New survey: Active
			658,920				137,415

Inpatient eligibility and sample sizes follow the HCAHPS (Hospital-Consumer Assessment of Healthcare Providers and Systems) sampling and data collection protocol put forth in the Quality Assurance Guidelines (QAG, currently version 16.0). In so doing, VA can publicly report scores on the CMS Hospital Compare website. While the annual number selected and the anticipated number responding is reflected in the table above, sample sizes of about 15,000 are drawn monthly. It should be noted that since 2017, VA has had HCAHPS data and other clinical metrics published on Care Compare (formerly Hospital Compare). In this way it is possible for Veterans to compare aspects of hospital care to include Patient survey rating, Timely & effective care, Complications & deaths, Unplanned hospital visits, Psychiatric unit services, and Payment & value of care.

Eligibility and sample size for the CAHPS-based surveys follow the administration guidelines specified by Agency for Healthcare Research and Quality (AHRQ), which is the industry best practice. Patients selected for the current month’s Inpatient sample, and responders to any SHEP survey in the last 12 months are excluded from SHEP samples for 3 months. While the annual number selected and the anticipated number responding is reflected in the table above, samples are drawn monthly. Sample sizes are calculated to provide accurate estimates at the facility level given current frame size and previous response rates and are adjusted quarterly.

Veteran patients having an eligible encounter with a VA primary care provider in the current month are eligible for the Patient Centered Medical Home (PCMH) survey (CAHPS Clinician & Group 3.0 with the PCMH item set). Patients selected for the current month’s HCAHPS or PCMH samples, and responders to any SHEP survey in the last 12 months are also excluded. Samples are drawn monthly for each site of ambulatory care in the VHA, with a total monthly sample size of about 60,000 Veteran patients.

Veteran patients having an eligible encounter with a VA specialty care provider in the current month are eligible for the Specialty Care (SC) survey (CAHPS Clinician & Group 3.0). Patients selected for the current month’s SC, HCAHPS or PCMH samples, and responders to any SHEP survey in the last 12 months are also excluded. Samples are drawn monthly for each site of ambulatory care in the VHA, with a total monthly sample size of about 56,000 Veteran patients. Specialty clinics include the following: AUDIOLOGY, CARDIOLOGY, DERMATOLOGY, GASTROENTEROLOGY, OPTHALMOLOGY/OPTOMETRY, ORTHOPEDICS, PODIATRY, UROLOGY, MENTAL HEALTH.

Veteran patients having an eligible encounter with a VA community care provider in the current month are eligible for the Community Care (CC) survey. This survey a CAHPS-like survey that assesses the patient experience with care received in the community as specified by the CHOICE Act and later the MISSION Act. Aspects of care assessed are like those in PCMH and SC and include Access, Communication, and Care Coordination. Patients selected for the current month’s CC, SC, HCAHPS or PCMH samples, and responders to any SHEP survey in the last 12 months are also excluded. Samples are drawn monthly for each site of ambulatory care in the VHA, with a total monthly sample size of about 10,000 Veteran patients.

Since AHRQ stopped publishing comparative data in 2019, there is no longer a publicly available source of comparative data. Using results from the PCMH. SC and CC surveys, VA can compare the Veteran experiences receiving care in VA to those receiving care in the community. This comparison is done in the primary care setting and for many of the specialty clinics

Veteran patients having an eligible encounter with a VA primary care provider in the current month are eligible for the Your Recent Visit (YRV) survey. This CG CAHPS 4.0 Visit survey recently was developed to assess patient experiences with primary care received via telephone, video, and face-to-face modalities. The CAHPS Consortium developed this survey to address patient experiences with virtual healthcare in the COVID-19 era. Aspects of care assessed are like those in PCMH and SC and include Access, Communication, and Care Coordination. In this way, we can compare the experiences across all three modalities. Patients selected for the current month’s YRV, CC, SC, HCAHPS or PCMH samples, and responders to any SHEP survey in the last 12 months are also excluded. Samples are drawn monthly for each site of ambulatory care in the VHA, with a total monthly sample size of about 15,000 Veteran patients.

Once Veteran patients are selected, materials are mailed and/or emailed soliciting their response to a variety of questions concerning their care at VA facilities. Veterans are asked to return the surveys to the Office of Analytics and Performance Improvement’s contracted data collection vendor. One exception is the Inpatient survey. At this time, the CMS data collection protocol does not allow for data to be collected via electronic means (emailed link to an online survey), so only mailed surveys are being used at this time. CMS is currently piloting an online mode of survey administration for the HCAHPS.

All survey questionnaires are translated into Puerto Rican Spanish for those Veterans seeking services at or through the VA Medical Center San Juan, Puerto Rico and its affiliated facilities. Questionnaires are provided in both English and Spanish, allowing the Veteran to choose the version that they are most comfortable with for their response.

3. Describe methods to maximize response rate and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.

Outpatient questionnaires are mailed to veterans via first class mail; a postage-paid return envelope is included. A modified Dillman Total Design Methodology (TDM) is used. The TDM process incorporates several strategies designed to minimize the burdens of survey participation and establish trust. Procedurally the TDM involves five carefully spaced mailings:

Week 1 Pre-notification letter mails and survey link sent via email

Week 2 First survey with cover letter mails

Week 3 Thank you / reminder postcard mails

Week 5 Close of data collection

The Inpatient sampling and administration protocols will be compliant with CAHPS® HOSPITAL SURVEY (HCAHPS) QUALITY ASSURANCE GUIDELINES V16.0 or later. The mailing sequence shall be a multiple step process consisting of a first survey and cover letter, followed in two weeks by a second survey and cover letter to non-responders.

The timing sequence of the mailings in each study is as follows:

Week 1 First survey with cover letter mails

Week 4 Second survey with cover letter

Week 7 Close of data collection

VHA has found the first mailing adequate to achieve decent (25% to 45% range) response rates among Veterans. Inpatient questionnaires follow a strict HCAHPS mailing protocol where the patient receives a survey no later than 42 days after discharge, then receives a second survey if the first is not returned within 21 days after the first mailing. This mailing method has shown to provide adequate response rates for the 4,000+ hospitals participating in the HCAHPS survey.

Socio-demographic information for the survey sample is obtained from central files in the VHA databases. Individual identifiers are subsequently stripped from the database to maintain respondent confidentiality. Analyses of patient experience data by race, ethnicity, and gender is currently underway to address congressional legislation.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions of 10 or more individuals.

The National Committee for Quality Assurance (NCQA) field tested a new version of the Consumer Assessment of Healthcare Providers and Services (CAHPS) Clinician & Group Survey specifically designed to evaluate Patient-Centered Medical Homes (C&G-PCMH). The constructs measured tested quite favorably and proved to be both valid and reliable.

A complete psychometric workup of the new C&G-PCMH survey has been conducted confirming the validity and reliability in the VHA population. The Office Analytics and Performance Improvement (API) has worked closely with the CAHPS Consortium to develop, pilot, and test the YRV survey before being administered to the general public and to Veterans in VA.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

James H. Schaefer Jr. MPH

Director of Surveys

Quality & Patient Safety | Analytics & Performance Integration

Mobile: (910) 536-0128

James Flaherty

Administrative Officer, Survey Section

Performance Analysis Center for Excellence

919-474-3917

Page 1