Memo 2

SUPPORTING STATEMENT FOR REQUEST FOR MODIFICATION OF -- 2900-0712

NATION-WIDE CUSTOMER SATISFACTION SURVEYS

B. Collections of Information Employing Statistical Methods

1. Provide a numerical estimate of the potential respondent universe and describe any sampling or other respondent selection method to be used. Data on the number of entities (e.g., households or persons) in the universe and the corresponding sample are to be provided in tabular format for the universe as a whole and for each stratum. Indicate expected response rates. If this has been conducted previously include actual response rates achieved.

Veteran patient population 5,000,000

Potential respondent universe: 632,000 patients/year

Expected response rate: Average of 48% (n=301,244)

Sampling methodology: Random selection based on social security numbers and specific visit type at or coordinated through a VA facility

2. Describe the procedures for the collection of information, including:

a. Statistical methodology for stratification and sample selection

b. Estimation procedure

c. Degree of accuracy needed

d. Unusual problems requiring specialized sampling procedures

e. Any use of less frequent than monthly data collection to reduce burden

Veterans are sampled monthly for inclusion in SHEP in the following way: Inpatients who have been discharged alive in the past month are eligible. Sufficient numbers are selected to ensure adequate representation at the facility division level for stable estimates of the percentage of veteran patients responding as highly satisfied to the overall rating of care question, within 5 percentage points. For outpatients, a similar methodology is used in sample selection at each site of care. Once veteran patients are selected, letters are sent soliciting their response to a variety of questions concerning their care at VA facilities. Veterans are asked to return the surveys to the Office of Quality and Performance contracted data collection vendor. The sample size has been determined to be of a sufficient size to give statistically reliable data for each VISN and VA facility. The monthly data collection provides for sufficiently timely identification of problem areas and undertaking of steps toward remediation. This allows VA to provide quality medical services to veterans.

All survey questionnaires are translated into Puerto Rican Spanish for those veterans seeking services at or through the VA Medical Center San Juan, Puerto Rico and its affiliated facilities. Questionnaires are provided in both English and Spanish, allowing the veteran to choose the version that they are most comfortable responding with.

3. Describe methods to maximize response rate and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.

Outpatient questionnaires are mailed to veterans via first class mail; a postage-paid return envelope is included. A modified Dillman Total Design Methodology (TDM) is used. The TDM process incorporates several strategies designed to minimize the burdens of survey participation and establish trust. Procedurally the TDM involves five carefully spaced mailings:

Week 1 Pre-notification letter mails

Week 2 First survey with cover letter mails

Week 3 Thank you / reminder postcard mails

Week 5 Close of data collection

The Inpatient sampling and administration protocols will be compliant with CAHPS® HOSPITAL SURVEY (HCAHPS) QUALITY ASSURANCE GUIDELINES V5.0. The mailing sequence shall be a multiple step process consisting of a first survey and cover letter, followed in two weeks by a second survey and cover letter to non responders.

The timing sequence of the mailings in each study is as follows:

Week 1 First survey with cover letter mails

Week 4 Second survey with cover letter

Week 7 Close of data collection

OQP has found the first mailing adequate to achieve very good (45% to 60% range) response rates among veterans. Inpatient questionnaires follow a strict HCAHPS mailing protocol where the patient receives a survey no later than 42 days after discharge, then receives a second survey if the first is not returned within 21 days after the first mailing. This mailing method has shown to provide adequate response rates for the 4,000+ hospitals participating in the HCAHPS survey.

The HCAHPS sampling protocol specifies a target of at least 300 complete cases per year for reporting facility. This target is examined on a quarterly basis to ensure this goal is being met. However, for many facilities, VHA often far exceeds this goal.

Sociodemographic information for the survey sample is obtained from central files in the VA Austin Automation Center. Individual identifiers are subsequently stripped from the database to maintain respondent confidentiality.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions of 10 or more individuals.

A separate request for a pilot study of HCAHPS in the VHA population has been administered. Results from this pilot are currently being analyzed. Results of testing in other federal healthcare systems are available, and have found excellent performance for the HCAHPS instrument in measuring patient satisfaction among inpatients.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Jim Schaefer, MPH

Director of Surveys

Office of Quality and Performance Durham

919-474-3914

James Flaherty

Administrative Officer, Survey Section

Performance Analysis Center for Excellence

Office of Quality and Performance

919-474-3917

Page 1