Part B 2019 CAHPS

B1.  Development of Concepts, Methods, and Design

As part of its FEHB Plan Performance Assessment (PPA), OPM is requiring all FEHB Carriers to utilize the Consumer Assessment of Healthcare Providers and Systems (CAHPS) commercial survey using a certified CAHPS vendor (more on certification below). The PPA ties FEHB Carriers profit to performance on key metrics.

The Agency for Healthcare Research and Quality (AHRQ) CAHPS surveys provide a uniform and industry accepted survey vehicle to assess member satisfaction. From the AHRQ website (https://www.ahrq.gov/cahps/surveys-guidance/hp/index.html):

The CAHPS Health Plan Survey is a tool for collecting standardized information on enrollees' experiences with health plans and their services. It was designed to support consumers in assessing the performance of health plans and choosing the plans that best meet their needs. Health plans can also use the survey results to identify their strengths and weaknesses and target areas for improvement. Survey results can be used to:

• Support consumers in assessing the performance of health plans and choosing the plans that best meet their needs.

• Identify the strengths and weaknesses of health plans and target areas for improvement.

OPM is using the Commercial survey. The survey utilizes a sampling methodology as required by the survey licensee, the National Committee for Quality Assurance (NCQA), outlined in extensive detail in the HEDIS Volume 3: Specifications for Survey Measures includes surveys and protocols for the CAHPS 5.0H Survey. Information can also be found on the AHRQ website here https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance/hp/fielding-the-survey-hp50-2013.pdf. Recommended data collection modes include mail and telephone, and protocols have been developed for each mode.

Two modes of survey administration are allowed during the national implementation to give facilities options in how they would like to administer the survey, based on their goals and resources. These two modes are described below:

• Mail-only Mode

  • Mailing of the questionnaire and cover letter to all sampled members.

  • Reminder postcard mailing

  • Second mailing of the questionnaire and cover letter to sampled members who do not respond to the first questionnaire mailing within 5 weeks

  • Second reminder postcard mailing

  • Third mailing of the questionnaire and cover letter to sampled members who do not respond to the first or second questionnaire mailing within 3 weeks of the second questionnaire mailing.

• Mixed Mode (Mail with Telephone Follow-up)

  • Mailing of the questionnaire and cover letter to all sampled members.

  • Reminder postcard mailing

  • Second mailing of the questionnaire and cover letter to sampled members who do not respond to the first questionnaire mailing within 5 weeks

  • Second reminder postcard mailing

  • Telephone follow-up with all sampled patients who do not respond to one of the questionnaire mailings. A maximum of six telephone contact attempts per sampled patient will be made to complete the survey.

AHRQ versions of the survey are administered by the Centers for Medicare and Medicaid Services (CMS) as part of their public reporting and reimbursement programs. The CAHPS surveys utilized by CMS have received PRA approval. CAHPS is also utilized by many private health insurance purchasers, and is part of the NCQA accreditation process.

B2.  Collection of Data

To collect the data, health plans contract with NCQA certified Survey Vendors to administer the CAHPS survey on their behalf. To become an NCQA-Certified survey vendor, “an organization must demonstrate that it has the capabilities, experience and expert personnel to collect and report survey results accurately. NCQA issues an annual request for proposal (RFP) for new organizations to apply for certification.”

The surveys collected and processed by NCQA-Certified survey vendors, and submitted thru the NCQA Portal, access to which is restricted by NCQA. NCQA validates the submission, determines benchmarks for each measure, and provides results to health plans and purchasers. Respondents are informed about the data collection in the instructions for the survey. OPM contracts with a vendor to further analyze and validate the aggregated results, and produce a dataset for OPMs use.

FEHB Carriers are required to report their CAHPS results, and many report the results for their entire book of business of which FEHB is a part. Consumer’s response to the survey is entirely voluntary. The burden on consumers is not considered onerous, the estimated time to complete the survey is fifteen minutes.

The results are aggregated, and used as part of the formula to determination of FEHB Plans profit factor, as well aggregated CAHPS survey results are provided on two OPM webpages to help inform selection of FEHB health plans. Specific Carrier scores are reported within a range.

B3.  Processing and Editing of Data

Standard data cleaning and analysis procedures are performed by the NCQA Certified Vendor. The vendor is responsible for validating the results, including making sure that the sample protocol is followed, and all other requirements are followed. Only then is the data submitted to NCQA. OPM vendor does additional validation and analytics under a separate contract with OPM. Currently, the OPM vendor is a NCQA Certified Vendor.

B4.   Production of Estimates and Projections

The CAHPS sampling protocols are set by NCQA, and applied by the NCQA Certified Vendor for each health plan. This ensures that they are statistically valid, and comparisons of results can be made across all Carriers. NCQA Certified Vendors identify any biased results, and those results are excluded from use. For a specific measure identified as biased, the results are not used in the analysis or for the calculation of profit factor. They are also not made available to consumers.

B5.  Data Analysis

Basic survey analysis techniques (e.g., comparing to commercial benchmarks produced by NCQA for CAHPS, calculating frequencies, patterns, etc.) will be used to interpret and explain the data provided by respondents. The results will be used in the FEHB Plan Performance Assessment, and ranges for each FEHB Plan posted on the OPM website (https://www.opm.gov/healthcare-insurance/healthcare/plan-information/compare-plans/quality) and included in the results for the FEHB Plan Comparison Tool (https://www.opm.gov/healthcare-insurance/healthcare/plan-information/compare-plans/).

B6.  Review and Evaluation Procedures

As noted above, the survey tool was developed by AHRQ and is the industry standard, used for multiple government healthcare programs including Medicare and Medicaid, by NCQA for accreditation, by states for their healthcare programs, and throughout the industry. PRA approval has been granted on numerous occasions for CAHPS in CMS programs.

B7.   Data Dissemination

Aggregated reporting on the CAHPS survey results are provided on two OPM webpages to help inform selection of FEHB health plans. Specific Carrier scores are reported within a range.

No PII is reported on the CAHPS results. The results are aggregated and no individual responses are reported.

B8.  Contact Person(s)

Joseph Kahn

Federal Employee Insurance Operations

U.S. Office of Personnel Management

(202) 606-0971

Joseph.kahn@opm.gov