Supporting Statement Part A -- Collection of Information for CAHPS Health Plan Survey Comparative DB 03172012

AHRQ’s mission. The mission of the Agency for Healthcare Research and Quality (AHRQ) set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999 (see http://www.ahrq.gov/hrqa99.pdf), is to enhance the quality, appropriateness, and effectiveness of health services, and access to such services, through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health systems practices, including the prevention of diseases and other health conditions. AHRQ shall promote health care quality improvement by conducting and supporting:

1. research that develops and presents scientific evidence regarding all aspects of health care; and

2. the synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators; and

3. initiatives to advance private and public efforts to improve health care quality.

Also, AHRQ shall conduct and support research and evaluations, and support demonstration projects, with respect to (A) the delivery of health care in inner-city areas, and in rural areas (including frontier areas); and (B) health care for priority populations, which shall include (1) low-income groups, (2) minority groups, (3) women, (4) children, (5) the elderly, and (6) individuals with special health care needs, including individuals with disabilities and individuals who need chronic care or end-of-life health care.

Request for information collection approval. The Agency for Healthcare Research and Quality (AHRQ) requests that the Office of Management and Budget (OMB) reapprove, under the Paperwork Reduction Act of 1995, AHRQ’s collection of information for the AHRQ Consumer Assessment of Healthcare Providers and Systems (CAHPS) Database for Health Plans: OMB Control number 0935-0165, expiration May 31, 2017. The CAHPS Health Plan Database consists of data from the AHRQ CAHPS Health Plan Survey. Health plans in the U.S. are asked to submit data voluntarily from the survey to AHRQ, through its contractor, Westat. The CAHPS Database was developed by AHRQ in 1998 in response to requests from health plans, purchasers, the Centers for Medicare and Medicaid Services (CMS) to provide comparative data to support public reporting of health plan ratings, health plan accreditation and quality improvement.

This research has the following goals:

1) To maintain the CAHPS Health Plan database using data from AHRQ’s standardized CAHPS Health Plan survey to provide comparative results to health care purchasers, consumers, regulators and policy makers across the country.

2) To offer several products and services, including comparative benchmark results presented through an Online Reporting System, summary chartbooks, custom analyses, and data for research purposes.

3) To provide data for AHRQ’s annual National Healthcare Quality and Disparities Report.

To achieve the goals of this project the following data collections will be implemented:

1) Health Plan Registration Form – The point-of-contact (POC), often the sponsor from Medicaid agencies and health plans, completes a number of data submission steps and forms, beginning with the completion of the online registration form (see Attachment D). The purpose of this form is to collect basic contact information about the organization and initiate the registration process.2) Data Use Agreement – The purpose of the data use agreement, completed by the participating sponsor organization, is to state how data submitted health plans will be used and provides confidentiality assurances (see Attachment C).

3) Health Plan Information Form – The purpose of this form (see Attachment E), completed by the participating organization, is to collect background characteristics of the health plan.

4) Data Files Submission - POCs upload their data file using the Health Plan data file specifications to ensure that users submit standardized and consistent data in the way variables are named, coded, and formatted.

Background on the CAHPS Health Plan Survey. The CAHPS Health Plan Survey is a tool for collecting standardized information on enrollees’ experiences with health plans and their services. The development of the CAHPS Health Plan Survey began in 1995, when AHRQ awarded the first set of CAHPS grants to Harvard, RTI, and RAND. In 1997 the CAHPS 1.0 survey was released by the CAHPS Consortium. The CAHPS Consortium refers to the research organizations involved in the development, dissemination, and support of CAHPS products. The current Consortium includes AHRQ, the Centers for Medicare & Medicaid Services (CMS), RAND, Yale School of Public Health, and Westat.

Since that time, the Consortium has refined and updated the survey instrument to reflect field feedback from industry experts; reports from health plan participants, data collection vendors, and other users; evidence from cognitive testing and focus groups; and extensive psychometric data analysis. Version 5.0 of the Health Plan Survey was released in 2012. The development of this update to the Health Plan Survey has been part of the "Ambulatory CAHPS (A-CAHPS) Initiative," which arose as a result of extensive research conducted with users. AHRQ released the CAHPS Health Plan Survey 5.0, along with guidance on how to customize and administer it. The National Quality Forum endorsed the 5.0 version of the Health Plan Survey in 2015.

Rationale for the information collection. The CAHPS Health Plan Database uses data from AHRQ’s standardized CAHPS Health plan survey to provide comparative results to health care purchasers, consumers, regulators and policy makers across the country. The Database also provides data for AHRQ’s annual National Healthcare Quality and Disparities Reports. Voluntary participants include public and private employers, State Medicaid agencies, State Children's Health Insurance Programs (SCHIP), the Centers for Medicare & Medicaid Services (CMS), and individual health plans.

Each year State Medicaid agencies, and individual health plans decide whether to participate in the database and prepare their materials and dataset for submission to the CAHPS Health Plan Database. Participating organizations are typically State Medicaid agencies with multiple health plans. However, individual health plans are also encouraged to submit their data to the CAHPS Database. The number of data submissions per registrant varies from participant to participant and year to year because some participants submit data for multiple health plans, while others may only submit survey data for one plan.

Each organization that decides to participate in the database must have their point-of-contact (POC) complete a registration form providing their contact information for access to the online data submission system, sign and submit a DUA, and provide health plan characteristics such as health plan name, product type, type of population surveyed, health plan state, and plan name to appear in the reporting of their results.

Each vendor that submits files on behalf of a Medicaid agency or individual health plan must also complete the registration form in order to obtain access to the online submission system. The vendor, on behalf of their client, may also complete additional information about survey administration, submit a copy of the questionnaire used, and submit one data file per health plan. Medicare health plan data are received from CMS.

The collection of information for the CAHPS Database for Health Plans is being conducted pursuant to AHRQ’s statutory authority to conduct and support research on health care and systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness and value of health care services. See 42 U.S.C. 299a(a)(1).

2. Purpose and Use of Information

Survey data from the CAHPS Health Plan Database is used to produce four types of products: 1) an annual chartbook available to the public on the CAHPS Database web site (https://www.cahpsdatabase.ahrq.gov/CAHPSIDB/Public/Chartbook.aspx); 2) individual participant comparative reports that are confidential and customized for each participating organization (e.g., health plan, Medicaid agency) that submits their data; 3) a research database available to researchers wanting to conduct additional analyses; and 4) data tables provided to AHRQ for inclusion in the National Healthcare Quality and Disparities Reports.

Information for the CAHPS Health Plan Database has been collected by AHRQ through its contractor Westat, on an annual basis since 1998. Health plans and State Medicaid agencies are asked to submit their data voluntarily to the comparative database in June/July of each year.

Once data are submitted the data are cleaned with standardized programs, then aggregated and used to produce comparative results for Medicaid (adult and child), and Medicare (adult) populations for the two most recent years. In addition, individual participant reports are produced that display the participating organizations’ own results compared to appropriate comparisons derived from the National, regional and product-type distributions on a password-protected section of the online reporting system.

The CAHPS Health Plan Database receives the data from two sources: 1) Medicare data are provided by the Centers for Medicare and Medicaid Services (CMS) through an agency Data Use Agreement. The Medicare data are collected by CMS and their contractor from beneficiaries who were enrolled in a managed care health plan and 2) Medicaid data are collected by the CAHPS Health Plan Database. Medicaid agencies and their vendors directly submit their Medicaid health plan survey data to the CAPHS Health Plan Database through an online data submission system. Data submitted by Medicaid plans, are compiled along with the Medicare data received from CMS to comprise the CAHPS Health Plan Survey comparative database.

AHRQ has produced CAHPS Health Plan Chartbook report since 2000. The most recent Health Plan results reported in the online reporting system includes Medicaid and Medicare data from two years, 2014 and 2015. For 2014, 707 health plans are included covering 335,897 respondents and for 2015, 719 health plans are included covering 320,979 respondents. The Health Plan comparative results can be viewed in the online reporting system on the CAHPS Database web site at: https://www.cahpsdatabase.ahrq.gov/CAHPSIDB/Public/about.aspx.

Submitted data are also used to give individual participants their own customized feedback report presenting their own plan results compared to the latest comparative data results. Trend data are also available to those plans that provide data across consecutive years.

Chartbook results and individual participant reports are used for a variety of purposes:

Identify areas for quality improvement and contracting goals
Brief senior leadership
Support public reporting
Compare care across payers (Medicaid, Medicare)
Combine with other quality measures to examine outcomes

The research database has provided health plan data to 72 researchers thus far. Results of this research have been used to examine:

Disparities in CAHPS scores by racial and ethnic characteristics of respondents
Comparison of adult and child CAHPS survey results
Analysis of case-mix factors affecting CAHPS scores
Differences in CAHPS scores at the state level
Comparison of CAHPS scores between civilian and military beneficiaries

3. Use of Improved Information Technology

All information collection for the CAHPS Health Plan Database is done electronically, except the Data Use Agreement (DUA) that health plans sign in hard copy and fax, mail or return by email. Registration is handled online and data submission information and data upload is handled through a secure web site. Delivery of confidential health plan survey results is also done electronically requiring submitters to enter a username and password to access their reports from a secure web site.

4. Efforts to Identify Duplication

While there are many survey vendors that collect the CAHPS Health Plan Survey and may maintain databases of their own clients’ data, AHRQ is the only entity that serves as a comprehensive repository of CAHPS Health Plan survey data. AHRQ collaborates with CMS and NCQA to combine their CAHPS Health Plan survey data with the CAHPS Health Plan Database in an effort to reduce the burden of submission on health plans. Vendors submit data files using the NCQA layout format in order to reduce the burden on vendors.

5. Involvement of Small Entities

The collection of information associated with data submission is very basic and does not unduly burden small business or health plans. The information being requested has been held to the absolute minimum required for the intended uses and is likely already being compiled by survey vendors for their health plan clients.

6. Consequences if Information Collected Less Frequently

Regulations issued by CMS and NCQA require health plans to follow a regular annual data collection and reporting cycle. As a result, the CAHPS Health Plan Database follows the same cycle for data submission. Because data are used to monitor patient experience and to evaluate specific quality initiatives, timely data are imperative to monitoring and quality improvement.

7. Special Circumstances

This request is consistent with the general information collection guidelines of 5 CFR 1320.5(d) (2). No special circumstances apply.

8. Federal Register Notice and Outside Consultations

8.a. Federal Register Notice

As required by 5 CFR 1320.8(d), notices were published in the Federal Register of Vol. 81, No. 246, December 22, 2016 on Page 93935 for 60 days and again for 30 days. (See Attachment A). Comments were received from the America’s Health Insurance Plans.

8.b. Outside Consultations

The CAHPS maintains a Technical Expert Panel (TEP) to provide expertise and guidance to the development, functioning, and expansion of the CAHPS Databases. The last TEP was convened February 13, 2014 and is comprised of 19 people who represent a wide range of stakeholders in the CAHPS surveys. They were invited by the CAHPS Consortium to provide input over the next 4 years of activities and direction of AHRQ’s CAHPS program. The current composition of the TEP is included in Attachment B.

9. Payments/Gifts to Respondents

No payment or remuneration is provided to health plans for submitting data to the CAHPS Health Plan Database.

10. Assurance of Confidentiality

Individuals and organizations will be assured of the confidentiality of their replies under Section 944(c) of the Public Health Service Act. 42 U.S.C. 299c-3(c). That law requires that information collected for research conducted or supported by AHRQ that identifies individuals or establishments be used only for the purpose for which it was supplied.

Confidentiality of the Point-of-Contact for a Health Plan. The health plans’ POC, who submits data on behalf of their plan, is asked to provide his/her name, phone number and email address during the data submission process to ensure that the plan’s individual survey feedback report is delivered back to that person for use by the health plan. In addition, the POC’s contact information is important when any clarifications or corrections of the submitted data set are required and follow up is needed. However, the name of the POC and name of the plan is kept confidential and not publicly reported. Only aggregated, de-identified results are displayed in any public reports.

Confidentiality of the Survey Data Submitted by a Health Plan. Health plans are assured of the confidentiality of their CAHPS Health Plan survey data through a DUA that they must sign that has been approved by AHRQ (see Attachment C). The DUA states that their data will be handled in a secure manner using necessary administrative, technical and physical safeguards to limit access to it and maintain its confidentiality. In addition, the DUA states the data will be used for the purposes of the database, that only aggregated results will be reported, and that the health plan will not be identified by name.

11. Questions of a Sensitive Nature

There are no questions of a sensitive nature.

12. Estimates of Annualized Burden Hours and Costs

Exhibit 1 shows the estimated burden hours for the respondent to participate in the database. The burden hours pertain only to the collection of Medicaid data from State Medicaid agencies and individual Medicaid health plans because those are the only entities that submit data through the data submission process (other data are obtained from CMS as noted earlier in Section 2). The 85 POCs in Exhibit 1 are a combination of an estimated 75 State Medicaid agencies and individual health plans, and 10 vendor organizations.

Each State Medicaid agency, health plan or vendor will register online for submission. The online Registration form (see Attachment D) will require about 5 minutes to complete. Each submitter will also complete a Health Plan information form of information about each Health Plan such as the name of the plan, the product type (e.g., HMO, PPO), the population surveyed (e.g., adult Medicaid or child Medicaid). Each year, the prior year’s plan data are preloaded in the plan table to lessen burden on the Sponsor. The Sponsor is responsible for updating the plan table to reflect the current year’s plan information. The online Health Plan Information form (see Attachment E) takes on average 30 minutes to complete per health plan with each POC completing the form for 4 plans on average. The data use agreement will be completed by the 75 participating State Medicaid agencies or individual health plans. Vendors do not sign or submit DUAs. The DUA requires about 3 minutes to sign and return by fax or mail. Each submitter will provide a copy of their questionnaire and the survey data file in the required file format. Survey data files must conform to the data file layout specifications provide by the CAHPS Database. Since the unit of analysis is at the health plan level, submitters will upload one data file per health plan. Once a data file is uploaded the file will be checked automatically to ensure it conforms to the specifications and a data file status report will be produced and made available to the submitter. Submitters will review each report and will be expected to fix any errors in their data file and resubmit if necessary. It will take about 1 hour to submit the data for each plan, and each POC will submit data for 4 plans on average (see Attachments F and G). The total burden is estimated to be 501 hours annually.

Exhibit 1. Estimated Annualized Burden Hours

Form Name	Number of respondents/ POCs	Number of responses per POC	Hours per response	Total burden hours
Registration Form	85	1	5/60	7
Health Plan Information Form	75	4	30/60	150
Data Use Agreement	75	1	3/60	4
Data Files Submission	85	4	1	340
Total	320	NA	NA	501

Exhibit 2 shows the estimated annualized cost burden based on the respondents’ time to complete one submission process. The cost burden is estimated to be $22,153 annually.

Exhibit 2. Estimated annualized cost burden

Form Name	Number of respondents	Total Burden hours	Average Hourly Wage Rate*	Total Cost burden
Registration Form	85	7	50.99^a	$357
Health Plan Information Form	75	150	50.99^a	$7,649
Data Use Agreement	75	4	89.35^b	$357
Data Files Submission	85	340	40.56^c	$13,790
Total	320	501	NA	$22,153

* National Compensation Survey: Occupational wages in the United States May 2015, “U.S. Department of Labor, Bureau of Labor Statistics.”

a) Based on the mean hourly wage for Medical and Health Services Managers (11-9111).

b) Based on the mean hourly wage for Chief Executives (11-1011).

c) Based on the mean hourly wages for Computer Programmer (15-1131).

13. Estimates of Annualized Respondent Capital and Maintenance Costs

Capital and maintenance costs include the purchase of equipment, computers or computer software or services, or storage facilities for records, as a result of complying with this data collection. There are no direct costs to respondents other than their time to participate in the study.

14. Estimates of Annualized Cost to the Government

Exhibit 3 shows the estimated annualized cost to the government for developing, maintaining and managing the Health Plan Database and analyzing the data and reporting results. The cost for one annual data submission is estimated to be $320,000. Annualized costs for collecting and processing the CAHPS Health Plan Database are based upon prior years of historical project costs. Start-up costs were present in the early years of the database only.

Exhibit 3. Estimated Annualized Cost

Cost Component	Total Cost	Annualized Cost
Database Maintenance	$150,000	$50,000
Data Submission	$370,000	$123,000
Data Analysis and Reporting	$440,000	$147,000
Total	$960,000	$320,000

Exhibit 4. Estimated Annual Cost to AHRQ for Project Oversight

AHRQ Staff Position	Percent Time	Annualized Cost
Health Science Administrator, GS 15, Step-5	5%	$7,466
Project Officer, GS 13, Step 5	5%	$5,371

Total		$12,837

https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/pdf/2017/DCB.pdf

15. Changes in Hour Burden

Participation is expected to slightly increase during the 3 year period covered by this renewal.

16. Time Schedule, Publication and Analysis Plans

Information for the CAHPS Health Plan Database is collected by AHRQ through its contractor, Westat, on an annual basis since 2000. Health plans are asked to submit their CAHPS Health Plan survey data voluntarily to the CAHPS Database in June each year. The data are then cleaned and aggregated and used to produce annual chartbooks and individual participants comparative reports each Fall. Results for 2014 and 2015 are available on the Online Reporting System at https://cahpsdatabase.ahrq.gov/CAHPSORS.aspx. Results for the participating Medicaid agencies and individual health plans are provided in a password protected portion of the web site.

17. Exemption for Display of Expiration Date

AHRQ does not seek this exemption.

List of Attachments:

Attachment A: Federal Register Notice

Attachment B: Technical Expert Panel List

Attachment C: Data Use Agreement

Attachment D: Registration Form

Attachment E: Health Plan Information Form

Attachment F: Screen Shots of Data Submission

Attachment G: Notification Emails

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	OMB Clearance Application
Author	Teresa Dodson
File Modified	0000-00-00
File Created	2021-01-22

Supporting Statement Part A -- Collection of Information for CAHPS Health Plan Survey Comparative DB 03172012