CMS_10764_SupportingStatement_30day

Supporting Statement for
Evaluation of Risk Adjustment Data Validation (RADV) Appeals and Health Insurance
Exchange Outreach Training Sessions

Background
The Center for Program Integrity (CPI) is requesting OMB Clearance for periodic surveys to be
conducted in support of an evaluation of Risk Adjustment Data Validation (RADV) Appeals and
Health Insurance Exchange Outreach Training Sessions.
The Social Security Act (SSA) §1853(a)(3) requires CMS to adjust payments to Medicare
Advantage Organizations (MAOs) based on risk factors derived from enrollee diagnoses, as
specified by the International Classification of Disease, 9th and 10th Revision Clinical
Modification guidelines (ICD-9-CM and ICD-10-CM). The risk adjustment methodology
prospectively adjusts payments for enrollees and ultimately determines the risk-adjusted
reimbursement from CMS to MAOs. CMS must report Medicare Advantage (MA) payment
error rates and therefore conducts annual RADV audits to ensure risk-adjusted payment integrity
and accuracy. The RADV audit is conducted pursuant to regulations under 42 CFR §422.310 –
Risk Adjustment data, section 422.310(e) and requires MAOs to submit medical records to CMS
for the successful completion of RADV audits.
Further, the Balanced Budget Act (BBA) of 1997 mandated that payments to MAOs be based on
the health status of their enrollees. By implementing risk adjustment, CMS created a level
playing field and MAOs are paid more accurately based on the health status of their enrollees.
Full risk adjusted payment to MAOs was phased in over a period of years; since 2007 payment
has been made on a 100 percent risk adjusted basis. Since risk adjusted payment is based on
enrollee health status, MAOs are provided with an incentive to enroll and treat less healthy
Medicare beneficiaries.
To receive risk adjusted payments, MAOs submit diagnostic data to CMS. CMS only requires a
one-time submission of all relevant diagnoses for each enrollee during a data collection period
for payment to occur. MAOs are required to submit risk adjustment data on a quarterly basis.
CMS conducts medical record reviews to validate the accuracy of risk adjustment data submitted
to CMS by Medicare Advantage (MA) organizations for Medicare Part C payments. The purpose
of RADV analysis is to measure the extent to which inaccurate diagnostic codes impact the CMS
Hierarchical Condition Category (HCC) assignments and the associated payment for MA
beneficiaries.
1

CMS is strongly committed to providing appropriate education and technical outreach to MAOs
and third-party administrators (TPAs). In addition, CMS recognizes that the success of
accurately identifying risk-adjustment payments and payment errors is dependent on the data
submitted by MAOs. CMS acquired the services of a contractor, ARDX, to support its training
and technical outreach efforts for contract-level (CON) RADV audits.
In addition, the Patient Protection and Affordable Care Act (PPACA) requires that there be a
health insurance exchange (also called ‘Exchanges’) to facilitate the purchase of health insurance
for individuals and small businesses. The Exchanges may be created and administered by the
state-based exchange (SBE) or by the U.S. Department of Health and Human Services (HHS) if
the state does not establish its own Exchange FFE. HHS administers FFEs in two ways – it may
fully run and administer the Exchange, or it may partner with a state and allow the state to
administer certain functions of the Exchange (State Partnership Exchange), while the federal
government remains the party with primary responsibility for all Exchange functions.
In collaboration with CMS, ARDX creates an outreach strategy focused on tailored messaging
and education to raise awareness among Exchanges stakeholders about contributors to noncompliance and risk behaviors and indicators. ARDX focuses on targeted audiences and
concentrates their outreach on these audiences’ roles pertaining to non-compliance activities.
As directed by CMS, ARDX will organize and conduct six (6) web-based training sessions on
the Risk Adjustment Data Validation (RADV) audit process and (2) two Exchange content webbased trainings annually. ARDX will also conduct periodic CBT, User Group and Onsite
trainings. ARDX will utilize surveys as part of a comprehensive evaluation process and will
solicit voluntary feedback from MAOs regarding training sessions and technical assistance
provided under this contract.
For the MA web-based training sessions, attendees will include representatives from Medicare
Advantage (MA) organizations that have been selected for participation in a RADV audit, as
well as representatives across all MA plans, for an industry wide training. For the Exchange
web-based training sessions, attendees will include SBE, HHS, SPEs, Navigators, Assistors,
Department of Insurance (DOIs) and Medicaid Agencies, and Exchange consumers.
CMS is requesting OMB Clearance for approval of the evaluation surveys performed under the
RADV Appeals and Outreach and the Center for Program Integrity (CPI) Exchange programs.
The survey results will help to determine participants’ level of satisfaction with trainings and
content delivered, and the level of satisfaction with training technology delivery, functionality,
and audibility.
2

Part A: Justification
A.1 Circumstances that Make the Data Collection Necessary (Legal Basis)

“The Payments Elimination and Recovery Improvement Act of 2012” requires agencies to
monitor programs it administers, identify programs that demonstrate substantial risk associated
with improper payments, report the estimations of improper payments and implement and
execute significant strategies to reduce overpayments. Medicare Part C payments in fiscal year
2019, accounted for an estimated $16.73 billion in improper payments which represents an
overall 7.87% of program funding (CMS, 2019). To comply with regulations set forth in IPERIA
of 2012, CMS has mandated program transparency on payments made to federally funded
insurance programs. Medicare Part C and Exchanges for the Health Insurance Marketplace
(HIM) are governed by regulations and administrative laws to assist in providing financial
transparency through efforts facilitated by CPI. CMS mandates that contracted entities providing
Medicare Part C and Exchanges benefits to beneficiaries comply with conducting training and
education and maintaining effective lines of communication. To maintain oversight, CPI
supports contracted entities in their efforts to ensure compliance by providing ongoing training
and education to Medicare and Exchange contractors through ongoing methods of
communication.
Industry training and webinars are methods of communication that CMS utilizes to provide
transparent notification in respect to qualitative and quantitative performance program areas.
This contributes to opportunities to provide proactive industry instructions to entities to assist
with avoiding improper payments and recovery of overpayments. RADV audits are combined of
qualitative and quantitative performance measurements that determine improper payments. Data
collection in RADV audits and adherence to request for overpayments are pursuant to laws
enforced under SSA SEC. §1893. [42 U.S.C. 1395ddd]. These laws combined with the
Government Performance and Results Act (GPRA) Modernization Act of 2010, justify CMS’
interactions in support of entities subjected to RADV audits parallel with data collection that
determines the effectiveness of training and education initiatives conducted to maintain program
integrity.

A.2 How the Information Will be Used, by Whom, and For What Purpose

CMS will use the evaluation data to assess the effectiveness of the RADV and Exchange
program training and webinar sessions. The RADV and Exchange programs’ webinars and
training sessions provide educational outreach to MAOs, Cost-Plans, Programs for All Inclusive
Care (PACE), Special Needs Plans (SNP), Third Party Submitters and Employer Group Waiver
Plans. The surveys will assist CMS in determining the most effective methods for:
a) Providing ongoing industry educational opportunities.
3

b) Providing the most cost-effective delivery of training sessions and webinars.
c) Providing health plans with information needed to be more effective and provide the best
patient outcomes based on prompt response to MAOs’ requests for additional training on
regulatory requirements regarding improper payments.
d) Assessing overall industry stakeholder satisfaction with training opportunities provided
by CMS.
Effective educational outreach facilitates compliance with regulations and administrative
practices governing Medicare Risk Adjustment and the Health Insurance Exchange.

A.3 Uses of Information Technology

The information for remote training events (e.g., webinars) will be collected electronically
through online surveys. In addition, onsite training participants will be given the option to
complete evaluation forms electronically or in hardcopy format.

A.4 Efforts to Avoid Duplication

The information collected in the evaluation instruments does not duplicate information collected
from other sources.

A.5 Efforts to Minimize Burden on Small Businesses

The data collection does not have a significant impact on small businesses (e.g., small health
plans). To minimize burden associated with completion of evaluations, the survey items have
been kept to the minimum required for the intended use of the data.

A.6 Consequences of Less Frequent Data Collection

The data collection is designed to assist CMS in the most effective manner to maximize content
knowledge, maintain oversight, and evaluate the overall effectiveness of the RADV and
Exchange training and webinar sessions. Less frequent data collection could impact the oversight
required to ensure proper educational opportunities related to RADV and Exchange processes to
maintain payment integrity.

A.7 Special Circumstances Requiring Collection of Information in a Manner
Inconsistent with Section 1320.5(d)(2)
There are no special circumstances associated with this data collection.

A.8 Public Comments/Consultation Outside of the Agency
A 60-day notice was published in the Federal Register on November 19, 2020 (85 FR 73720). No
comments were received during the 60-day comment solicitation period. A 30-day notice
published in the Federal Register on May 20, 2021 (86 FR 27435).
No outside consultation was sought in preparation for this data collection.
4

A.9 Payments to Respondents

Respondents are not compensated for their voluntary participation in RADV nor Exchange
surveys.

A.10 Assurances of Confidentiality Provided to Respondents

Personal identifiable information is not requested from participants in the survey instruments.
Further, respondents are assured (in the survey introduction) that their responses will remain
confidential and will be reported in aggregate form only. They are also requested not to include
any personally identifiable information (PII) in their responses.

A.11 Justification for Sensitive Nature Questions
The surveys for this evaluation do not contain sensitive questions.

A.12 Estimates of Respondent Burden
A maximum of six (6) unique Exchange training events and 10 unique RADV training
events are anticipated on an annual basis. The data collection activity involves a
survey of the population of approximately 800 Exchange training participants and
6100 RADV training participants. The table below shows the hours and cost per
response for the survey based on the estimated response rate and the estimated
completion time. Hourly cost burden is based on median hourly wages of General and
Operations Managers (11-1021), classified by Standard Occupational Classification
(SOC) codes. Source: U.S. Department of Labor, Bureau of Labor Statistics,
Occupational Employment Statistics, Occupational Employment and Wages, May
2019 (http://www.bls.gov/oes/current/oes_stru.htm). We are adjusting our employee
(respondent) hourly wage estimates by a factor of 100 percent. This is necessarily a
rough adjustment, both because fringe benefits and overhead costs vary significantly
from employer to employer, and because methods of estimating these costs vary
widely from study to study. Therefore, we believe that doubling the hourly wage to
estimate total cost is a reasonably accurate estimation method.

Estimated
Response
Rate

Final Number
of Respondents

Number of
Responses
per
Respondent

Total Annual
Responses

Time per
Response
(in minutes)

Hours
per
Response

Hourly Cost
Burden (+
100% fringe
benefits and
overhead)

Annual
Hour
Burden

Cost per
Response

70%

4,270

1

4,270

15

0.25

$96.90

1,068

$24.22

A.13 Estimate of Cost Burden to Respondents

There are no capital/start-up or ongoing operational/maintenance costs associated with this
information collection.
5

Annual
Cost
Burden

$103,489.20

A.14 Estimated Cost to the Federal Government

The estimated annual cost to the Federal government, including but not limited to the data
collection activities described in this submission is $350,000. All tasks related to the data
collections will be performed by a contractor. Included in this estimate are contractor costs
associated with background research, requirements gathering, evaluation design, instrument
design and pretest, systems development, data collection activities, analysis and reporting.

A.15 Explanation of Program Changes or Adjustments
This submission is a new request for OMB approval.

A.16 Plans for Tabulation and Publication

There are no plans to publish the information for statistical use.
The evaluation team will analyze the survey data to develop descriptive statistics including
frequencies and crosstabulations. The survey data will be coded and analyzed to show response
patterns and correlations among the responses. Data visualization will include charts, graphs and
tables to highlight key outcomes and trends. Narrative summary reports will be utilized to
support outcomes based on survey responses.

A.17 Explanation of Not Displaying Expiration Date for OMB Approval
All data collection instruments will prominently display the OMB approval number and
expiration date in the upper right-hand corner of the first page.

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS
B.1. Respondent Universe, Sample Selection and Expected Response Rates
B.1.1 Respondent Universe

The data collection is a census survey and therefore does not employ statistical methods.
B.1.2. Sample Selection

The data collection is a census survey and therefore does not include sampling methods.
B.1.3. Expected Response Rates

A response rate of approximately 70 percent is expected for this data collection effort. In
addition to strategic data collection methods and online technology, the expected response rate is
based on the fact that prospective respondents have a vested interest in the training events and
related outreach efforts.
6

B.2. Procedures for the Collection of Information

The questionnaire includes primarily closed-ended questions (e.g., Likert scale, rating scale, rank
order, or multiple response items), with a minimal number of “other (specify)” items, and openended questions. The survey will take approximately 15 minutes to complete.
ARDX will program and test the web-based survey instrument using online survey software.
Following OMB Clearance, the survey will be published (with the OMB clearance number and
burden statement) and administered following each training event.

B.3. Methods to Maximize Response Rates

Respondents will receive the evaluation instrument immediately following each event with a
follow-up reminder the following day, which should result in an increased response rate. The
online survey will remain available for three business days following each event, allowing
flexibility while minimizing survey recall effect. Further, the survey invites respondent to contact
CMS with any questions about the research study.

B.4. Pre-testing of Procedures and Methods
The evaluation team conducted an internal pre-test of the survey instrument to ensure that all
research questions were addressed, questions were not ambiguous, and response choices were
mutually exclusive and exhaustive. The pre-test was also used to estimate respondent burden in
terms of the amount of time required to complete the survey.
In addition to the design pretest, the online instrument was tested for technical factors such as
programming accuracy and browser compatibility. A checklist was developed to verify each step
of the programming process, and the instrument was subjected to a first-and second-level review
process prior to distribution to pretest respondents. The checklist included item-by-item checks
for:
•
•
•
•
•
•
•

Accuracy of skip patterns and logic checks for all appropriate scenarios
Accuracy of programming of radio buttons for single-response items or check boxes for
multiple response items
Adequate field length for open-ended questions
Connectivity
Functionality of survey links
508 compliance
Correct export order and variable names

Only minor revisions (e.g., expansion of response choices) were required as a result of the
pretest.
7

B.5. Individuals or Contractors Responsible for Statistical Aspects of the
Design
The agency responsible for receiving and approving contract deliverables is:
Center for Program Integrity
Centers for Medicare & Medicaid Services
7500 Security Boulevard
Windsor Mill, MD 21244
Person Responsible: Kathleen Anderson, (410) 786-8946
Kathleen.Anderson@cms.hhs.gov
The organization responsible for administering the surveys of Risk Adjustment Data Validation
(RADV) Appeals and Health Insurance Exchange Outreach training participants is:

ARDX (A. Reddix & Associates)
5800 Lake Wright Drive, Suite 301
Norfolk, VA 23502
Persons Responsible: Mr. Matt Lemma, (757) 410-7704, Matt.Lemma@ardx.net
Dr. Lateefah Hughes (757) 375-7291, Lateefah.Hughes@ardx.net
Ms. Dorothea Claytor (757) 227-4034, Dot.Claytor@ardx.net
The organization responsible for data analysis is:
ARDX (A. Reddix & Associates)
5800 Lake Wright Drive, Suite 301
Norfolk, VA 23502
Person Responsible: Ms. Sadie Bennett (757) 321-4123, Sadie.Bennett@ardx.net

8