Development of HH Survey

Attachment I - Development of HH Survey 508.pdf

Home Health (HH) National Provider Survey (CMS-10688)

Development of HH Survey

OMB: 0938-1364

Document [pdf]

Download: pdf | pdf

Development of the
National Provider Survey of
Home Health Agencies
Kanaka D. Shetty
Deborah Kim
Alice Kim
Erin Taylor
Cheryl L. Damberg

CONTRACT NUMBER: HHSM-500-2013-13007I
TASK ORDER: HHSM-500-T0002

PREPARED FOR: HEALTH SERVICES ADVISORY GROUP, INC. (HSAG)
SUBMITTED AUGUST 29, 2018, TO:

NONI BODKIN, CONTRACTING OFFICER’S REPRESENTATIVE (COR – TASK ORDER)
7500 SECURITY BOULEVARD

BALTIMORE, MD 21244-1850
NONI.BODKIN@CMS.HHS.GOV

Development of National Provider Survey of Home Health Agencies

Table of Contents
Executive Summary................................................................................................................................ 3

Summary of Findings from Environmental Scan ................................................................... 3

Summary of Key Issues Identified in Interviews with CMS Staff ..................................... 4
Summary of Findings from Formative Interviews ................................................................ 5
Summary of Findings from Cognitive Interviews .................................................................. 5

Introduction .............................................................................................................................................. 6

Environmental Scan ............................................................................................................................... 7
Methods.................................................................................................................................................. 7

Findings from the Environmental Scan ..................................................................................... 9
Findings from Studies in Other Provider Settings .............................................................. 12
Summary of Findings from Environmental Scan ................................................................ 13

Interviews with CMS Staff to Review Goals of Survey and Determine Priorities for
Data Collection Needs......................................................................................................................... 14
Methods............................................................................................................................................... 14
Findings from Interviews with CMS Staff .............................................................................. 14
Summary of Key Issues Identified in Interviews with CMS Staff .................................. 18

Formative Interviews with Home Health Agencies ................................................................ 20
Methods Used to Conduct Formative Interviews ............................................................... 20
Findings from Formative Interviews with Home Health Agencies.............................. 21
Summary of Findings from Formative Interviews ............................................................. 25

Cognitive Testing of Draft Survey with Home Health Agencies ......................................... 27

Methods for Conducting Cognitive Testing ........................................................................... 27
Findings from Cognitive Testing on Overarching Issues ................................................. 28
Detailed Findings from Cognitive Testing by Survey Topic............................................ 28
Summary of Findings from Cognitive Interviews ............................................................... 31

Appendix A: Formative Interview Guide for Home Health Agencies.............................. 32
Appendix B: Cognitive Interview Guide for Home Health Agencies ............................... 41
Citations ................................................................................................................................................... 64

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

Executive Summary

The Centers for Medicare & Medicaid Services (CMS) is committed to drive improvements
in health care quality by implementing quality measures across a variety of settings in
which Medicare beneficiaries receive care. Section 1890A(a)(6) of the Social Security Act
requires the Secretary of Health and Human Services (HHS) to conduct an assessment of
the quality and efficiency impact of the use of endorsed measures every three years and to
make the assessment available to the public. Further, CMS is committed to ensuring
improved quality while reducing measurement burdens on providers, as emphasized in the
Meaningful Measures Initiative.

There is a lack of information regarding how home health agencies (HHAs) are responding
to CMS quality measures and the impact of the use of CMS measures in HHAs. Therefore, in
preparation for the 2021 National Impact Assessment of CMS Quality Measures Report (2021
Impact Assessment Report), CMS collaborated with its contractors, the Health Services
Advisory Group (HSAG) and RAND, to develop a nationally representative survey and
qualitative interview series to be fielded in 2019–2020 to assess the impact of the use of
quality measures in the home health setting. The two proposed data collection
instruments—a structured survey and a qualitative interview guide—address the research
question “What changes are home health agencies making in response to the use of
performance measures by CMS?” This overarching question was translated into five
specific research questions that form the content of the surveys and interviews:
1. What types of quality improvement (QI) changes have HHAs made to improve their
performance on CMS measures?
2. If a QI change was made, has it helped the HHA improve its performance on one or
more CMS measures?
3. What challenges or barriers do HHAs face in reporting CMS quality measures?
4. What challenges or barriers do HHAs face in improving performance on the CMS
quality measures?
5. What unintended consequences do HHAs report associated with implementation of
CMS quality measures?
To develop the data collection instruments and refine the survey design, we:
(1) Conducted an environmental scan of the published and grey literature to identify
prior studies that had examined the effects of use of quality measures in HHAs;
(2) Interviewed CMS staff responsible for quality measurement programs and
improvement initiatives for HHAs to determine CMS information-gathering needs
and policy priorities;
(3) Conducted formative interviews with HHAs to develop the content for the survey
instrument and interview guide and to test survey question language; and
(4) Cognitively tested the proposed questions to assess respondents’ understanding of
the survey and interview questions and to identify problematic terms, items, or
response options.

Summary of Findings from Environmental Scan
In our review of the literature, we identified no recent nationally representative surveys
examining the prevalence of QI interventions employed by HHAs or responses by HHAs to

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

quality measurement programs. However, case studies in HHAs and studies of usage of QI
strategies by hospitals and nursing homes suggest that HHAs are changing care delivery to
improve the quality of their care; similarly, studies from other settings suggest that HHAs
are likely facing barriers to improvement and experiencing potential unintended
consequences. The environmental scan suggests that these topics should be explored in a
nationally representative survey of HHAs, with modifications to accommodate QI
interventions uncommon outside of the HH setting.

Summary of Key Issues Identified in Interviews with CMS Staff
Our interviews with CMS staff suggested that the survey instrument and interview guide
from the Hospital and Nursing Home National Provider Surveys i that were part of the 2018
National Impact Assessment of CMS Quality Measures Report [1] would be broadly
applicable with appropriate modifications for HHAs. For example, although HHAs
implement common QI changes such as risk-management tools and provider education,
HHAs also use strategies that hospitals and nursing homes do not commonly use, such as
front-loading visits by clinicians, telehealth activities, remote monitoring of patients, and
patient self-management. We made numerous changes to the survey terminology,
including curtailing use of physicians in examples, as they are not as relevant to HHAs. We
also aligned survey questions with the terminology of the Meaningful Measures framework
to better express CMS priorities.

CMS staff noted that HHAs have faced unique difficulties obtaining and sending needed
information from other providers to focus QI efforts and may have insufficient electronic
tools for reporting data accurately to CMS. We therefore modified and retested survey
questions regarding EHRs.

The consensus among CMS staff was to consider using some combination of size, quality,
and enrollment in the Home Health Value-Based Purchasing (HHVBP) model to identify
strata for the survey. CMS staff agreed that it would be impractical to stratify by ownership
type, census region, or state. However, nearly all CMS staff agreed that HHAs without
quality ratings should be included in the survey, which suggests that size and HHVBP status
are more important stratification criteria.
Finally, the Center for Medicare and Medicaid Innovation (CMS Innovation Center) is
sponsoring recurring nationally representative surveys of HHAs with the aim of
determining whether HHAs enrolled in the HHVBP model differ in their QI activities
compared with HHAs not enrolled in HHVBP. The CMS Innovation Center surveys devote
less attention to unintended consequences, barriers to reporting, and barriers to
improvement related to quality measurement than the proposed survey. The project team
will coordinate with QMVIG and the CMS Innovation Center to avoid excessive overlap
between the two survey approaches.
Hospital National Provider Survey – OMB Control Number 0938-1290 and Nursing Home National Provider Survey –
OMB Control Number 0938-1291

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

Summary of Findings from Formative Interviews
We used findings from the formative interviews to modify the survey instrument and
interview guide to better suit the home health setting. First, respondents noted that their
participation in Home Health Compare has focused their efforts and raised consciousness
about quality issues, leading to changes in care delivery among home health providers.
However, a portion of their effort was related primarily to improving the quality of data
sent to CMS. In both the interviews and the survey instrument, we asked about each
agency’s emphasis on improvements in documentation. Second, despite making changes in
response to public reporting via Home Health Compare, no home health agencies
considered public reporting to be the most significant driver of improvement because they
perceived that their patients did not use Home Health Compare Star Ratings when selecting
an HHA. However, most HHAs viewed the Home Health Value-based Purchasing (HHVBP)
model as potentially a very significant driver of improvement. As noted above, to
understand this issue in greater detail, we will stratify sampling by HHVBP enrollment to
provide adequate power for examining differences in responses between HHAs enrolled in
HHVBP and those not enrolled.
Summary of Findings from Cognitive Interviews
Overall, cognitive survey respondents did not encounter much difficulty answering the
various sections of the survey. In addition, respondents thought the survey items were
meaningful and relevant to HHAs. All respondents had the necessary knowledge about
CMS quality measures to answer survey questions without the assistance of others in their
agency.

The research team also used recommendations and feedback from the two rounds of
cognitive interviews to revise the survey instrument to better assess the impact of the
quality measures. First, respondents provided feedback to reduce redundancy across
certain sections of the survey. Second, respondents indicated that several survey items
were vague; their feedback led the research team to reword survey items to provide more
specificity and clarity. Third, respondents assisted in testing and selecting more
appropriate response scales for two questions and suggested more precise instructions for
the scales. Finally, the respondents provided more commonly used terminology for items
and suggested changing some response items to be more relevant to HHAs.

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

Introduction

CMS is committed to drive improvements in health care quality by implementing quality
measures across a variety of settings in which Medicare beneficiaries receive care. Section
3014(b) of the Patient Protection and Affordable Care Act (ACA) of 2010, as amended by
section 10304, requires the Secretary of Health and Human Services (HHS) to conduct an
assessment of the quality and efficiency impact of the use of endorsed measures every
three years and to make the assessment available to the public [1, 2]. Further, CMS is
committed to ensuring improved quality while reducing measurement burdens on
providers, as emphasized in the Meaningful Measures Initiative [3].

There is a lack of information regarding how home health agencies (HHAs) are responding
to CMS quality measures and how using CMS measures affects HHAs. Therefore, as part of
the 2018 Impact Assessment Report, CMS, collaborated with its contractors, the Health
Services Advisory Group (HSAG) and RAND, to develop a nationally representative survey
and qualitative interview series of HHAs to assess the impact of use of quality measures.
CMS expects to field the survey and conduct qualitative interviews with HHAs as part of the
2021 Impact Assessment Report, following review and approval of the survey and
interview guide by the Office of Management and Budget (OMB). This report summarizes
the background and development of the National Provider Survey of Home Health Agencies
and Qualitative Interviews, which will address the research question “What changes are
home health agencies making in response to the use of performance measures by CMS?”
This overarching question was translated into five specific research questions to form the
content of the surveys and interviews:
1. What types of QI changes have HHAs made to improve their performance on CMS
measures?
2. If a QI change was made, has it helped the HHA improve its performance on one or
more CMS measures?
3. What challenges or barriers do HHAs face in reporting CMS quality measures?
4. What challenges or barriers do HHAs face in improving performance on the CMS
quality measures?
5. What unintended consequences do HHAs report associated with implementation of
CMS quality measures?
To develop the data collection instruments and refine the survey design, we:
(1) Conducted an environmental scan of the published and grey literature to identify
prior studies that had examined the effects of use of quality measures in HHAs;
(2) Interviewed CMS staff responsible for quality measurement programs and
improvement initiatives among HHAs to determine CMS information-gathering
needs and policy priorities;
(3) Conducted formative interviews with HHAs to develop the content for the survey
instrument and interview guide and to test survey question language; and
(4) Cognitively tested the proposed questions to assess respondents’ understanding of
the survey and interview questions and to identify problematic terms, items, or
response options.

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

Environmental Scan
Methods
To find existing studies of HHAs that addressed any of the five research questions listed
above, we conducted targeted searches of PubMed, searching for studies of quality
measurement in HHAs and systematic reviews of QI efforts undertaken by agencies
(independent of quality measurement). Table 1 describes the search strategy and terms
used to identify relevant publications and studies published January 1, 2000, through April
3, 2018; the search was conducted April 3, 2018. The team also searched Google Scholar
and HHS websites (such as www.cms.gov) to locate prior systematic reviews and highly
cited publications and technical reports [4-7]. We conducted “reference mining” of articles
identified in the primary search to locate additional relevant studies; i.e., we examined
studies citing the article in question, as well as those cited by the article in question, to find
additional relevant articles. The team obtained 1,107 citations from PubMed and 23
citations using Google Scholar and reference mining.
Table 1. PubMed Search Strategy
Search #
#1
#2

Category
Home Health

Quality Measures

Search Terms
Home Health Nursing[MH] OR Home Care Services[MH] OR Home Care
Agencies[MH] OR "home health"[TIAB] OR home health agenc*[TIAB]
Quality Indicators, Health Care[MH] OR "quality rating"[TIAB] OR
"performance measure"[TIAB] OR "quality measure"[TIAB] OR pay-forperformance[TIAB] OR "public reporting"[TIAB] OR process
measures[MH] OR "process of care" OR "process measures" OR
"processes of care" OR "process measure" OR (("NQF" OR "national
quality forum") AND (practices[TIAB] OR measures[TIAB])) OR "process
and outcome" OR "process to outcome" OR "payment for performance"
OR "pay for performance"[TIAB] OR p4p[TIAB] OR "pay for value"[TIAB]
OR "financial incentive" OR ((bonus[TIAB] OR reward[TIAB]) AND
(payment[TIAB] OR reimburse*[TIAB] OR incentive*[TIAB]) AND
(quality[TIAB] OR value[TIAB])) OR "quality and outcomes framework"
OR Outcome Measures[MH:noexp] OR Outcome and Process
Assessment [MH:noexp] OR quality indicators, health care[MH:noexp]
OR quality of health care[MH:noexp] AND (measure*[TIAB] OR
indicator[TIAB] OR public reporting OR public disclos* OR "pay for
performance"[TIAB]) OR "quality indicators"[TIAB] OR ((CMS OR
Medicare) AND measure*[TIAB]) OR "quality measures"[TIAB] OR
"quality measure"[TIAB] OR "performance measure"[TIAB] OR
"performance measures"[TIAB] OR "process performance" OR "process
metrics" OR ("performance-based" AND outcome) OR "Dialysis Facility
Compare" OR "Consumer Assessment of Healthcare Providers" OR
CAHPS OR "Home Health Compare" OR (OASIS AND quality) OR "home
health compare"[TIAB] OR Home Health Quality Reporting Program OR
"HH QRP"[TIAB] OR "public reporting"[TIAB] OR (public AND "report
card"[TIAB])'

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies
Search #
#3

#4
#5

Category
QI Changes

Systematic or
Integrative
Reviews
Language
restriction, date
range, and
abstract
requirement
Final Search for
studies of quality
measurement in
home health
setting
Final Search for
studies of quality
improvement in
home health
setting

Search Terms
"learning organization"[TIAB] OR (continuous learning AND (staff OR
employee)) OR "culture of safety"[TIAB] OR electronic health records OR
clinical decision support OR CDSS OR CPOE OR computerized physician
order entry OR medication administration system OR electronic alerts
OR automated prompts OR information exchange OR risk prediction OR
manage high-risk OR standardized care protocols OR checklists OR
appropriateness OR decision support OR "care redesign" OR reengineering OR Deming OR Lean Engineering OR "Six Sigma" OR Plan Do
Study Act OR PDSA OR improvement cycles OR interdisciplinary OR
huddle* OR multi-specialty OR (collaboration staff OR Situation
Background Assessment Recommendation OR SBAR OR hand off OR
paging protocols) OR discharge clinic OR (track* AND outcomes) OR
feedback OR (nurse OR physician OR doctor OR clinician OR frontline)
AND (bonus OR incentive) OR (management OR leader) AND (bonus OR
incentive) OR staff AND (award OR recognition) OR staff AND increase*
OR champion OR ((staff OR staffing OR hours) AND (change OR deploy*
OR increase* OR decrease*)) OR quality improvement initiative OR QII
OR continuous quality improvement OR CQI OR quality improvement
training OR technical assistance OR "quality improvement organization"
OR QIO OR quality improvement collaborative OR (consulting firm OR
consultant) OR (innovat* OR improv* OR implement* OR adopt* OR
create* OR creating OR creation OR strategy OR strategies)
"systematic review" OR meta-analysis OR metaanalysis OR (review AND
(MEDLINE OR PubMed))
English[LA] AND 2000:2018[DP] AND hasabstract*

#1 AND #2 AND #5

#1 AND #3 AND #4 AND #5

* “hasabstract” is a term used to limit a search to those articles that have abstracts.

To obtain the final studies, we determined whether the retrieved citations met three
criteria, using the title, abstract, and, if necessary, the full-text of each study. First, we
required that the studies take place in HHAs. Second, we excluded commentaries and
editorials but allowed other publication types, including the following: nationally
representative surveys, randomized controlled trials of QI interventions, cohort studies,
case studies, and reviews of any type of intervention. Finally, we required that studies
describe QI interventions, drivers of improvement (such as public reporting), barriers to
Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

reporting, barriers to improvement, and unintended consequences of quality measures
faced by HHAs.

The heterogeneity of the literature precluded us from performing a formal meta-analysis.
We therefore summarized all identified surveys qualitatively while using other studies to
identify potential survey and interview questions. To supplement these results, we
reviewed the results of the Hospital and Nursing Home National Provider Surveys [1], as
well as the environmental scans used to develop both surveys.

Findings from the Environmental Scan
Below we present findings from the literature review, organized by research question and
within each category under each research question. The categories and subcategories
follow those used in the Hospital and Nursing Home National Provider Surveys [1].

QI Changes Employed by HHAs

Changes to Organizational Culture. We did not identify studies that described how HHAs
were trying to spur changes to organizational culture in their agencies, such as fostering a
culture of safety. Furthermore, although a previous review noted two small studies
describing interventions to improve work environment, there was little correlation with
improved outcomes among existing studies [8].
Health Information Technology. Using data from the 2007 National Home and Hospice
Care Survey, Resnick and Alwan (2010) estimated that 43% of HHAs were using electronic
health records (EHRs) and 29% used point-of care documentation [9]. These usage
patterns represent an increase from the 32% of HHAs that used any type of computerized
medical record system in 2000 [9]. However, the 2007 survey did not report whether
HHAs had implemented EHRs to improve performance on quality measures, to improve
quality of care overall, to add efficiency, or to meet other business goals. HHAs also used
EHR and point-of care documentation for multiple purposes, including email, scheduling,
and other functions.

A review of QI interventions in home health agencies identified a small, uncontrolled study
that used email reminders to agency staff to improve patient education for heart failure
(e.g., dietary guidelines); reminders were found to be helpful for improving adherence, at
least in the short term [10]. Parker et al. (2014) also noted case studies in which home
health agencies and health care organizations implemented methods for transmitting
clinician orders to agency staff [10]. However, we did not identify studies of EHRs and
associated functionalities in HHAs published in the last decade, a period in which EHR
usage has grown substantially in hospitals [11] and might have grown among HHAs as well.

Care Process Redesign. We identified two studies of care redesign processes in home
health agencies. A 2007 CMS national campaign prepared QI educational materials to
reduce hospitalizations among home health patients [12, 13]. Strategies provided to HHAs
included hospitalization risk assessment, patient self-reports of changes in health status,
phone monitoring and triage, medication management, protocols and standing orders for
tracking immunizations, improved communication with physicians, fall risk assessment
Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

and reporting techniques, patient education, transitional care coordination, and disease
management [12, 13]. The study demonstrated a greater reduction in those most engaged
with adoption but no significant effect overall; in addition, the campaign was voluntary (not
randomized), which may have biased the results.
We identified subsequently published studies on care process redesign, but these were
case studies or small trials; they did not determine what proportion of HHAs adopted these
processes. For example, Husebø (2014) reviewed telehealth interventions for social
inclusion but did not find studies of cost savings, clinical effectiveness, or usage [14].
Additional small studies examined methods for improving care transitions, including the
Transitional Care Model (TCM) and the Care Transitions Program (CTP); small randomized
trials showed reductions in acute hospitalizations [10]. Parker et al. (2014) identified five
uncontrolled (before-after) studies that examined the effects of multidisciplinary teams on
hospitalization rates; the interventions used combinations of administrators, quality
leaders, nursing, pharmacists, and other staff and were noted to reduce hospitalization
rates [8]. One case study used demographic, Outcome and Assessment Information Set
(OASIS), and clinical data to predict and manage hospitalization risk among patients;
another case study used a combination of telemonitoring, multidisciplinary teams
(including administrators, quality leaders, nursing leadership, pharmacist, physical
therapist/occupational therapist, and others), care protocols, and in-service education to
reduce hospital readmissions for congestive heart failure (CHF) [15, 16].

We also identified other types of QI changes not included in the Hospital and Nursing Home
National Provider Surveys, including telehealth initiatives, remote monitoring of patients,
and promotion of patient self-management techniques, although their utilization rates are
unknown [12, 13, 15].

Performance Monitoring and Feedback Reports. Feedback reports can be an individual
innovation designed to improve quality or one part of a performance measurement system
implemented to improve overall quality. We did not identify studies of the effectiveness or
usage of feedback reports in home health agencies.
Linking Quality Indicators to Financial Incentives for Clinicians and Senior Clinical
Leaders. Payers (e.g., commercial insurers, Medicare, and Medicaid) and organizations
such as hospitals or HHAs may seek to influence provider and organizational practices by
using financial incentives tied to performance. We did not identify studies testing
incentives in HHA staff or usage rates among home health agencies.

Changes in Staffing. We did not identify studies of home health agencies changing staffing
patterns or responsibilities because of CMS quality measurement programs or as part of
unrelated QI efforts.

Technical Assistance from Quality Improvement Organizations (QIOs). QIOs contract
with CMS to provide technical assistance services to CMS-contracted health care providers.
In a CMS-funded study of QIOs, HHAs were assigned to receive typical QIO technical
assistance; 37% received standard interventions, while 63% volunteered to participate in a
Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

more intensive technical assistance program [17]. Physician groups and nursing homes
were also studied. The study found that HHAs (and providers in other settings) that opted
for the more intensive technical assistance program had greater improvements in quality
measure scores than providers that did not pursue more intensive technical assistance.

Provider Education. We did not identify nationally representative studies on the usage of
staff education as a QI change in home health agencies, nor reviews of the effectiveness of
staff education. Despite the lack of evidence, it is likely that provider education is a widely
used QI strategy in HHAs; for example, three case studies noted the use of staff training
(alone or as part of a multicomponent intervention) to reduce urinary incontinence, CHF
readmissions, and overall hospitalization rates [13, 16, 18].
Drivers of Improvement

Public Reporting. Incentives or mandates for releasing quality measure scores to the
public (as raw data or as provider report cards) represent an attempt by payers to increase
provider accountability for quality of health care. For example, by strongly incentivizing
hospitals (or other organizations) to publicly release quality scores, CMS has been
attempting to steer patients and provider referrals toward higher-quality facilities [5].
However, in a systematic review, Totten et al. (2012) were able to identify only a single
quantitative study of how home health public reporting affected clinical outcomes [7]. In
that uncontrolled time series study, the authors showed that the launch of Home Health
Compare was associated with possible improvement in functional measures but no
improvement in hospitalizations [19]. Furthermore, Totten, et al. (2012) were unable
to identify evidence for providers or patients using the information in Home Health
Compare [7].
Pay-for-Performance Programs. We did not identify studies that directly address
penalties or incentives for quality performance in home health agencies, such as the Home
Health Value-Based Purchasing (HHVBP) model. However, prior payment reforms
(including per-person caps on spending) were associated with reduced hours provided by
HHAs [20], which suggests that financial incentives related to the HHVBP model could
drive changes in care delivery as well.
Regulatory Requirements. CMS sets regulatory requirements that home health agencies
participating in the Medicare program must follow [21]. We did not identify studies
comparing the relative importance of home health regulatory requirements versus CMS
quality measurement programs.
Barriers to Reporting

We did not identify studies of barriers to reporting in the home health setting.
Barriers to Improvement

We did not identify studies of barriers that home health agencies face in improving
performance on CMS quality measures.

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

Unintended Consequences

There were no high-quality studies directly estimating the prevalence of unintended
consequences of home health quality measurement programs [2]. One observational study
noted that additional measures being reported publicly was not associated with greater
agency exit rates in areas with lower socioeconomic status [22]; this finding suggests that
public reporting did not worsen disparities, but the measure of public reporting used
(proportion of measures reported) would be unlikely to distinguish between providers.
Additional potential unintended consequences such as the following were not assessed:
(1) fewer resources for QI in areas of clinical care that are not the focus of CMS
performance measures; (2) focus on narrow improvement for specific measures rather
than across-the-board improvement in care; (3) overtreatment of patients to ensure that a
measure is met; (4) increased focus on documentation or coding of data to attain a higher
score; (5) changing coding of data or documentation to ensure that a measure is met; and
(6) avoiding sicker or more challenging patients when providing care.

Findings from Studies in Other Provider Settings
Given the paucity of research studies pertaining specifically to actions taken by HHAs, we
also used the Hospital and Nursing Home National Provider Surveys conducted as part of
the 2018 National Impact Assessment to identify survey and interview questions and
potential response options [1]. In addition, we reviewed the environmental scan that was
used to develop those surveys; it summarized studies of responses to quality measurement
programs by hospitals, nursing homes, and physician practices.
Using both sources, we identified numerous potential QI changes in the following
categories that HHAs may also be using to improve care:
• Changes to organizational culture [23]
• Health information technology (health IT) [6, 24]
• Care process redesign [25]
• Provider incentives (financial and non-financial) [26, 27]
• Changes to staffing levels or responsibilities [28-31]
• Performance monitoring and feedback reports [32, 33]
• Technical assistance from QIOs [17, 34, 35]
• Provider education [36-38]
• Other actions, including measure-specific QI initiatives

In addition, the environmental scan noted external factors that potentially drove
performance changes in other settings, including pay-for-performance programs [4], public
reporting initiatives [39], and regulatory requirements (including accreditation) [40, 41].
In a study of primary care providers, the time required to implement indicators was the
most common barrier to successful implementation reported by providers [42]. Financial
costs also have been shown across studies to be a barrier to QI measure reporting in
diverse settings such as primary care and hospitals [42-45]. These studies from other
settings suggest that barriers to reporting might be a consideration in the home health
setting, although, EHR systems are becoming an important component of quality reporting
in the U.S. home health setting and may ease reporting burdens [9]. The prior

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

environmental scan also identified barriers to improving performance in other settings,
including data-gathering inefficiencies and other technological barriers, financial costs, lack
of provider acceptance, lack of support from leadership, and excessive labor requirements
for measure reporting [23, 42, 46-48]. Finally, the prior environmental scan included a
systematic review of studies of unintended consequences of quality measures but found
mixed results linking quality measures to key unintended consequences in hospitals and
nursing homes [2].

Summary of Findings from Environmental Scan
In our review of the literature, we identified no recently published nationally
representative surveys examining the prevalence of QI interventions employed by HHAs or
responses by HHAs to quality measurement programs. However, it is likely that a large
proportion of HHAs are using EHRs and relying on technical assistance from QIOs. In
addition, numerous case studies and usage of QI strategies by hospitals and nursing homes
suggest that HHAs also are changing care delivery to improve the quality of their care;
similarly, studies from other settings suggest that HHAs are likely facing barriers to
improvement and experiencing potential unintended consequences. This environmental
scan suggests that these topics should be explored in a nationally representative survey of
HHAs, with modifications to accommodate QI interventions uncommon outside of the HH
setting.

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

Interviews with CMS Staff to Review Goals of Survey and Determine
Priorities for Data Collection Needs
Methods
To gauge CMS priorities for data collection, the study team conducted one-hour telephone
interviews with CMS staff involved in home health QI from the Quality Measurement &
Value-Based Incentives Group (QMVIG), the Quality Improvement and Innovation Group
(QIIG), and the Center for Medicare & Medicaid Innovation (CMS Innovation Center).
Conversations were later continued over email if necessary. The purpose of these
interviews was to review the goals of the survey, to identify the content areas the surveys
would cover, and to understand how the survey and interviews would address issues
central to the work of these diverse groups within CMS. We also shared proposed survey
questions and response options to get feedback on specific areas. Findings are arranged by
CMS group.
Findings from Interviews with CMS Staff
Quality Improvement and Innovation Group

QIIG provides technical assistance to HHAs and oversees the Quality Improvement
Organization Program. We discussed the survey instrument and CMS program priorities
with QIIG representatives with expertise in the home health setting by telephone on April
12, 2018, and delivered written feedback on April 25, 2018. QIIG staff provided feedback
on barriers to improving performance, barriers to reporting data, and QI changes adopted
by home health agencies.
Barriers to Improving Performance on Quality Measures

QIIG staff suggested that inability to retrieve quarterly data removed from Home Health
Compare was a barrier to improving performance. Furthermore, HHAs had faced difficulty
obtaining needed information from other providers to focus QI efforts (e.g., reasons for
hospitalizations or ED visits).
Barriers to Reporting Quality Measures

QIIG staff members asserted that larger home health agencies do not face significant
barriers to reporting because they can hire persons specifically for assisting with measure
reporting. However, smaller agencies may face difficulties because agency staff are
required to fulfill several roles in addition to quality reporting.

QIIG staff suggested that the survey responses regarding barriers to reporting focus on
OASIS because agencies would likely focus on OASIS if describing problems with quality
measure reporting. For example, OASIS has changed substantially over time, and each
change requires effort to ensure data accuracy. In addition, HHAs may not be aware of how
to use CMS data resources or state-based OASIS educators, and CMS education on OASIS
items primarily addresses older OASIS versions. As a result, HHAs typically pay extra for
OASIS training, and HHAs expend considerable time on responding to version changes.
Finally, QIIG staff asked us to list definitions of key words and phrases such as “CMS
measures” to make sure that all survey questions were understandable to respondents.
Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

Survey Terminology

Both QIIG and QMVIG staff recommended that we tailor the response options to exclude
physicians because physicians have limited relevance as frontline clinicians for home
health agencies, although they may be involved as home health administrators. In addition,
staff suggested that the survey instrument substitute “clinical” or “clinician” for “nursing”
or “nurse” in questions. Unless the survey topic specifically concerns nurses rather than
home health clinical staff more generally, survey respondents should consider a wide array
of frontline providers (such as physical therapists and dieticians) when providing survey
responses.
QI Changes Employed by HHAs

QIIG staff confirmed that QI activities undertaken by hospitals and nursing homes are
similar to those in HHAs but suggested changes in several areas.

Health Information Technology. QIIG staff confirmed that health IT would be a useful
topic to explore on the survey. They noted that although most HHAs use EHRs, few have
systems that are interoperable with “upstream” providers (i.e., hospitals and ambulatory
providers). In addition, HHAs are not enrolled in Medicare and Medicaid EHR Incentive
Programs or other programs designed to increase interoperability. QIIG staff did note that
hospital-based HHAs have an advantage because most of their referrals originate within
the same system. (To address this issue, the survey will ask HHAs to indicate whether they
belong to a system or are owned by a hospital as part of our routine background
questions.) QIIG staff were interested in interoperability, especially for larger agencies,
and in determining how EHR systems are used (patient management versus strictly OASIS
reporting versus interoperability in systems). Additional QI initiatives related to EHRs,
such as decision support, are likely underrepresented in HHAs (or are happening only in
larger corporate agencies), but understanding their use is a priority for CMS.

Care Process Redesign. QIIG staff noted that one practice not typically considered by
hospitals or nursing homes includes front-loading of visits by clinicians to ensure that
patients have greater contact with clinicians at the beginning of their care episodes. QIIG
staff confirmed that HHAs use telehealth activities and remote monitoring of patients, as
well as patient education efforts for self-management. QIIG staff also indicated that the
term “huddles” is less commonly used in home health settings and suggested the terms
“interdisciplinary team meetings (IDTs)” and “case conferences” instead of or in addition to
“huddles.” Similar to providers in other settings, HHAs have collaborated with external
health care providers to improve performance (e.g., reducing readmissions, disease
management). Given the difficulties HHAs have experienced integrating with outside
providers, it is unclear to what extent this collaboration has occurred, but understanding
the usage of this strategy is a key need for CMS.
Linking Quality Indicators to Financial Incentives for Clinicians and Senior Clinical
Leaders. QIIG staff suggested that home health staff might find alternative language more
understandable when describing provider incentives, such as “incentive-based
performance reviews or raises” or “used performance on CMS measures as basis for
determining incentives for frontline clinical staff or care management teams.”

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

Changes in Staffing. In addition to asking about increases in staff dedicated to QI, QIIG
staff recommended that the survey ask about agencies adding “scrubber-type” programs.
These programs are electronic systems that review OASIS answers and use algorithms to
assist with improving OASIS accuracy, which reduces use of manual clinical data extractors.
QIIG staff had some concerns that the systems were not tested sufficiently and may work to
extract inaccurate responses in an automated fashion.
Technical Assistance from QIOs and CMS. QIIG staff members mentioned that technical
assistance is an important issue for HHAs and should be explicitly asked about in the
survey. QIIG works with roughly 50% of the agencies that are Medicare certified, and it
would be helpful for CMS to know how many agencies know about the existence of
technical assistance. QIIG staff also recommended that we specifically describe the CMS
Home Health Quality Improvement initiative as an example of an organization that directly
works with HHAs and with QIOs.
Defining Universe for Sampling Purposes

QIIG staff recommended that all certified agencies be targeted, including those without
quality scores or OASIS assessment data. QIIG staff also indicated that using patient counts
as a proxy for size was acceptable but said we should consider using home health claims
rather than OASIS assessments because some Medicare-certified agencies decline to submit
OASIS assessments.
Quality Measurement and Value-Based Incentives Group

QMVIG oversees the development of quality and efficiency measures and patient
assessment instruments and the implementation of quality reporting programs for HHAs.
Representatives from QMVIG with expertise in home health, as well as contractors,
provided feedback on the survey instrument and sampling design by telephone and by
email during March and April 2018.
Meaningful Measures Initiative

QMVIG staff noted that both the survey instrument and interview guide should account for
the new Meaningful Measures Initiative, which focuses measurement on high-impact areas
and aims to incorporate additional patient-centered and outcome-based measures while
minimizing the level of burden for providers. In accord with this concern, the contractor’s
staff recommended that survey content address both positive and negative responses to
quality measures (e.g., barriers and facilitators to improvement on quality measure
performance).
Survey Design and Sampling

CMS contractors managed by QMVIG noted that size was a reasonable stratification
criterion because generating well-powered estimates for smaller agencies and larger
agencies would be helpful. QMVIG contractors also commented on specific thresholds for
size, suggesting that a cutoff of approximately 60 patient episodes would identify small
agencies, since HHAs with fewer patients do not need to report the Consumer Assessment
Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

of Healthcare Providers and Systems (CAHPS®) Home Health Care Survey ii (HHCAHPS).
Medium-sized would be ~60–1,000, while >1,000 would be used to define large HHAs.
QMVIG staff noted that some HHAs previously made a business decision to accept the 2%
penalty by CMS for not reporting quality data, but OASIS reporting became mandatory for
home health claims processing as of April 1, 2017. It is also mandatory for all HHVBP
participants (i.e., all agencies in the nine states included in the HHVBP model). As a result,
either home health claims or OASIS assessments may serve as reasonable proxies for
agency size when preparing sampling frames when the survey is fielded in 2019 or 2020.
In discussing additional stratification criteria, one interviewee suggested stratifying on
ownership (government, non-profit, or for-profit), but the consensus was to use some
combination of size, HHVBP status, and three categories for quality (1≠2 stars, 2.5–3.5
stars, and 4–5 stars). In addition, CMS staff recommended selecting those without quality
ratings in the sampling frame.
Specific Measures of Interest

No consensus developed about how to gauge HHA concerns regarding key measures during
the upcoming survey. However, pressure ulcer measures were of interest to QMVIG staff
because aligning pressure ulcer measures across post-acute care settings is a key goal for
CMS. Measures of drug regimen reviews raised concerns among HHAs because agencies
believed they lacked the ability to coordinate with physicians and because it was difficult to
meet the time requirements specified in the measures (< 48 hours). Other topics of
concern to HHAs included standardization across settings and the transfer of health
measures, but these would not be adopted until 2021, after the survey is scheduled to be
fielded.
Center for Medicare & Medicaid Innovation

We interviewed the CMS Innovation Center staff by telephone on April 9, 2018, regarding
the HHVBP model and other areas within quality measurement.
Drivers of Improvement

In the hospital and nursing home surveys, providers were asked to rank several external
factors (e.g., threat of financial penalties, public reporting) regarding their importance in
driving performance improvement; a similar question will be included in the proposed
survey. Per the CMS Innovation Center staff, there will be no major changes to payment
policy before 2020 that this question should address. However, the survey instrument was
modified to incorporate the CMS Innovation Center staff’s recommendation for substituting
“risk” for “threat” when discussing responses to financial penalties for low performance.
The staff also suggested that we consider adding responses such as “improve care delivery”
or “excellence in care” (i.e., internal drivers) and that we ask respondents to rate all
responses using a 1–10 or Likert scale. In cognitive testing, this option was ranked highly
by all but one HHA. The question primarily aims to identify external drivers, though, and
including a response option for internal motivation might unduly affect the responses.
ii Hospital National Provider Survey – OMB Control Number 0938-1290 and Nursing Home National Provider Survey –
OMB Control Number 0938-1291 date is 01/31/2021.

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

Recurring Surveys of HHAs by the CMS Innovation Center

The CMS Innovation Center is sponsoring nationally representative surveys of HHAs with
the aim of determining whether HHAs enrolled in the HHVBP model differ in their QI
activities in comparison with HHAs not enrolled in HHVBP. (HHAs in Arizona, Florida,
Iowa, Maryland, Massachusetts, Nebraska, North Carolina, Tennessee, and Washington
were automatically enrolled in HHVBP, while HHAs in the other 41 states and the District of
Columbia were not enrolled.) The CMS Innovation Center is aiming for >1,500 completed
surveys; the survey asks HHAs about which measures (if any) are targets for QI activities,
the agencies’ current and planned QI activities (and whether those activities were taken in
response to the HHVBP model), and challenges associated with the HHVBP model. The
current survey is being fielded through the summer of 2018, while future surveys are
anticipated for 2020 and 2024. The CMS Innovation Center surveys devote less attention
to unintended consequences, barriers to reporting, and barriers to improvement related to
quality measurement than the proposed survey.
Sampling Design

We discussed potential survey stratification variables, including agency size, quality
ratings, and enrollment in the HHVBP model. The CMS Innovation Center staff agreed with
staff from other CMS groups that we should consider stratifying by size and quality. They
noted that it would be useful to compare small and large HHAs, but there are too few to
compare small versus large at the state level because there may be just eight agencies at
the state level. Comparing high- versus low-quality HHAs also would be useful to CMS, as
well as including HHAs with missing quality information.

The CMS Innovation Center staff noted that examining differences between HHAs enrolled
in HHVBP and those not enrolled would be a priority for CMS. As a result, they agreed with
oversampling HHAs enrolled in the HHVBP model to ensure that there is adequate power
for this comparison. The CMS Innovation Center staff provided one caveat: that the
HHVBP model is unlikely to affect penalties and incentives for enrolled HHAs until 2019.

Summary of Key Issues Identified in Interviews with CMS Staff
Our interviews with CMS staff suggested that the survey instrument and interview guide
from the Hospital and Nursing Home National Provider Surveys would be broadly
applicable to HHAs but would require modifications to be appropriate for this setting. For
example, although HHAs implement commonly used QI changes such as risk management
tools and provider education, HHAs also employ strategies not commonly used by hospitals
and nursing homes, such as front-loading of visits by clinicians, telehealth activities, remote
monitoring of patients, and patient self-management. We also made numerous changes to
the survey terminology, including curtailing use of physicians in examples, as they are not
as relevant to HHAs, and mentioning “scrubber” programs for data extraction instead of
more generic terms. We also aligned survey questions with the terminology of the
Meaningful Measures framework to better address CMS priorities. These changes were
tested in cognitive testing with HHAs.
CMS staff also noted that HHAs had faced unique difficulties obtaining and sending needed
information from other providers to focus QI efforts and may also have insufficient
Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

electronic tools for reporting data accurately to CMS. We therefore modified and retested
survey questions regarding EHRs.

The consensus among CMS staff was to consider using some combination of size, quality,
and enrollment in the HHVBP model to identify strata for the survey; they agreed that it
would be impractical to stratify by ownership type, census region, or state. In addition, the
consensus for quality cutoffs was 1–2 stars (below average), 2.5–3.5 stars (average), and 4–
5 stars (above average). However, nearly all CMS staff agreed that HHAs without quality
ratings should be included in the survey, which suggests that size and HHVBP status are
more important stratification criteria. Finally, the CMS Innovation Center is sponsoring
recurring nationally representative surveys of HHAs with the aim of determining whether
HHAs enrolled in the HHVBP model differ in their QI activities in comparison with HHAs
not enrolled in HHVBP. The CMS Innovation Center surveys do not focus on unintended
consequences, barriers to reporting, or barriers to improvement. The project team will
coordinate with QMVIG and the CMS Innovation Center to avoid excessive overlap between
the two survey approaches.

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

Formative Interviews with Home Health Agencies
Methods Used to Conduct Formative Interviews
The study team conducted formative interviews with nine HHAs by telephone in MarchApril 2014. (See Appendix A for the formative interview guide). The interviews focused
on:
• Assessing whether HHA respondents could understand the nature of the
information CMS sought to collect;
• Exploring language potential respondents would use to describe the topics that the
survey and interview guides would cover;
• Identifying potential response options or areas to probe related to specific survey
items or interview guide questions; and
• Determining the structure of the survey (e.g., open- or closed-ended questions and
potential response options for closed-ended questions) and an approach to
identifying appropriate survey respondents in HHAs.

The questions were qualitative and exploratory in nature, and the sample was limited to
nine HHAs; given the small number of interviews conducted, the results are not intended to
provide nationally representative results. Respondents received a check for $250 after
completing their interviews.

As part of the formative work, the research team purposefully selected nine HHAs that
varied with respect to performance on CMS measures, number of home health episodes,
for-profit status, and geographic region. iii We used data from calendar year (CY) 2013, the
most recent data available at that time. We selected three HHAs from the highest quintile,
three from the lowest; and three from the middle three quintiles. The formative interview
participants included four small HHAs (100–999 home care episodes in 2013), four
medium-size HHAs (1,000–7,499 episodes) and one large HHA (7,500+ episodes). Among
the participants were five for-profit HHAs and four not-for-profit HHAs. The final sample
included agencies from all four census regions: Northeast (n = 2); Midwest n = 2); South
(n = 3); and West (n = 2).

For the formative interviews, we sought to speak with senior leaders who were responsible
for clinical quality and safety in their HHAs. Such individuals would also be targeted to
complete the survey or qualitative interview. Among the nine HHAs that participated in
the formative interviews, the respondents had the titles of “Administrator” (n = 3),
“Director of Nursing” (n = 2), Chief Operating Officer (n = 2), and “Director of Clinical
Services” (n = 1); one had a clinical leadership position, but no title was noted in the
interview. This suggests that no single title is likely to predominate, so research staff need
to provide a broad range of titles when trying to identify appropriate respondents in each

iii Size was based on the number of patient assessments for the home health agency in CY 2013. Quality was based on
average of individual Home Health Compare measures from CY 2013 because Home Health Star Ratings were not
available at the time the formative interviews were conducted. To make the individual measures comparable, the project
team first standardized by subtracting the mean and dividing by the standard deviation.

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

agency. In addition, eight respondents were current or former nurses, although not all
nurses had experience delivering care in the home health setting.

Findings from Formative Interviews with Home Health Agencies
We summarize the formative interview discussions with home health agencies below,
including how the respondents understood and discussed the topics raised and the wide
range of responses evident in these small samples.
Notable Themes

Respondents tended to agree that their participation in Home Health Compare has led to
changes in care delivery by focusing their efforts and raising consciousness about quality
issues among home health providers (including nurses, therapists, and other clinical staff).
However, when probed about innovations in care delivery, respondents were less likely to
be able to identify specific QI changes without prompting. This was specifically true of the
smaller agencies. Those innovations in care delivery that were identified tended to relate
to specific processes of care in response to participation in the quality measurement
program rather than systemic changes. For example, several agencies identified new care
protocols and self-management strategies implemented to improve rehospitalization rates
and other functional outcomes associated with specific CMS measures. Two of the agencies
interviewed have not yet made major systematic changes to improve their scores on
quality measures. In addition, three agencies identified training on reporting OASIS iv data
as their main “innovation” rather than any changes in care delivery within their agency.

Public reporting via Home Health Compare was not considered a significant driver of
improvement by any agency; most respondents placed it last when asked to rank drivers of
improvement. All respondents reported that patients are not using the Home Health
Compare scores to choose a provider. On the other hand, while home health agencies had
not been subject to performance-based financial penalties or incentives (such as the
HHVBP model) at the time the interviews were conducted, only five agencies viewed this as
potentially a very significant driver of improvement.
The need for intensive training on OASIS data was a recurring theme. OASIS functional
measure data are collected at intake and at discharge, as well as at interim time points,
depending on episode length. Often a different provider collects this information at each
time point (e.g., a nurse will collect information at intake and a therapist at discharge).
Two agencies noted that this can introduce significant problems with inter-rater reliability
and, if staff is not properly trained on reporting of OASIS measures, can reportedly bias the
results of the measure, making it appear that patients have worsened when they have
actually stabilized or improved.
Finally, several respondents we spoke with had an EHR, and two noted that they worked
with an external vendor to cull their electronic medical record data and provide them with
more timely feedback reports than the lagged reports that CMS provides quarterly.
OASIS is the data collection tool used to collect and report performance data by home health agencies. OASIS data
measure changes in a patient's functional or health status between intake and discharge from care.

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

QI Changes Used by HHAs

Seven respondents agreed that their participation in Home Health Compare had led to
changes to improve care delivery, although they were mixed in their ability to pinpoint
specific QI or system changes. Two respondents were unable to identify any major
systematic changes undertaken to improve care delivery.

Among respondents from larger agencies with one or more dedicated QI staff (typically the
respondents themselves), major QI changes were easier to pinpoint. Six respondents who
could identify changes in care delivery identified changes to care protocols, patient selfmanagement strategies, and clinician education related to specific diagnoses and ultimately
to specific measures. Two agencies attempted to improve patient self-management by
distributing “stoplight” protocols that indicated what patients should do if they are in the
“yellow” or “red” zone for a condition; such protocols aimed to empower patients to better
manage their own health. The goal of these strategies is to identify any issues before they
worsen, which may lead patients to admit themselves into a hospital. In addition to the
above-mentioned strategies, one agency noted having implemented case management
system wide to try to improve across the spectrum of measures and quality in general. Five
respondents agreed, when asked, that the changes that they had implemented had
improved their performance on the quality measures. Several noted that rehospitalization
rates have improved, as this was identified as an area of focus for several agencies.
Rather than identify specific QI changes, five respondents noted that their most significant
improvement had been in monitoring measures and clinician education. Frequently
agencies report measure results to clinicians at regular staff meetings and trainings, and
sometimes the reporting of those results is paired with training on proper care protocols
related to specific measures. Agencies also conduct regular training on the reporting of
OASIS data, and this was sometimes described in the context of innovation related to
participation in Home Health Compare. In addition, five respondents noted that they were
already doing their best to maintain high standards of quality and that any innovations
identified would have been done in absence of the Home Health Compare program. One
representative respondent noted that participation in Home Health Compare “has focused
our attention on some of the outcomes we’re looking at. I’m not sure if that drives how we do
quality care here.” Rather than specifically driving their organization to improve quality
across the board, two agencies noted that their participation in the quality rating program
“raised consciousness,” particularly with respect to metrics not previously tracked by the
organizations.

Drivers of Improvement

Respondents were asked whether public reporting, feedback reports, regulatory
compliance, financial incentives, and the threat of penalties might drive QI efforts.

Public Reporting. While some mentioned that public reporting was a potential driver of
improvement, nearly all agencies indicated that this was the least important driver of
improvement among those mentioned. Six agencies suggested that families simply do not
refer to Home Health Compare when making their choices about HHAs, with one
respondent noting, “I have never had anybody come to me and say, ‘I chose your agency
Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

because I went online, and CMS says that you had really good outcomes.’” Some agencies
suggested that public reporting was more important in generating referrals from
physicians or hospitals, with one observing that “Physicians love to see quality scores.” This
sentiment was not universal, however, with other agencies suggesting that hospitals and
physicians make referrals according to a list of “pet agencies” with which they prefer to do
business.

Feedback reports were typically seen as useful, and most agencies used external vendors or
internal information technology to cull electronic medical record data and provide them
with more frequent and timely reports on their performance than the reports that CMS
provides. HHAs use their reports to disaggregate data—for example, at the level of the
branch in the case of larger providers with multiple branches. Only one agency specifically
mentioned the CMS Outcome-Based Quality Improvement (OBQI) reports. v Another
agency suggested that feedback reports would be useful but reported that it was not
currently getting such reports.
Pay-for-Performance Programs. The possibility of financial incentives and penalties
raised concerns with six respondents, although the HHVBP model had not yet been
initiated when the interviews were conducted. Five of those same respondents also
indicated that were their agency to enroll, the HHVBP model would become one of the most
important drivers of improvement. One reported concern of several agencies was
manipulation of data should financial incentives be incorporated into the program; one
respondent worried that other organizations would “just start lying on the answers. ... Every
program and every mandate that CMS puts together is just one more opportunity for the bad
guys to win.” Other agencies were concerned that their case mix would have a negative
impact on their performance. Particularly relating to financial penalties, several
respondents worried that performance on most of the Home Health Compare measures
was high across the board so that, for example, sliding from 98% to 97% on a certain
measure could change a percentile ranking.
Regulatory Requirements. Regulatory compliance was seen by all agencies to be an
important driver of improvement; six agencies ranked it as first or second most important
in their agency.
Barriers to Reporting

Respondents were nearly evenly split in their perceptions of the adequacy of their current
information technology to support reporting of OASIS measures. Respondents were mixed
on the usefulness of EHR and point-of-care documentation systems. For example, one
respondent suggested that “OASIS is pretty complex and time-consuming, and there isn’t a
vendor out there that has the perfect software for OASIS.” However, about half of agencies
reported no major challenges in this regard.
The OBQI outcome report is derived from OASIS data that is provided to agencies on a quarterly basis. OBQI reports
must be requested by agencies through an online system called CASPER.

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

Data quality was mentioned by nearly all respondents as a barrier to accurate reporting.
Two respondents also identified problems with inter-rater reliability on OASIS measures,
for which OASIS data are collected at intake and discharge. When different clinicians, often
in dissimilar roles, conduct the pre- and post-assessments, bias may be introduced on any
measures gauging functional improvement. Respondents noted that without extensive
training on the proper coding of the OASIS measures, functional outcomes can appear to be
declining. This issue often carries with it a perceived need to have a staff person dedicated
to auditing OASIS data to ensure that data are accurately coded.
Barriers to Improvement

In contrast to difficulties encountered in identifying barriers to reporting, respondents
found it easier to identify barriers to improvement on the Home Health Compare measures.
Three agencies identified the case mix of their patient population as a barrier to
improvement on measures, particularly those requiring improvement of patient
functionality. This was noted even though Home Health Compare outcome measures are
case mix-adjusted. Given the often-significant limitations of the patient population served
by HHAs, these agencies suggested that expecting improvement on functional limitations
might be too high of a hurdle:
“It’s just fact that … realistically speaking, these folks are not going to get better.
You’re trying to keep their … functionality at a reasonable level, where they can get
out of the house once a week, not necessarily trying to increase their functionality so
they can get out of the house three times a week.”

Ensuring reliable coding from intake to discharge was also noted as a barrier to
improvement. Specifically, one respondent noted that functional outcomes can appear to
be declining if their staff do not receive extensive training on the proper coding of the
OASIS measures.

Unintended Consequences

When asked about unintended consequences associated with participation in Home Health
Compare, six respondents identified, unprompted, that a significant consequence was the
staff burden of reporting the OASIS measures, resulting in less time to work with patients
(OASIS reporting at intake can take between two and three hours to complete). One agency
noted that “this stuff is so comprehensive and takes so much time to do, in some instances it
may overshadow taking care of somebody.”
Respondents were prompted to consider other potential unintended consequences. When
asked about overtreatment of patients, eight respondents did not think that this was an
issue with their agency. Similarly, when asked whether measures might cause agencies to
focus only on the areas measured by Home Health Compare, eight agencies did not
consider this to be an issue, and several agencies noted that they were focused on quality
more generally. Respondents did not perceive inaccurate or manipulative coding practices
as an issue, although they did conduct training on proper OASIS coding to support their
participation in CMS quality measurement programs. Respondents tended to agree that
agencies might avoid sicker patients; however, most noted that this was probably not a
Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

way to improve quality scores specifically, but rather a way to increase reimbursement and
profitability.
Lessons Learned

Only two respondents could identify additional lessons learned beyond what they had
reported elsewhere in the interview. These two responses were, however, instructive as
far as the range of possible attitudes toward the Home Health Compare quality initiative.
One respondent reported that the most important lesson learned was that staff education
and revised protocols could substantially improve performance on quality measures.
Another respondent suggested that improvement on the Home Health Compare measures
was dependent on greater attention to data quality rather than changes in care:

“It’s simply how you answer the question and if you have enough resources to monitor
the way that that information is presented to CMS. … If I could throw another FTE
and, you know, some more time in it, I would change the outcome of my numbers, but I
wouldn’t change the delivery of care.”

Concerns with Quality Measurement Programs and Suggested Changes

Throughout the interviews, respondents identified several areas of concern with the
program. Two respondents mentioned the clustering of quality measure scores around
very high values, an issue also mentioned above. They noted this as a concern with the
measurement program generally, but also with respect to the linking of financial incentives
to performance, as discussed above. Rates of adherence to process measures are
frequently quite high; thus, high levels of adherence that are lower in relation to other
organizations might still be penalized. As an example, one agency used a measure related
to the prevention of pressure sores: “Right now, we’re at 97%, but Medicare says we should
be at 98% and therefore when we rank against all the other agencies, we’re at the 41st
percentile.”
Two other agencies described the Home Health Care Consumer Assessment of Healthcare
Providers and Systems (HHCAHPS) vi survey as too long and overly burdensome for their
population. One such respondent noted that prior to HHCAHPS, the HHA had been
conducting its own much shorter patient survey with a good response rate, but the
HHCAHPS survey is six times longer and currently has a “dismal” response rate.

A patient survey that measures the experiences of patients receiving care from Medicare-certified Home Health agencies

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

Second, no HHAs considered public reporting to be the most significant driver of
improvement. Though HHAs reported making changes in response to public reporting via
Home Health Compare, they perceived that their patients did not use Home Health
Compare Star Ratings when selecting an HHA. However, most HHAs viewed the Home
Health Value-Based Purchasing (HHVBP) model as potentially a very significant driver of
improvement. To understand this issue in greater detail, we stratified sampling by HHVBP
enrollment to provide adequate power for examining differences in responses between
HHAs enrolled in HHVBP and those not enrolled.
Finally, six respondents had an EHR, and three noted that they worked with an external
vendor to provide them more timely feedback reports than the quarterly CMS reports. We
will therefore ask HHAs several in-depth questions about use of EHR data and data
reporting to CMS.

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

Cognitive Testing of Draft Survey with Home Health Agencies
Methods for Conducting Cognitive Testing
The study team drafted a closed-ended (i.e., standardized) survey based on what was
learned from the environmental scan, formative interviews with HHAs, and CMS
stakeholder interviews, as well as experience the team accrued in fielding similar surveys
in the hospital and nursing home settings as part of the 2018 Impact Assessment Report.
Using the draft instrument, the research team conducted eight cognitive interviews by
telephone in two rounds between April 2018 and August 2018 to assess respondents’
understanding of the draft survey items and key concepts and to identify problematic
terms, items, or response options. The survey instrument was revised after each round.

The project team planned to test the survey instrument on participants representing large,
medium, and small HHAs from different census regions, including a mix of high, medium,
and low performers on CMS quality measures. To do so, the research team purposefully
recruited eight HHAs that varied with respect to performance on the Home Health
Compare Quality Star Ratings, the number of OASIS assessments submitted in 2015 (as a
proxy for size), HHVBP participation, and census region. The cognitive interview
respondents included three small HHAs (1–100 OASIS assessments), two medium-sized
HHAs (100–1,000 assessments), two large HHAs (1,000+ assessments), and one agency
that submitted no OASIS assessments in 2015. The participants included two HHAs
participating in HHVBP and six non-participants. The final sample included agencies from
all four census regions: Northeast (n = 2), Midwest (n = 2), South (n = 2), and West (n = 2).
Finally, the respondents included HHAs with varying Star Ratings scores: one highperforming HHAs (4, 4.5, or 5 stars), three medium-performing HHAs (3–3.5 stars), two
lower-performing HHAs (1–2 .5 stars), and two HHAs that lacked a quality Star Rating. vii

The research team sought quality leaders from each organization who were responsible for
or familiar with QI activities within the organization and with CMS quality measures.
Cognitive interview respondents had the title of “Administrator” (n = 2), “Director of
Quality Assurance” (n = 2), or “Director” (n = 4). Participating HHAs received a mailed
hard-copy survey that they were asked to complete and have accessible during the
telephone interview.

Using a scripted protocol (see Appendix B), an experienced survey researcher conducted
the cognitive interview, reviewing each question with the respondent and probing to assess
the respondent’s understanding of the goal of the question and whether the response
options adequately and accurately captured the provider organization’s experience. Each
telephone interview was approximately 1½ hours in length. The interviewer noted survey
items or terms that were unclear or not relevant to the HHA and sought to determine why
the respondent selected particular response options. The interviewer compiled
respondent suggestions to clarify the wording of questions. Respondents received a check
or gift card for $300 for participating in the cognitive test of the survey. The RAND Human

vii

The project team subsequently revised the size and quality thresholds based on feedback from CMS staff.

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

Subjects Protection Committee reviewed and approved all interview protocols and
instruments used for formative and cognitive testing.

Findings from Cognitive Testing on Overarching Issues
Ability to Complete the Survey. Findings from the cognitive interviews demonstrate that
respondents had the knowledge needed to complete the survey. Seven respondents had
significant experience in their present managerial positions and had firsthand knowledge
regarding performance on CMS quality and efficiency measures, as well as efforts to
improve performance. Occasionally the designated respondents stated that they needed to
consult with other members in their agency to answer about their board of directors or to
obtain information about payment methods.
Respondents noted that the survey topics addressed important issues and that survey
items were meaningful and relevant to HHAs; for example, respondents provided
additional information easily when asked to elaborate on responses to individual survey
items.

Length of the Survey. The final version of the HHA survey includes 43 questions. Five
respondents indicated they completed the survey in sections because other work activities
prevented them from completing it in one sitting. However, all but one respondent was
able to finish the survey in one hour. Five questions and eight response options were
eliminated during testing to ensure that respondents will be able to finish the proposed
survey instrument in one hour.
Need for More Specificity. Ten survey items included in the first version of the survey
were described by respondents as vague. In response, the project team reviewed and
edited each item flagged as vague. For example, the first version included an item that
asked about difficulties “in reporting the CMS measures” in referring to data transmission
from the reporting platform. The item was revised to “transmitting data.”
Redundancy. Feedback obtained from the first round of cognitive interviews identified
redundant items (e.g., questions about dedicated resources and leadership support
included in various parts of the survey). In response, the research team deleted items to
eliminate redundant content and minimize the burden on respondents.

Detailed Findings from Cognitive Testing by Survey Topic
HHA’s Experience with CMS Measures. Cognitive interview respondents did not have
difficulty answering this section of the survey. They indicated that survey items were
specific and response options were relevant to home health agencies. The information
requested in this section was easily accessible via their reporting platform, as well as from
previous-year reports.
Based on respondents’ feedback, four response options in this section were reworded to
provide more clarity. For example, the option “insufficient resources (e.g., staffing)” was
reworded to “insufficient staffing to implement quality improvement strategies.” In

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

addition, the response scale to the question asking about an HHA’s experience on
improving certain types of measures was changed from a binary (yes/no) scale to a 5-point
Likert scale. Cognitive interview respondents found the Likert scale easier to answer; the
research team therefore incorporated the Likert scales in the final survey instrument
One common theme expressed in this section was the agencies’ difficulty in responding to
some of the CMS measures. Participants stated that it was difficult at times to improve
performance on some quality measures, especially when patients with multiple chronic
medical conditions were unable to improve. Although the agency provided the best
possible treatment, some patients would not improve, thereby lowering its scores.
Participants also commented that some CMS indicators do not measure improvements
made by home health agencies (e.g., patient education and follow-ups), which therefore are
not captured in the quality measures.

Innovations in the Delivery of Care. Cognitive interview respondents reported some
difficulty with answers in this section. They said that the question about changes home
health agencies had made to improve quality performance was lengthy with multiple
subsections and that some responses lacked specificity. In response to the feedback, edits
were made to improve specificity, either by rewording or by providing examples. For
example, respondents indicated that “team huddles” is not a common term among HHAs; it
was removed. To address the length of this section, two questions were moved to another
section, and four response options were shortened. During the second wave of interviews,
respondents seemed to have less difficulty with this section and said that questions seemed
clear and easy to follow. They also noted the importance of asking about this topic.

Challenges to Reporting the CMS Measures. Based on feedback from the first wave of
cognitive interviews, two questions about HHAs’ challenges in reporting the CMS measures
were reworded and combined into one question. Cognitive respondents indicated that
“reporting” OASIS data to CMS should be replaced with “transmitting OASIS data.” This
change was made and tested with the second wave of respondents; they did not report
difficulty answering this section, but one HHA suggested that the response option be edited
to reflect a broader range of reporting activities. Accordingly, the project team revised the
final version to ask respondents about their difficulty in “submitting and reporting OASIS
data.” Some respondents reported no difficulties in transmitting data to CMS, while others
mentioned the frequency of OASIS version changes or noted some difficulties in capturing
the data needed for measure construction and interpreting measure specifications.
Factors Associated with Change in Quality Performance. Cognitive interview
respondents did not have difficulty with these questions. This section of the survey
included a question that required respondents to rank factors that were most important in
their agency’s decision to invest in quality performance. Respondents had no difficulty
with the rank-ordering task. However, one common theme expressed by respondents was
that some factors were equally important, and they had to think carefully how to rank the
items. Six respondents indicated that regulatory requirements were the most important
factor, followed by Quality Assessment and Performance Improvement (QAPI)
Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

requirements as the second most important. In the first phase of testing, respondents
indicated that the ranking instructions were not clearly stated. In response, the project
team revised the instrument with new instructions. In the second wave of testing,
respondents did not have difficulty.

The second question in this section asked participants to rate the level of importance of
factors that helped their agency improve performance on all or some of the CMS measures.
Based on feedback from respondents, five factors on this list were removed to reduce
redundancy with questions from a previous section (e.g., dedicated resources and
incentives to staff).

Undesired Effects of CMS Quality Measurement Programs. During the cognitive testing,
the research team wanted to determine whether respondents would be willing to report on
undesired effects of the CMS quality measurement programs as part of a survey.
Participants did not have any difficulty in answering the questions in this section of the
survey. They were forthright in reporting unintended and undesired effects of the CMS
measures. For example, respondents reported that the focus on the CMS measures has led
many HHAs to focus on trying to raise their Star Ratings scores. Other indicated that some
HHAs might manipulate their OASIS reporting, while other agencies stated that the focus on
outcome measures reduces time to focus on patients’ care and well-being. When asked
about respondents’ willingness to honestly answer survey questions, responses were
mixed. Three respondents indicated that they were honest in answering this section
because they can provide justifications for their responses or want to let CMS know that
“these things are causing issues.” However, four respondents indicated that they or other
agencies might not be as honest because “the stakes are extremely high.”
Based on feedback from participants, “areas of care not measured by CMS” was revised to
“broader improvements in areas of care beyond what is measured by the CMS quality
measures.” In addition, two questions from the Innovations in the Delivery of Care section
were placed in this section for better survey content flow.

Perspectives of Your HHA’s Leadership and Other Stakeholders. Participants did not
encounter difficulties answering this section of the survey, but they indicated that
participants not in managerial or supervisory positions would encounter difficulties
answering questions about an agency’s board of directors and leadership. Furthermore,
they stated that questions rating the promotion of a culture of quality or the level of
support by the agency’s leadership should not be answered by respondents in leadership
positions. They indicated a possible bias in rating these factors, as well as lack of
objectivity in their responses. However, the appropriate survey respondent may be in a
leadership position (such as the chief executive officer or chief operating officer), and in
such cases it will be impossible to direct surveys toward lower-level staff. To address this
potential source of bias, the project team will consider conducting sensitivity analyses
excluding such respondents when analyzing survey responses regarding leadership
evaluations.
Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

Participants reported that their agency’s board and senior leadership regularly review and
discuss the agency’s performance on CMS measures and that agency leadership is equally
engaged in financial performance issues and quality performance issues. Seven
respondents described their agency’s leadership, board of directors, and clinical staff as
being supportive of the agency’s efforts to improve performance on CMS measures.
Use of Health Technology. Cognitive interview respondents did not have difficulty
answering this section of the survey. Participants indicated that the questions and answer
options are relevant to home health agencies. Six participants reported having an EHR
system that allows health care providers to access clinical data, diagnostic summary, lab
tests and other information. Two agencies reported not having an EHR, which hindered
their ability to quickly exchange information with providers or report OASIS data to CMS.
Characteristics of Your HHA. Participants were generally able to answer the questions in
this section, but only two of nine respondents were familiar with “accountable care
organizations (ACOs)” or “global budgets” or “bundled payments”; one respondent
reported participating in an ACO. Another respondent needed to consult with her billing
department to answer this section of the survey. In addition, respondents suggested
revisions for clarifying “home health agency affiliated with” to “home health agency
freestanding.” Three questions were therefore reworded to clarify HHAs’ structure, and
four low-priority questions were dropped.

Respondent Background. Cognitive interview respondents did not have difficulty
answering questions in this section. Respondents suggested adding “Clinical Manager” to
the job title question and requested examples of formal training/certification on QI
strategies. To clarify, the project team noted that the Institute for Healthcare Improvement
offers courses and certification in QI strategies, including Plan-Do-Study-Act cycles.

Summary of Findings from Cognitive Interviews
Overall, cognitive survey respondents did not encounter much difficulty answering the
various sections of the survey. In addition, respondents thought that the survey items were
meaningful and relevant to home health agencies. All respondents had the necessary
knowledge about CMS quality measures to answer survey questions without assistance of
others in their agency.
The research team also used recommendations and feedback from the two rounds of
cognitive interviews to revise the survey instrument to better assess the impact of the
quality measures. First, respondents provided feedback to reduce redundancy across
certain sections of the survey. Second, respondents indicated that 10 survey items were
vague; their feedback led the research team to reword survey items to provide more
specificity and clarity. Third, respondents assisted in testing and selecting more
appropriate response scales for two questions and suggested more precise instructions for
the response scales. Finally, the respondents provided more commonly used terminology
for items and suggested changing response items to be more relevant to HHAs. Following
are Appendices A and B (interview guides) and the list of Citations.

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

Appendix A: Formative Interview Guide for Home Health Agencies

Respondent Type
Organization Name:
Respondent Name:
Respondent Position:
Interviewer Name:

Interview Date:
INTRODUCTION AND PURPOSE OF THE INTERVIEW
Before we get started, I’d like to briefly review the purpose of this interview and the
confidentiality provisions that were described in detail in the email we sent you.

 As you know, the Centers for Medicare & Medicaid Services (CMS) uses a number of
quality measures to assess the quality and efficiency of the care provided to
Medicare beneficiaries. For example, CMS creates and reports quality measures in
Home Health Compare. RAND has been asked by CMS and the Health Services
Advisory Group (HSAG) to help assess how quality measures affect organizations
and the care they provide.

 We’ve come to you to help us better understand how home health agencies have
experienced the CMS quality measures. Your insights will help us develop a survey
that we may conduct in the future with a large of group of home health agencies
across the country.
 We would like to ask you about the impact of these measures on the delivery of
home health care, any unintended consequences that may have resulted, and
barriers your home health agency has encountered in participating in quality
reporting and making improvements on these measures, but first we’d like to
review the confidentiality provisions for this interview.

CONSENT

→ All of your responses are confidential.
→ No one outside of the research project will have direct access to the information you
provide. The evaluation team will only produce summary information from our
collective set of interviews. You will not be identified by name or home health
agency affiliation.
→ You do not have to participate in the interview, and you can stop at any time for any
reason.
→ You should feel free to decline to discuss any topic that we raise
Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

Do you have any questions? (Yes/No)
Do you agree to participate in the interview? (Yes/No)
As we mentioned in our email, we would like to tape the interview if that is alright with
you.
Do you agree to being tape recorded? (Yes/No)

If yes: Great. Let’s get started. I’ll start the recording.
If no: That’s fine. We will take notes—and not tape the discussion. Let’s get started.

Interview

We’d first like to ask you a couple of questions about your position and professional
background.

Respondent Background

1. We understand that you are the Administrator of [agency name]. Is that correct?
2. What is your educational background?

3. How long have you been working at [agency name]?
4. How long have you been the [title] here?

5. [If not already volunteered:] Did you work for any other home health agencies
before [agency name]? How long have you been working for home health
agencies?

CMS Quality Measures—General

As you know, CMS requires home health agencies to collect and report OASIS data that are
then used to create a number of quality measures. Data on these measures are made
available to the public through the Home Health Compare website. Examples include how
often patients got better at getting in and out of bed and how often the home health team
included treatments to prevent pressure sores (bed sores) in the plan of care.
6. How are you involved in reporting and improving performance on CMS quality
measures here at [agency name]? [Possible prompt: Do you have experience
completing OASIS? How do you participate in QI?]
Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

7. Does your agency have an electronic medical record?

8. Is your agency part of a larger organization or corporation?

9. Who has responsibility for the overall quality and improvement of the services
provided by your agency? [Possible prompt: Does your agency have a quality
improvement or quality assurance administrator? A quality committee?]

We’ve sent you the full list of measures under discussion, and we’d like to ask you to think
generally about these measures—and how they’ve affected the quality and efficiency of
care at [agency name]. Let’s start with innovations or changes in the way care is delivered.

Innovations in Delivery of Care [M2 and M3]

10. In your experience, has the CMS measurement program for home health quality
led to changes to improve the delivery of care at [agency name]?

11. [If no:] Why is that? [Possible prompts: Improvement has not been needed? Lack
of resources?]
12. [If yes:] Let’s talk a little more about the changes in the delivery of care. What
kinds of changes has [agency name] made to improve performance on the CMS
quality measures? Would you give us a couple of examples?

13. In working to improve in this area(s), did you monitor a particular home health
measure? Is so, which measure or measures?

14. There are multiple aspects of the CMS quality measurement program that might
motivate or drive home health agencies to undertake efforts to improve the
delivery of care. I’m going to mention some specific, possible drivers of
improvement and ask you to discuss the importance of each as a driver of
improvement in your experience. The possible drivers include (1) public
reporting of quality scores, (2) receipt of feedback reports with quality data, and
(3) regulatory compliance. While I know that home health agencies have not yet
been subject to CMS Pay-for-Performance or Value-Based Purchasing programs,
I’d also like to ask you to discuss how important you expect (4) the potential for
financial incentives and (5) the threat of penalties to be as drivers of
improvement, when these programs do start up.
How important is public reporting of quality scores as a driver of improvement?
[Possible prompts: Patient response? Impact on referrals from hospitals and
ACOs?]

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

How important is the receipt of feedback reports with quality data as such a
driver? How do you use them? [Possible prompts: OASIS reports, OBQI reports,
OBQM reports—how do you use them to prepare for state surveys?]

How important is regulatory compliance as a driver of improvement? [If
important:] In what ways? Could you give us an example?

How important do you expect the potential for financial incentives to be in
driving future improvement efforts in your agency? [If important:] Please
elaborate.

How important do you expect the threat of penalties to be in driving further
improvement efforts?

15. [If yes:] Which of these possible drivers—public reporting of quality scores,
feedback reports, regulatory compliance, financial incentives, or penalties, –
would you say is most important—or potentially most important? Which is least
important? [Possible prompt: corporate assistance]
16. For the national survey we’re developing, we are considering a question that
asks the respondent to rank the relative importance of each of these drivers in
motivating improvement efforts (from most important to least important).
Would you find this possible to do?

17. Are there other drivers of improvement that are important in your home health
agency? [Possible prompts: risk reduction? corporate pressure or policy?
accreditation? working to stay competitive?]
18. Has your agency initiated major system changes to policy and/or processes to
expand staff ability to provide quality care and perform well on quality
measures? Please give some examples. [Possible prompts: own internal
incentive program, provider feedback reports, care coordination innovations,
enhanced information technology, attempts to improve documentation of
existing care.] [If training is mentioned, probe if focused on quality measures.]

19. [If respondent had difficulty understanding what we meant by “major system
changes to policy and/or processes:”] I just used the phrase “system changes to
policy and/or processes.” What does that make you think of? Would you suggest
we use a different term in the survey?
20. Has individual staff made any changes in response to these measures? If yes, tell
us what they’ve done.

18. Thinking back over the different changes we’ve talked about, do you believe
these have impacted your agency’s performance on the CMS quality measures? If
so, in what ways? Please elaborate.

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

19. Which efforts in particular have been associated with changes in performance
over time?

Unintended Consequences [E5]

We’ll turn now to some questions on other consequences of CMS measurement programs.
20. Have you or your organization seen any unintended consequences—either
negative or positive—resulting from quality reporting? Please describe.
21. If so, are they related to certain measures in particular? Which?

22. We’ve heard concerns voiced about possible unintended consequences of the CMS
measurement programs. I’m going to mention five that have been raised and ask if
you’ve experienced them in any way. They include: over-treatment of patients to
ensure that a metric is met; improvements in areas other than those captured by the
quality measures; lack of improvements in areas not measured; coding modifications in
order to score better; and avoidance of sicker patients in order to achieve higher scores.
I will go through each of these in turn.
a. We’ve heard concerns that measurement programs may create a potential for overtreatment of patients—say, for example, if measuring improvement in pain scores
leads to over-use of scheduled narcotics in some patients. Do you think this
happens? With any specific measures, in particular? Do you have any examples?

b. We’ve heard some reports that improved performance on some measures has at
times spilled over to generate improvements in other clinical areas that are not part
of what is measured or financially incentivized by Medicare or other payers—
resulting in quality improvement across the board. Do you think this happens at
[agency name]? Would you give us an example? [Example, if needed: A focus on
pain management might lead to improved performance on mobility and ADL
measures.]
c. On the other hand, home health agencies might focus all their improvement efforts
on areas of care where performance is being measured and ignore or pay less
attention to areas of care that are not measured. Do you think this happens? [If
yes:] How does this happen? Does it happen with any specific measures in
particular?

d. [If not mentioned above:] Do you think people have modified their coding or
reporting of the data to score better on quality measures? [If yes:] For any specific
measures in particular? For example, a functional outcome score (e.g., got better at
walking) by underestimating a patient’s baseline score during the “start of care”

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

assessment, or by overestimating a patient’s score at the “end of care assessment.”
Which (if either) is more likely?

e. Have you heard of home health agencies avoiding sicker or more challenging
patients when providing care in order to achieve higher scores on quality measures?
[If yes:] Would you give us an example of the kind of scenario you’ve heard of? (You
don’t need to mention any names.)

23. [If did not identify any general unintended consequences before the five prompts:] Does
this list make you think of any other unintended consequences that we have not yet
discussed? If so, what are they? Can you provide examples?

24. [If some unintended consequences have been mentioned:] Why do you think these
unintended consequences have occurred? [Possible prompts: poor measure design, large
financial incentives, difficult patients, other.]
25. Are they related to certain measures in particular? If so, which?

Challenges and Facilitators to Implementation [I1]

We’d now like to talk about two types of challenges or difficulties that might arise—first,
challenges around the reporting of data and, second, challenges to improving performance
on quality measures.
26. Have you encountered any major challenges or difficulties to reporting [agency
name]’s performance on the Home Health Compare quality measures? Please
describe. [Prompts: Inadequate IT capabilities, provider training, difficulty
capturing / reporting OASIS data, the measure specification, insufficient
resources]
27. With any measures or OASIS element in particular?

28. Have you experienced any major challenges or difficulties to improving [agency
name]’s performance on CMS quality measures? Please describe. [Prompts:
Difficulty identifying appropriate improvement strategies, difficulty identifying
the appropriate process measures that lead to the outcome measures reported,
provider training, insufficient resources, inadequate IT capabilities, staff
turnover, lack of sufficient support or time from physicians or other staff.]
29. With any measures in particular?

We’d now like you to think about facilitators to reporting the data and to improving
performance on quality measures—things that make doing it easier or more effective.
Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

30. What has helped your agency do well at reporting Home Health Compare quality
data?
31. What has helped your agency do well at improving performance on CMS quality
measures?

Home Health Reporting Background

32. Has [agency name] taken part in a Pay for Performance (P4P) program or
demonstration?

33. [If yes:] Is it ongoing? Who administers(ed) it? What measures does it focus on?
34. Did [agency name] participate in the Home Health Quality Initiative (HHQI) or
any similar initiatives?
35. Has [agency name] participated in any other quality measure reporting
programs? Please specify. [Possible prompt: state programs?]

36. [If so:] Have some of the various quality measure reporting programs had
greater effect on the quality and efficiency of care at [agency name], than others?
Which ones? Why do you think this is the case?

Identification of Survey Respondents

37. As we mentioned at the beginning of the interview, we plan to conduct a large
national survey of home health agencies on their experiences with CMS quality
measures. In an organization such as yours, who would you say is the most
appropriate person to direct it to? [Prompt: home health administrator, quality
director or manager, director of nursing?] Would more than one person need to
provide the information to fully complete a survey?

Lessons Learned

38. What have been the most important lessons learned to date from participating in
the CMS Home Health Quality Initiative or Home Health Compare programs?

39. Have these lessons led to any changes in the way things are done at [agency
name]?

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

40. Do you have any experiences or concerns around CMS measurement programs
that we haven't discussed that you would like to raise?

41. Based on your experience to date using CMS home health agency measures, what
changes to the measures or the reporting program would you recommend? Any
changes you’d really like to see?

Additional Questions if Time Allows
Re: Barriers to Implementation:

43. What actions have been taken to address or reduce the barriers you mentioned
around reporting data or improving performance on quality measures?

Re Unintended Consequences:

44. [If no to Q15—have not encountered any negative or positive consequences:]
Have you had concerns that some negative consequences might occur? [If yes:]
What concerns have you had?
45. Have other leaders or staff at [agency name] raised concerns about possible
negative consequences of these quality measures? Would you describe these
concerns?
46. Has [agency name] modified any reporting procedures in response to
unintended consequences?

Re Innovations in Delivery of Care:

47. How do you think your staff understands the CMS measure program and how it
works?
48. Would you tell us a little about how these changes to improve the delivery of
care are initiated and undertaken?

49. Does [agency name] have a quality performance improvement committee? If so,
what role does the committee place with respect to CMS quality measures?

50. [If no:] Has a specific individual been designated to work on quality issues? If so,
what position or individual has been designated?
51. Has [agency name] hired an outside consultant to help improve clinical care or
resident assessments? If so, what prompted this? Please give a brief description
of these change efforts.

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

52. Who makes sure that changes are implemented?

53. Has [agency name] initiated any changes to improve care transitions? To reduce
psychotropic med use? If so, please describe. What prompted this focus?
54. Do changes to improve the quality of care usually address the work of one type
of provider, say nurses or certified nursing assistants? Or, are they usually
interdisciplinary efforts?

55. Are they typically rolled out team by team or across the whole agency all at one
time?
56. Do patients or families mention your quality scores? In what situations?

57. [If yes on Q13—they have seen some QI efforts reflected in their nursing home’s
performance on quality measures over time:] How do you let others know about
this improvement?
58. Have you participated in any way in the development and selection of quality
measures (e.g., through your professional association or through providing
public comments)? If so, please describe.

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

Appendix B: Cognitive Interview Guide for Home Health Agencies

Respondent Type
Organization Name:
Respondent Name:
Respondent Position:
Interviewer Name:
Interview Date:

INTRODUCTION AND PURPOSE OF THE INTERVIEW
Thank you for agreeing to participate in this interview today.
Before we get started, I’d like to briefly review the purpose of this interview and
confidentiality.

 To assess the impact of the measurement programs, CMS plans to conduct a survey
of home health agencies’ leadership to understand how agencies have responded to
CMS quality measurement programs. The survey will also identify any challenges in
reporting measurement data or in improving performance on quality measures, as
well as undesired consequences that may have occurred in response to CMS quality
measures. CMS has tasked the RAND Corporation with designing the survey, and we
are conducting a small number of interviews with home health agencies to
understand how we can best construct the survey.
 Recently we sent you a survey asking about your organization’s experiences with
CMS quality measures. Thank you for taking the time to fill out that survey. Today I
am going to be asking you questions about the survey to make sure that the
questions on the survey are clear and capture your organization’s experience in
reporting CMS measures. Your feedback will be used to refine and improve the
survey. Please have the survey with you as we conduct the interview.
 The interview today should take about an hour. During the interview, I will be
taking notes and with your permission, would also like to record the interview.

 To thank you for taking the time to participate in the interview, we will be sending
you a [check/gift card] for $300.

 I would like to ask you some specific follow up questions throughout the survey but
first we’d like to review the confidentiality provisions for this interview.

CONSENT

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

→ All of your responses are confidential.
→ No one outside of the research project will have direct access to the information you
provide. The evaluation team will produce only summary information from our
collective set of interviews.
→ You will not be identified by name or organizational affiliation in the summary report
produced from these interviews. We also will not identify by name the organizations
that are represented in the interviews.
→ You do not have to participate in the interview, and you can stop at any time for any
reason.
→ Feel free to decline to discuss any topic that I raise in the course of the interview.
→ If there is a particular question you don’t want to answer, just let me know and we’ll
skip to the next one.
→ After the study is completed, we will destroy the interview notes and the recording of
the interview.
→ If you have any questions or concerns about this project, please contact Cheryl
Damberg, Principal Investigator, at damberg@rand.org, 310-393-0411, x6191.
→ If you have any questions about your rights as a research subject, please contact the
RAND Human Subjects Protection Committee at (310) 393-0411, ext. 7173, and ask to
speak to Jim Tebow.
Do you have any questions? (Yes/No)

Do you agree to participate in the interview? (Yes/No)
As we mentioned in our email, we would like to tape the interview if that is all right with
you.
Do you agree to be tape-recorded? (Yes/No)

If yes: Great. Let’s get started. I’ll start the recording.

If no: That’s fine. We will take notes—and not tape the discussion. Let’s get started.
Start time:

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

YOUR HOME HEALTH AGENCY’S EXPERIENCE WITH CMS MEASURES
1.

How would you describe your home health agency’s performance on CMS quality
measures in 2018 compared to 2017?
[Please check one]
1
2
3
4
5

•

Tell me more about your response to this question?

•

IF MORE MEASURES IMPROVED THAN DECLINED: Which measures did you improve
on? Are there any measures where your performance declined since 2017?
In your opinion, how well does your home health agency’s performance on the CMS
quality measures reflect the improvements in care that your home health agency makes?
1
2
3

2
3
4

Yes
Mostly yes
Mostly no
No

Do you think home health agencies should be held responsible for performance on the
CMS quality measures?
1
2
3
4

Very well
Somewhat well
Not well at all

Thinking about the full list of CMS home health measures, do you think the CMS quality
measures are clinically important?
1

Improved across the board on all measures
More measures improved than declined
Most measures stayed about the same
More measures declined than improved
Declined across the board on all measures

Yes
Mostly yes
Mostly no
No

Have you experienced difficulties with improving performance on any of the CMS quality
measures?
1
2
3

Yes on many of the measures
Yes on some of the measures
No [GO TO QUESTION 8]

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

•

Did you have any difficulty answering these questions? IF YES: which ones?

•

Why did you think CMS quality measures are clinically important (or why not)? (Q3)

•

Tell me why you answered [response]? (Q4 and Q5)

6. Based on your home health agency’s experience, how difficult has it been for your home
health agency to improve on the following types of measures?

Clinical process measures (for
example: How often the home
health team made sure that
patients have received a flu shot
for the current flu season?)
Patient outcome measures (for
example: How often did patients
get better at walking or moving
around?)
Patient experience measures
(for example: Home Health
CAHPS Survey measure “How
often the home health team gave
care in a professional way”)
Patient safety measures (for
example: How often the home
health team checked patients’ risk
of falling)
Other- please specify_______

Not
Difficult

Slightly
Difficult

Moderately
Difficult

Difficult

Very
Difficult

--//

•

Did you have any difficulty answering this question? If YES: which item?

•

Tell about the scale on this question, how did you rate your level of difficulty? How
easy of difficult was to use this scale?

•

In your own words, what does “Clinical process measures (for example: How often the
home health team made sure that their patients have received a flu shot for the
current flu season)” mean to you?

•

What about Patient safety measures (for example: How often the home health team
checked patients’ risk of falling)? Can you define it for me?

•

Are the examples provided helpful? What other examples should we provide?

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

7.
Have any of the following contributed to your home health agency’s difficulties with
improving performance on the CMS measures?
[Circle yes or no on each row]
a. Difficulty identifying improvement strategies .................................... Yes

b. Difficulty implementing improvement strategies ............................... Yes

c. Difficulty identifying processes of care that lead to
improved patient outcomes .............................................................. Yes

d. Difficulty getting frontline staff to change behavior to improve
performance .................................................................................... Yes

e. Insufficient staffing to implement quality improvement strategies ..... Yes

Inadequate Health Information Technology (IT) capabilities
(e.g., clinical decision support or longitudinal tracking of outcomes,
or electronic medication administration system [eMAR]) ……………. Yes

g. Staff turnover ................................................................................... Yes

h. Lack of senior leadership support .................................................... Yes

Difficulty with coding or documentation (e.g., inconsistent or
insufficient documentation by staff) .................................................. Yes

Lack of training on improvement processes ..................................... Yes

k. A difficult patient mix (i.e., low socioeconomic status,
clinically complex) ............................................................................ Yes

Your home health agency’s organizational culture not supporting
improvement efforts ......................................................................... Yes

m. Inability to retrieve timely data from CMS or data from other
providers such as hospitals............................................................... Yes

n. Other reason [Please specify] ........................................................... Yes

•

Are there any items in this question that you don’t think apply to your home health
agency? IF YES: Which ones?

•

Are there any items on this question that are difficult to understand? Which ones?

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

•

Are there any additional factors that contributed to your agency’s difficulties with
improving performance on CMS measures?

INNOVATIONS IN THE DELIVERY OF CARE
8.

We are interested in understanding what changes your home health agency has made in
the way care is being delivered to improve its quality performance.

Type of Change or Innovation

Has your home
health agency
Implemented this
change?

Organizational Culture
a. Adopted practices to
☐ Yes →
become a “learning
☐ No ↓
organization” that
encourages and supports
continuous employee
learning, critical thinking,
and risk-taking with new
ideas.
b. Implemented a “culture of ☐ Yes →
safety” characterized by
☐ No ↓
communications founded
on mutual trust, by
shared perceptions of the
importance of safety, and
by confidence in the
efficacy of preventive
measures.
Health Information Technology
c. Implemented an
☐ Yes →
electronic health record
☐ No
(EHR).

d. Implemented electronic
tools to support frontline
clinical staff, such as
clinical decision support
(CDS), or medication
administration system
(MAR).
e. Implemented systems for
electronically exchanging
clinical information with
providers in the
community (e.g.,
hospitals and ambulatory
care providers).

Was this change
implemented to improve
performance on CMS
quality measures?

Did the change help
performance on CMS
quality measures?

☐ Yes, mostly →
☐ Yes, partly →
☐ No ↓

☐ Yes, definitely
☐ Yes, somewhat
☐ No
☐ Don’t know/
Not sure

☐ Yes, mostly →
☐ Yes, partly →
☐ No ↓

☐ Yes, definitely
☐ Yes, somewhat
☐ No
☐ Don’t know/
Not sure

☐ Yes, mostly →
☐ Yes, partly →
☐ No 

☐ Yes, definitely
☐ Yes, somewhat
☐ No
☐ Don’t know/
Not sure
☐ Yes, definitely
☐ Yes, somewhat
☐ No
☐ Don’t know/
Not sure

☐ Yes →
☐ No ↓

☐ Yes, mostly →
☐ Yes, partly →
☐ No ↓

☐ Yes →
☐ No ↓

☐ Yes, mostly →
☐ Yes, partly →
☐ No ↓

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

☐ Yes, definitely
☐ Yes, somewhat
☐ No
☐ Don’t know/
Not sure

Development of National Provider Survey of Home Health Agencies

Has your home
health agency
Implemented this
change?

Was this change
implemented to improve
performance on CMS
quality measures?

Did the change help
performance on CMS
quality measures?

☐ Yes →
☐ No ↓

☐ Yes, mostly →
☐ Yes, partly →
☐ No ↓

g. Implemented
standardized care
protocols or checklists.

☐ Yes →
☐ No ↓

☐ Yes, mostly →
☐ Yes, partly →
☐ No ↓

h. Implemented
telemonitoring or remote
patient monitoring

☐ Yes →
☐ No ↓

☐ Yes, mostly →
☐ Yes, partly →
☐ No ↓

i. Adopted care redesign/reengineering (e.g., Lean
Engineering, Six Sigma,
Plan, Do, Study, Act
improvement cycles).

☐ Yes →
☐ No ↓

☐ Yes, mostly →
☐ Yes, partly →
☐ No ↓

j. Implemented
interdisciplinary rounds,
case conferences, or
multi-specialty patient
care teams.

☐ Yes →
☐ No ↓

☐ Yes, mostly →
☐ Yes, partly →
☐ No ↓

☐ Yes, definitely
☐ Yes, somewhat
☐ No
☐ Don’t know/
Not sure
☐ Yes, definitely
☐ Yes, somewhat
☐ No
☐ Don’t know/
Not sure
☐ Yes, definitely
☐ Yes, somewhat
☐ No
☐ Don’t know/
Not sure
☐ Yes, definitely
☐ Yes, somewhat
☐ No
☐ Don’t know/
Not sure
☐ Yes, definitely
☐ Yes, somewhat
☐ No
☐ Don’t know/
Not sure

k. Implemented or changed
communication protocols
to support or improve
collaboration between
referring providers and
agency staff
l. Increased coordination
with hospitals, nursing
homes, and other
providers to improve care
transitions and reduce
hospitalization rates.

☐ Yes →
☐ No ↓

☐ Yes, mostly →
☐ Yes, partly →
☐ No ↓

☐ Yes →
☐ No ↓

☐ Yes, mostly →
☐ Yes, partly →
☐ No ↓

Type of Change or Innovation
Care Process Redesign
f. Implemented risk
prediction tools to identify
and manage high-risk
patients.

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

☐ Yes, definitely
☐ Yes, somewhat
☐ No
☐ Don’t know/
Not sure

Development of National Provider Survey of Home Health Agencies

Type of Change or Innovation
m. Increased number of
visits at beginning of care
episode (i.e.,
“frontloading”) so that
patients have greater
contact with clinicians
earlier in care episode.
n. Addition of after-hours
on-call availability to
patients.

Has your home
health agency
Implemented this
change?
☐ Yes →
☐ No ↓

Was this change
implemented to improve
performance on CMS
quality measures?
☐ Yes, mostly →
☐ Yes, partly →
☐ No ↓

Did the change help
performance on CMS
quality measures?

☐ Yes →
☐ No ↓

☐ Yes, mostly →
☐ Yes, partly →
☐ No ↓

☐ Yes, definitely
☐ Yes, somewhat
☐ No
☐ Don’t know/
Not sure

☐ Yes, mostly →
☐ Yes, partly →
☐ No ↓

☐ Yes, definitely
☐ Yes, somewhat
☐ No
☐ Don’t know/
Not sure
☐ Yes, definitely
☐ Yes, somewhat
☐ No
☐ Don’t know/
Not sure

Feedback and Monitoring of Performance
o. Developed a system for
☐ Yes →
tracking patient
☐ No ↓
outcomes.

p. Provided routine
☐ Yes →
feedback on your home
☐ No ↓
health agency’s
performance on CMS
measures to nurses,
physical therapists, and
other staff.
Changing Provider Incentives
q. Used performance on
☐ Yes →
CMS measures as a
☐ No ↓
basis for determining pay
for nurses or other
frontline staff.

☐ Yes, mostly →
☐ Yes, partly →
☐ No ↓

☐ Yes →
☐ No ↓

☐ Yes, mostly →
☐ Yes, partly →
☐ No ↓

☐ Yes →
☐ No ↓

☐ Yes, mostly →
☐ Yes, partly →
☐ No ↓

r. Implemented an internal
incentive or bonus
program for senior
management based on
performance on CMS
measures.
s. Gave staff awards or
other special recognition
tied to quality
performance.

☐ Yes, mostly →
☐ Yes, partly →
☐ No ↓

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

☐ Yes, definitely
☐ Yes, somewhat
☐ No
☐ Don’t know/
Not sure

☐ Yes, definitely
☐ Yes, somewhat
☐ No
☐ Don’t know/
Not sure
☐ Yes, definitely
☐ Yes, somewhat
☐ No
☐ Don’t know/
Not sure
☐ Yes, definitely
☐ Yes, somewhat
☐ No
☐ Don’t know/
Not sure

Development of National Provider Survey of Home Health Agencies

Has your home
health agency
Implemented this
change?

Was this change
implemented to improve
performance on CMS
quality measures?

Did the change help
performance on CMS
quality measures?

☐ Yes →
☐ No ↓

☐ Yes, mostly →
☐ Yes, partly →
☐ No ↓

u. Identified champions for
quality improvement
initiatives or projects
among clinical staff.

☐ Yes →
☐ No ↓

☐ Yes, mostly →
☐ Yes, partly →
☐ No ↓

v. Implemented changes to
how clinical staff are
deployed (e.g., change in
staffing levels or clinical
roles/responsibilities).

☐ Yes →
☐ No ↓

☐ Yes, mostly →
☐ Yes, partly →
☐ No ↓

Type of Change or Innovation
Changes in Staffing
t. Increased the number of
staff dedicated to quality
improvement or quality
management.

Obtained Technical Assistance
w. Obtained technical
☐ Yes →
assistance from CMS
☐ No ↓
(i.e., via a CMS Quality
Improvement
Organization or the CMS
Home Health Quality
Improvement initiative) to
collect and report CMS
quality measures.
x. Obtained technical
☐ Yes →
assistance from private
☐ No ↓
organizations (e.g.,
quality improvement
collaboratives, consulting
firms).
Provider Education and Training
y. Implemented quality
☐ Yes →
improvement initiatives
☐ No ↓
targeted to specific CMS
measures.

☐ Yes, mostly →
☐ Yes, partly →
☐ No ↓

☐ Yes, definitely
☐ Yes, somewhat
☐ No
☐ Don’t know/
Not sure

☐ Yes, mostly →
☐ Yes, partly →
☐ No ↓

☐ Yes, definitely
☐ Yes, somewhat
☐ No
☐ Don’t know/
Not sure

☐ Yes, mostly →
☐ Yes, partly →
☐ No ↓

☐ Yes, definitely
☐ Yes, somewhat
☐ No
☐ Don’t know/
Not sure

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

Type of Change or Innovation
z. Provided training to
nurses, physical
therapists and other
clinical staff on quality
improvement strategies.
a1. Provided training to
clinical staff on teaching
patient self-management
techniques.
Other Improvements
b1. Other change or
innovation. (please
specify:_____________)

Has your home
health agency
Implemented this
change?
☐ Yes →
☐ No ↓

Was this change
implemented to improve
performance on CMS
quality measures?
☐ Yes, mostly →
☐ Yes, partly →
☐ No ↓

☐ Yes →
☐ No ↓

☐ Yes, mostly →
☐ Yes, partly →
☐ No ↓

☐ Yes →
☐ No

☐ Yes, mostly →
☐ Yes, partly →
☐ No

Did the change help
performance on CMS
quality measures?
☐ Yes, definitely
☐ Yes, somewhat
☐ No
☐ Don’t know/
Not sure
☐ Yes, definitely
☐ Yes, somewhat
☐ No
☐ Don’t know/
Not sure
☐ Yes, definitely
☐ Yes, somewhat
☐ No
☐ Don’t know/
Not sure

ISSUES FOR COGNITIVE TESTING: Do you or other agencies employ “multi-specialty patient
care teams”? Are there other important QI changes that we should ask agencies about?
•

Overall, what did you think of this question? Did you have any difficulty answering this
question?

•

IF YES: Tell me about that? What made it difficult? (Probe: wording of items/answer
options, format of questions, logic of skips, etc.)

•

What does ““multi-specialty patient care teams” mean to you?

•

Are these terms used among home health agencies? IF NO, what term/s is used
instead?

•

Are there any other actions or changes that we should include? Which one/s?

•

How clear are the question headings? How can we make it clearer?

•

What was your time frame you used when answering this question? (Probe: Last 12
months? 3 years? 5 years?)

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

FACTORS ASSOCIATED WITH CHANGE IN QUALITY PERFORMANCE
9. There are many factors that influence a home health agency’s decision to invest in efforts
to improve its quality performance.
Please rank the importance of the following six external factors in your home health
agency’s decision to invest in quality improvement efforts for CMS measures.
(Please rank by order of importance where 1 is the most important and 6 is the least
important. Do not use the same rank number more than once)
_____ a. Potential to receive financial incentives for improved performance (i.e., pay for
performance)
_____ b. Risk of financial penalties for low performance (e.g., non-payment for home health
agency readmissions within 30 days or for home health agency-acquired infections)
_____ c. Public reporting of your home health agency’s performance results on the CMS
Home Health Compare website
_____ d. Participation in alternative payment models (e.g., ACOs, bundled payment
arrangements) or managed care contracts where there is an opportunity for shared
reward (savings) and shared financial risk
_____ e. State or federal regulatory requirements regarding certification/accreditation
_____ f. Addition of Quality Assessment and Performance Improvement (QAPI)
requirements to conditions of participation

•

Did you have any difficulty answering this question? IF YES: Tell me about that?

•

How did you come about ranking these factors? Tell me more about that?

•

Why did you choose factor (most important picked) as the most important? Tell me
about that.
Why did you choose factor (least important picked) as the least important? Tell me
about that.

•
•

Are there any other factors that have influenced your decision to invest in trying to
improve your home health agency’s performance on CMS measures?

10. Has your home health agency improved its performance on any of the CMS measures?
1
2

Yes
No [GO TO QUESTION 12]

10a. Many different factors may help a home health agency improve its performance. How
important are the factors below in helping your agency improve performance on CMS
measures?
Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

Not
Important

--a. Your home health
agency’s organizational
culture

Slightly
Important

Moderately
Important

Important

Very
Important

Not
Applicable
-

---

d. Having strong data
systems

e. Having a system-wide
focus on quality and
quality improvement

----

b. Effective relationship
between management
and staff
c. Internal accountability
for performance on CMS
measures

f. Networking with other
home health agencies
and health systems to
identify elements of highperforming organizations
g. Investments in patient
safety
h. Focus on improved
documentation
i. Other (please specify)

ISSUES FOR COGNITIVE TESTING: Testing Likert scale for different responses to test relative
importance of each factor. If there are no differences between agency weights on each factor,
could you probe further to determine any way to ask the question to get some kind of
ranking?
•

How easy or difficult was to answer this question using a 5-point scale to answer each
item?

•

How did you assign a level of importance to each of the factors?

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

•

Are there other formats (e.g., ranking) that would make it easier to answer this
question?

•

Which of the factors listed in this question has been the most important in improving
your performance on CMS performance measures?

•

Which has been the least important?

CHALLENGES TO REPORTING THE CMS MEASURES
11. Has your agency experienced any of the following challenges in transmitting OASIS data
(for CMS measures)?
[Please check all that apply]
1

2
3
4
5
6
7

Difficulty extracting the data from the EHR or other data systems/registries for
OASIS
Difficulty interpreting measure specifications
Frequency of OASIS version changes
Insufficient or inadequate staffing or other resources
Challenges with interface for transmitting OASIS data
Other reason (Please specify:_______________________________________)
Has not experienced any difficulties

•

What do you think this question is asking?

•

Is the question clear to you?

•

Are there any other difficulties in reporting to CMS we should include? Which ones?

UNDESIRED EFFECTS OF CMS QUALITY MEASUREMENT PROGRAMS
12. Has your home health agency observed any undesired effects stemming from using or
reporting CMS measures?
1
2
3

Yes, definitely
Yes, somewhat
No

13. In your opinion, do you think any of the following has occurred in your home health agency
as a result of your home health agency being held accountable for performance on CMS
measures?
[Circle yes or no on each row]

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

a. Fewer resources for quality improvement in areas of clinical
care that are not the focus of CMS performance measures ............. Yes

b. Focus on narrow improvement for specific measures rather
than across the board improvement in care ..................................... Yes

c. Overtreatment of patients to ensure that a measure is met .............. Yes

d. Increased focus on documentation or coding of data to attain
a higher score .................................................................................. Yes

e. Changing coding of data or documentation to ensure that
a measure is met ............................................................................. Yes

Avoiding sicker or more challenging patients when
providing care .................................................................................. Yes

•

In your own words, what are undesired effects?

•

What concerns do you have about answering questions on negative effects of CMS
performance measures such as those mentioned in items b, d, and f?

•

If you got this survey, would you answer this type of question? Why not?

•

Do you think that other home health agencies are likely to report in the survey that
some of these negative effects are happening within their organization?

14. Have the changes your home health agency has made in response to the CMS measures
resulted in broader improvements in areas of care beyond what is measured by the CMS
quality measures?
1
2
3

Yes
No [GO TO QUESTION 16]
Don’t know [GO TO QUESTION 16]

15. Has your home health agency measured or documented the actual improvements in the
areas of care not measured by CMS?
1
2

Yes
No

ISSUES FOR COGNITIVE TESTING: If respondents say yes: What areas of care do they have
in mind? (We need to ascertain if they are thinking of areas of care that are truly not
addressed by CMS. If they are only thinking of areas covered by CMS measures, the
question might not be useful. This was the case last time.)
•

In your own words, what do you think Q14 is asking?

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

•

What do you understand for broader improvements in areas of care beyond what is
measured by the CMS quality measures?

•

Question 15, What would you include in areas of care not measured by CMS?

PERSPECTIVES OF YOUR HOME HEALTH AGENCY’S LEADERSHIP AND OTHER
STAKEHOLDERS
16. Does your home health agency have a board of directors?
1
2

Yes
No [GO TO QUESTION 20]

17. How often do meetings of your home health agency’s board of directors include a review
and discussion of the home health agency’s performance on the CMS measures?
1
2
3
4
5

More than four times per year
Quarterly
Twice per year
Annually
Less than once per year

18. Which of the following best describes your home health agency’s board of directors?
1

Board is more engaged in financial performance issues than quality performance
issues.

Board is equally engaged in financial performance issues and quality performance
issues.

Board is more engaged in quality performance issues than financial performance
issues.

•

Did you have any difficulty answering these questions? IF YES: Tell me about that?

•

When you answer about your board of directors (q 17), is this the board for your
agency or part of a larger system?

•
•

Tell me about your boards engagement or interest in the CMS performance measures
For question 18, how did you come up with your answer?

19. On a scale from 0 to 10, where 0 is not at all supportive and 10 is extremely supportive,
how would you describe your home health agency’s board of directors’ support of your
home health agency’s efforts to improve performance on CMS measures?
Supportive scale 0–10
Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

20. On a scale from 0 to 10, where 0 is not at all supportive and 10 is extremely supportive,
how would you describe the home health agency leadership’s (e.g., the C-Suite executive
management) support of your home health agency’s efforts to improve performance on
CMS measures?
Supportive scale 0–10
21. On a scale from 0 to 10, where 0 is not at all supportive and 10 is extremely supportive,
how would you describe the clinical staff’s support of your home health agency’s efforts to
improve performance on CMS measures?
Supportive scale 0–10
22. On a scale from 0 to 10, where 0 is not at all and 10 is a great deal, how much does your
home health agency leadership promote a culture of quality?
0 – Not at all – 10 a great deal scale

ISSUES FOR COGNITIVE TESTING: Should we have a screener question that asks about
whether there are any leadership levels above them? If agency is 1-2 persons they might not
answer to anyone above them or have anyone below them to supervise them. In that case, for
questions 20 and 21, we should have a screener question asking if the respondent reports to
anyone (CMO, owner, more senior leadership, CEO, a leadership team, etc.) If no, then skip 20
and 21. For 21, we might add a screener question on whether any staff report to them.
Otherwise, asking about clinical staff doesn’t make sense.
•

Tell me a little bit about your agency composition—what is the size of your agency? IF
agency has 1-2 people or respondent is the owner, does your agency have a leadership
structure or leadership team?

•

What does home health agency’s leadership mean to you? Would you include yourself
in that group? Who would you include in that group?

•

Q21, what do you think of this question? Do you think is relevant to your agency? Why
not?

•

How did you pick a number?

USE OF HEALTH INFORMATION TECHNOLOGY
23. Does your home health agency have an electronic health record (EHR)?
1
2

Yes
No [GO TO QUESTION 29]

24. Is your home health agency able to receive physician orders and feedback on care using its
EHR?
Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies
1
2

Yes
No

25. Are health providers in your community (i.e., ambulatory care physicians, hospitals) able to
access your home health agency’s EHR or health information system to obtain key clinical
data on patients?
1
2
3

Yes, all key clinical data
Yes, some key clinical data
No [GO TO QUESTION 27]

•

Tell me how your EHR system works with physician orders.

•

What does “health providers in your community” mean to you? Who are they?

•

Would you say your EHR helps or hinders reporting of quality measures?

26. Which of the following types of information are health providers in your community (i.e.,
ambulatory care physicians, hospitals) able to access electronically through your home
health agency’s EHR or health information system?
[Please circle each item]
a. Diagnostic/treatment summary

Yes, All..

Yes, Some

b. Discharge instructions

Yes, All

Yes, Some

c. Lab tests/Imaging results

Yes, All

Yes, Some

d. Prescribed medications

Yes, All

Yes, Some

27. Is your home health agency able to electronically access information on your patients from
other providers in your community (i.e., ambulatory care physicians, hospitals)?
1
2
3

Yes, for all or most patients
Yes, for some patients
No

ISSUES FOR COGNITIVE TESTING: Are these types of data likely to be produced by agencies?
•

What do you think Q27 is getting at?

•

Is your agency able to access X (responses from Q26)? What other types of data is your
agency able to access electronically?

•

If NO to 26 a-d, do you know if other agencies are able to provide such data to health
providers in the community?

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

•

What other types of data is your agency able to produce electronically?

28. Does your home health agency’s EHR have an interface or other tools that help with …
[Circle yes or no on each row]
a. Medication tracking and reconciliation?

Yes

b. Evidence-based treatment or clinical decision support?

Yes

d. Software prompts or validation to improve OASIS accuracy

Yes

e. Reporting of CMS measures?

Yes

and/or patient outcomes?

Yes

g. Administration of medication?

Yes

c. Collection of data for CMS measures
(including OASIS “scrubbing” programs)?

f. Tracking or monitoring of quality of care

29. Not including an EHR, does your home health agency use any other software or electronic
tools that help with….
a. Collection of data for OASIS (including “scrubbing” programs for OASIS data)?
.............................................................................................. Yes

b. Reporting of CMS measures? ............................................... Yes

ISSUES FOR COGNITIVE TESTING: Does respondent understand the term “scrubber” or
“scrubbing program” – it is supposed to refer to software that automatically extract data for
quality measurement.
•

What do you understand by the term “scrubbing programs”? (software that
automatically extract data for quality measurement)

•

What other terminology your agency uses for automatic data extraction?

•

What other types of software or electronic tools your agency uses?

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

CHARACTERISTICS OF YOUR HOME HEALTH AGENCY

30. Is your home health agency freestanding (and not owned by or affiliated with a larger
system/chain, hospital, or integrated delivery system)?
1
2

Yes, freestanding [GO TO QUESTION 34]
No, owned by or affiliated with a larger entity

31. Is your home health agency affiliated with or owned by a home health agency system or
chain?
1
Yes
2
No
32. Is your home health agency owned by a hospital?
1
Yes
2
No
33. Is your home health agency part of an integrated delivery system?
1
Yes
2
No
34. Do you face a shortage of nurses, physical therapists, or other frontline clinicians in your
area?
1
2

Yes
No

•

How easy or difficult was to answer these questions? Tell me about that.

•

What does “home health agency system or chain” mean to you?

•

Do you refer to it as “system,” “chain” or something else?

•

What is the most commonly used term in your area?

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

35. Does your home health agency participate in any of the following types of accountable care
organizations (ACOs)?
[Circle yes or no on each row]
a. Medicare Shared Savings Program

Yes

b. Medicare Pioneer ACO .......................................................... Yes

c. Medicare’s Advanced Payment Model ACO ........................... Yes

d. Medicare’s Next Generation ACO Model ................................ Yes

e. Medicaid ACO ........................................................................ Yes

f. A private, commercially insured ACO arrangement
(within an HMO or PPO) ....................................................... Yes

36. Is your home health agency participating in any other type of alternative payment model
that may have shared savings or shared risk (e.g., global budgets, bundled payments for
selected procedures)?
1
2

Yes
No

ISSUES FOR COGNITIVE TESTING: Do agencies understand the examples given? These refer to
payment models in which a fixed payment is given for a group of services (care after knee
replacement, for example) rather than on a fee-for-service basis in which agencies receive
payment for each visit.
•

What other alternative payment models exist? What are other payment models?

•

Q35, Tell me about the ACOs your home health agency participates in?

•

Do you think these questions relevant for home health agencies (Q35)? Why?

•

Q36, how would you describe a savings or shared risk payment models?

•

What do you understand by global budgets, bundled payments for selected
procedures?

•

Are the examples provided for shared savings or shared risk helpful?

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

RESPONDENT BACKGROUND
37. Which of the following best describes your job title or position within this home health
agency?
1

Chief Executive Officer

Administrator

Director of Nursing

Senior leader responsible for quality of clinical care (e.g., VP for Quality)

Clinical Manager

Member of a team responsible for measuring and reporting quality of clinical care

Some other role (please specify): ____________________________

ISSUES FOR COGNITIVE TESTING: Have respondents heard of other titles that are frequently
used by quality leaders at home health agencies? (This aims to get at other titles we should
ask about during contact identification.)
•

Do the titles on this question seem appropriate for your home health agency? Why
Not?

•

Do some of the titles overlap? Which ones?

•

What does administrator mean to you?

•

What other titles are not listed on Q37? What other titles should be included?

38. Has your home health agency quality team received formal training/certification on quality
improvement strategies?
1
2

Yes, indicate strategy and certification:____________________
No

•

What does this mean to you? What would you include as “formal training or
certification on quality improvement strategies?

•

Have you personally received any training in quality improvement strategies?

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

SURVEY PROCESS QUESTIONS

Are there any other issues related to the implementation of the CMS Quality
Measures that were not covered in this survey? If yes, briefly describe them below.
1

Other issues:

How familiar are you with the CMS Performance Measures?

1
2
3

2
3
4

Very familiar
Familiar
Not very familiar
Not at all familiar

After completing the survey, do you feel that you are the most appropriate person to
complete the survey?
1

Very familiar
Familiar
Not very familiar
Not at all familiar

How familiar are you with the impact (positive or negative) the CMS Performance
Measures has had on the quality of care your home health agency delivers?
1

Very familiar
Familiar
Not very familiar
Not at all familiar

How familiar are you with the steps your organization has taken to implement the
CMS Performance Measures?
1

Yes
No If “No”, go to question 3

Yes If “No”, go to question 8
No

IF NO: Who should the survey be sent to instead? (You do not need to provide a
name, but rather a job description or job title).

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

Were you able to complete the survey entirely on your own or did you have to
consult others within your organization?
1

9.
10.
11.
12.

Completed the survey on my own If “No”, go to question 10
Completed the survey with others within my organization

If Others:
Who did you have to consult? (Please provide the job title or job description of the
people you consulted as well as the department they work in).
Were any of the questions in the survey unclear or confusing?
1
2

Yes If “No”, go to question 12
No

IF YES: Which ones?

How long did it take you to complete the survey? (your best estimate is fine)

→ These are all the questions that I have for you. Thank you for completing the survey
and for allowing me to talk to you about the survey. To thank you for your time, we will
send you a (check or gift card) for $300. You should get it within the next 2 weeks. If
you have any other comments or any questions or concerns about this study, please
contact Cheryl Damberg, Principal Investigator at damberg@rand.org, 310-393-0411
x6191.

INTERVIEWER: VERIFY THE NAME AND ADDRESS OF THE PERSON WHO WILL
RECEIVE THE CHECK

End Time:

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

Citations
1.
2.
3.
4.
5.
6.

7.
8.
9.
10.
11.
12.
13.
14.
15.

Centers for Medicare & Medicaid Services, 2018 National Impact Assessment of the
Centers for Medicare & Medicaid Services (CMS) Quality Measures Report. 2018, US
Department of Health and Human Services, Centers for Medicare & Medicaid
Services: Baltimore, MD.
Centers for Medicare & Medicaid Services, 2015 National Impact Assessment of the
Centers for Medicare & Medicaid Services (CMS) Quality Measures Report. 2015:
Baltimore, MD.
Centers for Medicare & Medicaid Services. Meaningful Measures Hub. 2018 [cited
2018 June 4]; Available from: https://www.cms.gov/Medicare/Quality-InitiativesPatient-Assessment-Instruments/QualityInitiativesGenInfo/MMF/General-infoSub-Page.html.
Damberg, C.L., et al., Measuring success in health care value-based purchasing
programs: Findings from an environmental scan, literature review, and expert panel
discussions. Rand health quarterly, 2014. 4(3).
Ivers, N., et al., Audit and feedback: effects on professional practice and healthcare
outcomes. The Cochrane Library, 2012.
Jones, S.S., et al., Health information technology: an updated systematic review with a
focus on meaningful use. Annals of internal medicine, 2014. 160(1): p. 48-54.
Totten, A.M., et al., Closing the quality gap: revisiting the state of the science (vol. 5:
public reporting as a quality improvement strategy). Evidence report/technology
assessment, 2012(2085): p. 1.
Ellenbecker, C.H., et al., Patient safety and quality in home health care., in Patient
Safety & Quality-An Evidence-based Handbook for Nurses., R. Hughes, Editor. 2008,
AHRQ: Rockville, MD. p. 1–40.
Resnick, H.E. and M. Alwan, Use of health information technology in home health and
hospice agencies: United States, 2007. J Am Med Inform Assoc, 2010. 17(4): p. 38995.
Parker, E., et al., Exploring best practices in home health care: a review of available
evidence on select innovations. Home Health Care Management & Practice, 2014.
26(1): p. 17-33.
Adler-Milstein, J., et al., Electronic health record adoption in US hospitals: the
emergence of a digital "advanced use" divide. J Am Med Inform Assoc, 2017. 24(6): p.
1142-1148.
Esslinger, E.E., et al., Exploratory analysis of the relationship between home health
agency engagement in a national campaign and reduction in acute care
hospitalization in US home care patients. J Eval Clin Pract, 2014. 20(5): p. 664-70.
Schade, C.P., et al., Impact of a national campaign on hospital readmissions in home
care patients. Int J Qual Health Care, 2009. 21(3): p. 176-82.
Husebo, A.M. and M. Storm, Virtual visits in home health care for older adults.
ScientificWorldJournal, 2014. 2014: p. 689873.
Russell, D., et al., Using technology to enhance the quality of home health care: three
case studies of health information technology initiatives at the visiting nurse service of
New York. J Healthc Qual, 2010. 32(5): p. 22-8; quiz 28-9.

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

16.
17.

18.

19.

20.
21.
22.

23.
24.
25.
26.
27.

28.
29.

30.

31.
32.
33.

34.

Hall, P. and M. Morris, Improving heart failure in home care with chronic disease
management and telemonitoring. Home Healthc Nurse, 2010. 28(10): p. 606-17; quiz
618-9.
Rollow, W., et al., Assessment of the Medicare quality improvement organization
program. Ann Intern Med, 2006. 145(5): p. 342-53.
Egnatios, D., L. Dupree, and C. Williams, Performance improvement in practice:
managing urinary incontinence in home health patients with the use of an evidencebased guideline. Home Healthc Nurse, 2010. 28(10): p. 620-8; quiz 629-31.
Jung, K., D. Shea, and C. Warner, Agency characteristics and changes in home health
quality after Home Health Compare. J Aging Health, 2010. 22(4): p. 454-76.
McKnight, R., Home care reimbursement, long-term care utilization, and health
outcomes. Journal of Public Economics, 2006. 90(1-2): p. 293-323.
Medicare and Medicaid Programs; Conditions of Participation for Home Health
Agencies; Delay of Effective Date., F. Register, Editor. 2017.
Jung, K. and R. Feldman, Public reporting and market area exit decisions by home
health agencies. Medicare Medicaid Res Rev, 2012. 2(4).
AHRQ, AHRQ Conference on Health Care Data Collection and Reporting: collecting and
reporting data for performance measurement: moving toward alignment., in AHRQ
Conference on Health Care Data Collection and Reporting: Report of Proceedings.
2007, U.S. Agency for HealthCare Research and Quality.
Kawamoto, K., et al., Improving clinical practice using clinical decision support
systems: a systematic review of trials to identify features critical to success. BMJ, 2005.
330(7494): p. 765.
Deblois, S. and L. Lepanto, Lean and Six Sigma in acute care: a systematic review of
reviews. Int J Health Care Qual Assur, 2016. 29(2): p. 192-208.
Petersen, L.A., et al., Does pay-for-performance improve the quality of health care?
Annals of internal medicine, 2006. 145(4): p. 265-272.
Flodgren, G., et al., An overview of reviews evaluating the effectiveness of financial
incentives in changing healthcare professional behaviours and patient outcomes.
Cochrane Database Syst Rev, 2011(7): p. CD009255.
Byrnes, J. and J. Fifer, The people structure of quality improvement. Healthc Financ
Manage, 2010. 64(3): p. 64-70.
Castle, N.G. and J. Engberg, The influence of staffing characteristics on quality of care
in nursing homes. Health Services Research, 2007. 42(5): p. 1822-1847.
Castle, N.G., J. Engberg, and A. Men, Nursing home staff turnover: Impact on nursing
home compare quality measures. The Gerontologist, 2007. 47(5): p. 650-661.
Horn, S.D., et al., Beyond CMS quality measure adjustments: identifying key resident
and nursing home facility factors associated with quality measures. Journal of the
American Medical Directors Association, 2010. 11(7): p. 500-505.
Grimshaw, J.M., et al., Changing provider behavior: an overview of systematic reviews
of interventions. Med Care, 2001. 39(8 Suppl 2): p. II2-45.
Unverzagt, S., et al., Strategies for guideline implementation in primary care focusing
on patients with cardiovascular disease: a systematic review. Fam Pract, 2014. 31(3):
p. 247-66.
Bradley, E.H., et al., From adversary to partner: have quality improvement
organizations made the transition? Health Serv Res, 2005. 40(2): p. 459-76.

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

Development of National Provider Survey of Home Health Agencies

35.

36.

37.

38.
39.

40.
41.
42.
43.
44.

45.

46.
47.
48.

Snyder, C. and G. Anderson, Do quality improvement organizations improve the
quality of hospital care for Medicare beneficiaries? JAMA, 2005. 293(23): p. 2900-7.
Bravata, D.M., et al., in Closing the Quality Gap: A Critical Analysis of Quality
Improvement Strategies (Vol. 5: Asthma Care). 2007: Rockville (MD).
Shojania, K.G., et al., in Closing the Quality Gap: A Critical Analysis of Quality
Improvement Strategies (Vol. 2: Diabetes Care). 2004: Rockville (MD).
Walsh, J., et al., in Closing the Quality Gap: A Critical Analysis of Quality Improvement
Strategies (Vol. 3: Hypertension Care). 2005: Rockville (MD).
Fung, C.H., et al., Systematic review: the evidence that publishing patient care
performance data improves quality of care. Ann Intern Med, 2008. 148(2): p. 111-23.
Barnett, M.L., A.R. Olenski, and A.B. Jena, Patient Mortality During Unannounced
Accreditation Surveys at US Hospitals. JAMA Intern Med, 2017. 177(5): p. 693-700.
Devers, K.J., H.H. Pham, and G. Liu, What is driving hospitals' patient-safety efforts?
Health Aff (Millwood), 2004. 23(2): p. 103-15.
Halladay, J.R., et al., Cost to primary care practices of responding to payer requests for
quality and performance data. Ann Fam Med, 2009. 7(6): p. 495-503.
de Vos, M., et al., Using quality indicators to improve hospital care: a review of the
literature. Int J Qual Health Care, 2009. 21(2): p. 119-29.
de Vos, M.L., et al., Implementing quality indicators in intensive care units: exploring
barriers to and facilitators of behaviour change. Implement Sci, 2010. 5: p. 52.
Filipova, A.A., Electronic health records use and barriers and benefits to use in skilled
nursing facilities. Comput Inform Nurs, 2013. 31(7): p. 305-18.
Addington, D., et al., Facilitators and barriers to implementing quality measurement
in primary mental health care: Systematic review. Can Fam Physician, 2010. 56(12):
p. 1322-31.
Federman, A.D. and S. Keyhani, Physicians' participation in the Physicians' Quality
Reporting Initiative and their perceptions of its impact on quality of care. Health
Policy, 2011. 102(2-3): p. 229-34.
Sloane, P.D., et al., How eight primary care practices initiated and maintained quality
monitoring and reporting. J Am Board Fam Med, 2011. 24(4): p. 360-9.

Confidential Material: Not for Dissemination or Disclosure
Internal Use Only

File Type	application/pdf
File Title	Development of the National Provider Survey of Home Health Agencies
Subject	Provider, Survey, Home Health, CMS
Author	HSAG
File Modified	2018-11-05
File Created	2018-10-29