CONTENTS
I. SUPPORTING STATEMENT FOR PAPERWORK REDUCTION ACT SUBMISSIONS 3
A. JUSTIFICATION 3
1. Need and Legal Basis 3
2. Information Users 3
3. Improved Information Technology 3
4. Duplication of Similar Information 4
5. Small Businesses 4
6. Less Frequent Collection 4
7. Special Circumstances 4
8. Federal Register Notice/Outside Consultation 4
9. Payment/Gift to Respondents 5
10. Confidentiality 5
11. Sensitive Questions 5
12. Burden Estimate (Total Hours and Wages) 5
13. Capital Costs (Maintenance of Capital Costs) 5
14. Cost to Federal Government 6
15. Program or Burden Changes 6
16. Publication and Tabulation Dates 6
17. Expiration Date 8
18. Certification Statement 8
B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS 8
1. Respondent Universe and Sampling Methods 8
2. Procedures for the Collection of Information 9
3. Methods to Maximize Response Rates and Deal with Nonresponse 9
4. Tests of Procedures or Methods to Be Undertaken 10
5. People Consulted on Statistical Aspects, and People Collecting or Analyzing Data 10
This data collection is needed by the Centers for Medicare & Medicaid Services to evaluate care and disease management programs (C/DM) under Medicare Advantage (MA). The proposed survey is an important step in achieving the study goals to understand the types of programs and models of C/DM utilized by plans under MA, the population receiving the C/DM services, the role of the health plans, and what has been learned on the effectiveness of these programs. The survey will provide basic information on whether and how MA contractors use C/DM. Survey responses will also provide the plan data needed to help select candidates for the case studies as well as pre-site visit context that will help the team identify the appropriate array of individuals with whom the team should meet. The mail survey will allow the evaluators to develop an inventory of C/DM programs offered by MA plans that characterizes key structural and operational features, as well as their approaches to monitoring and assessing program effectiveness. Given the current lack of information about how MA plans use C/DM programs to improve member health and manage financial risk, this inventory will provide a benchmark against which to chart the use of such programs as they evolve over time.
The information collected will provide a detailed picture to CMS of the kinds of C/DM programs utilized by MA plans and some preliminary information on how plans assess the effectiveness of these programs. The survey of MA plans will result in an overall picture of C/DM programs available to the MA population that can be used for national comparisons. The information will allow CMS to identify 6 C/DM programs from around the country to investigate in more depth through case studies. In doing this, the evaluation will also contribute to CMS’ interest in understanding the evidence that plans are using to inform their decisions for investing and continuing to invest in these programs and on any insights on whether managed care environments like Medicare Advantage plans lend themselves to more effective care and disease management programs for the Medicare population.
Data collection will primarily be achieved through a self-administered mail survey. However, in order to reduce respondent burden as much as possible, respondents will be given the option of responding to the survey by mail or electronically. At the commencement of data collection, each respondent will receive a hard copy of the questionnaire and a letter explaining the purpose of the survey and information about how to complete the questionnaire electronically, if desired. The name and contact information of the respondent is requested, but no signature is required. Respondents who elect to respond by mail will have 14 days to fill out the questionnaire and return it in the enclosed pre-paid envelope or by fax. Respondents who elect to respond electronically can request a copy of the questionnaire by e-mail, or can download an electronic copy of the survey from a website identified in the introductory letter. Completed electronic questionnaires can be returned by e-mail. We expect 50% of the respondents will elect to fill out and return the electronic questionnaire by e-mail. The survey team believes that offering multiple methods of responding to the questionnaire will yield a higher response rate, as respondents can select the method with which they are most comfortable and that is most convenient to them.
This collection is focused on information about provider networks and electronic records maintained for the MA plans at the contract level, how MA plans identify members for its C/DM intervention, specific features of the MA plan’s C/DM intervention (provided that such interventions are offered), and how plans assess their C/DM effectiveness. This survey will ask health plans only about information that they have not already reported to CMS and that is not available on the HPMS.
Small businesses or other small entities are neither involved in nor significantly impacted by this program.
This is a one-time data collection. Not conducting this survey would limit CMS’s understanding of the operation and structure of MA C/DM programs.
There are no special circumstances that would cause the collection of information to be inconsistent with 5 CFR 1320.6.
The notice required in 5 CFR 1320.8(d) was published in the Federal Register on January 28, 2008. Comments were received and addressed under a separate cover.
Expert consultants on the work of MA plans from CMS and outside of CMS were solicited for suggestions on the content and wording of the survey instrument. The suggestions of these experts were integrated into the survey to add clarity to the questions and to reduce the time burden on respondents. In order to fine-tune the survey instrument before actual data collection, the survey instrument will be pre-tested with nine MA plan representatives who have knowledge about C/DM programs.
There are no plans for payment of any kind to respondents.
MPR will take several steps to assure respondents that the information they provide will be treated as private to the fullest extent permitted by law and used for research purposes only. Advance letters to contracts will inform respondents that data will be aggregated in reports to CMS and that contract level data will not be reported. Staff assigned to work on the project sign confidentiality pledges as a term of employment. This pledge requires staff to maintain the privacy of all information collected.
There are no questions of a sensitive nature.
Table A1 presents estimates of the response burden. We estimate the pre-survey initial call will take 10 minutes to complete. The pre-survey initial call will identify whether and how the plan operates C/DM programs that will inform whether we send them the survey questionnaire. We estimate that the survey will take approximately 45 minutes to complete. The information requested in the survey is information that is usual and customary for MA plan representatives working on C/DM programs. There are no cost burdens as there are no capital and startup costs and no operations/maintenance of services costs to respondents.
Table
A.1
Estimated Annualized Burden Hours
Form Name |
Time per response |
Hour per response |
Annual Hour Burden |
Initial Call |
10 |
.1667 |
79.2 |
Mail Survey |
45 |
.7500 |
356.3 |
Total |
55 |
.9167 |
435.5 |
There are no direct costs to respondents other than their time to participate in the study.
The estimated cost to the federal government for conducting the survey is $132,172. This figure is the contract amount for L&M and MPR to conduct the survey, and includes questionnaire development and testing, as well as the development of a conceptual framework, approval of the OMB package, data collection, and analysis.
This is a new data collection.
The survey will be conducted in June/July 2008 or as soon as possible after OMB clearance. The following table shows the overall schedule for the survey, including the beginning and ending dates for data collection.
PROPOSED SURVEY SCHEDULE
Activity |
Time Frame |
Pre-Survey Screening Call |
June 1, 2008 – July 1, 2008 |
Mail out advance letter and copy of survey to all respondents |
June 1, 2008 – July 1, 2008 |
Send fax, mail, or electronic reminders about survey to all respondents |
June 15, 2008 – July 15, 2008 |
Make follow-up calls to respondents who have not yet returned survey |
June 15, 2008 – July 15, 2008 |
End data collection |
August 10, 2008 |
Data cleaning |
August 10, 2008 – August 31, 2008 |
Prepare analytic file and analyze data |
September 1, 2008 – October 15, 2008 |
Final interim report submitted to CMS |
December 10, 2008 |
Final evaluation report |
October 30, 2009 |
The findings of the plan survey will be reported through an interim report submitted by the contractor to CMS in December 2008. A first and fundamental step in the analysis of survey results will be to examine the variables of interest for normality, identifying those with a skewed distribution and potentially transforming the data (e.g., log form) to impose normality. For continuous variables, this univariate analysis can be conducted with visual inspection of the variables through scatter plot matrixes, box-plots, and other graphical displays. Analysis of outliers will also be an important component of the univariate analysis. In addition, simple frequencies of study variables will be calculated for the population total and by stratum of interest (e.g., plan type, geography, program type, plans with or without C/DM programs to provide a basic description of the study population. The next step will involve bivariate analyses to examine the relationships among the variables of interest. Again, visual inspection through graphical analyses will be performed to observe the directionality of the relationships.
While this study is largely descriptive, multivariate analyses can be instructive in identifying the magnitudes and likelihoods of relationships between health plan or C/DM program characteristics and outcomes of interest. For example, we can look at which characteristics might be associated with whether a health plan has a C/DM program, whether the program is managed internally or through a vendor, whether the plan has a particular type of program (e.g., diabetes-focused only, or coordination of care only, or multiple condition-focused). The specific models will be developed after receipt of the data. However, a sample multivariate regression model can be summarized by the equation below along with brief examples of data elements.
Yi = 0 + 1’X1’ + 2’X2’ + 3’X3’ + 4’X4’ + i
i = unique health plan (1 to n, where n=1,2,3…sample size)
Yi = Outcome variable for health plan ‘i’ and may include:
Has C/DM program
Management type (in house, vendor, mixed)
Participation/attrition %
X1’ = represents the vector of health system/health plan characteristics and may include:
Health plan features (size, enrollment, model type/contracting)
Geography (region, state)
X2’ = represents the vector of program characteristics and may include:
Program orientation (e.g., patient/provider/both)
Data system type
Mode/frequency of identification approaches
Types of professionals providing C/DM
Assessment/Care-planning features
Monitoring/Education features
Care coordination/Support service features
Duration
Provider support tools
X3’ = represents the vector of target population characteristics and may include:
Inclusion/exclusion criteria for C/DM program enrollment
General characteristics of enrollees (e.g., eligibility category, gender, race/ethnicity, dual eligible status, age) – we will pilot test capacity of plans to provide this
i = error term
A comprehensive analytic plan specifying the range of univariate, bivariate, and multivariate analyses to be conducted will be submitted to the CMS Project Officer for review following preliminary data inspection, but prior to formal data analysis. We anticipate that this will include prevalence estimates of health plan and C/DM program characteristics. Assuming an 80% response rate for the survey, we anticipate approximately 380 observations will be available for analysis. This sample is sufficient for robust multivariate analyses using the entire sample. However, the team will take care when conducting any stratified analyses (e.g., C/DM program managed internally vs. those relying on external vendors, for-profit vs. nonprofit plans, single disease management programs vs. multiple conditions) to ensure that the sample size is still sufficient. However, these determinations will be made once the data are available and the distribution of observations across the strata of interest can be assessed. Depending on the distribution of the survey responses, we would explore the following types of research questions:
What health plan factors are associated with offering C/DM programs?
What characteristics are associated with in-house, vendor, or mixed management of the C/DM program?
What program characteristics are associated with program attrition?
Is there a relationship between modes of population identification and participation or attrition rates?
The final report will include updated sections from the interim and case study reports, a detailed presentation of synthesized results by the Aims and research questions, and conclusions. The final report will be submitted to CMS in October 2009.
The OMB expiration date will be displayed on the mail questionnaire, on the letters, and on any advance material sent to respondents.
The data collection will conform to all provisions of the Paperwork Reduction Act.
Rather than employ statistical sampling of MA plans for the mail survey, CMS intends to conduct a survey of the entire population of plans operating C/DM programs in 2008. About 472 MA plan contracts were operating in June 2007 and would be eligible for the survey; therefore, CMS has chosen to survey the entire population of MA plans.
The universe of MA plans for the mail survey is about 600 MA contracts that will be operating in May 2008. The mail survey will be administered to about 475 of these. Of the 600 contracts or plans, 130 will be excluded from the evaluation because they are demonstrations, pilots, Medical Savings Accounts, and Cost or Health Prepayment Plans which either do not include financial risk as MA plans normally do or are unlikely to have C/DM programs. Contact information for MA plans will come from CMS’s contract and plan contact databases maintained in the Health Plan Management System. We plan to survey all eligible MA plans and so will not sample from this population.
The survey will be conducted with all MA plans under contract with CMS in 2008. An initial call will be placed to determine whether plans operate C/DM programs and identify the most knowledgeable person about the program(s). The survey will be sent to each plan along with a cover letter explaining the purpose of the survey. Respondents will be asked to fill out the questionnaire and return it by e-mail, fax, or in the pre-paid envelope within one month. Two weeks after sending out the survey, the research team will send a fax, mail, or electronic reminder about the survey to all respondents. The research team will conduct follow-up phone calls to respondents who have not yet returned the survey after one month.
Response rates in the mail survey of MA plans will be maximized in a number of ways. Just before the survey begins, CMS will, during its weekly conference call, inform plans of the coming survey and the importance of participation. We will mail introductory letters on CMS stationery and follow with telephone calls to determine whether selected plans operate C/DM programs and are therefore eligible for the survey. During these calls, we will also identify the person most knowledgeable about these programs; we will then mail the survey to this designated survey respondent. The cover letter, which will be personally addressed and on CMS letterhead, will include contact information and the signature of the CMS project officer, as well as the toll-free number of the MPR survey director. The letter will describe the evaluation and the purpose of the mail survey, and will provide instructions and a date for responding. It will indicate that the survey is voluntary and give the estimated time for completion.
Follow-up telephone calls by trained interviewers (during which plans can complete the survey) extend our strategy. We will send one questionnaire by mail and place a follow-up call if the plan has not responded in two weeks. The questionnaire is relatively short and has only a few open-ended response categories. There are clear instructions on the first page. We considered making the survey available on the web but concluded that the response rate might be lower for this modality; we believe that a mail survey will be convenient for respondents because they may need to check administrative records as they complete the questionnaire.
The response rate for the mail survey will be calculated as the number of MA plans that complete the questionnaire (either by mail or by telephone) divided by the total number of MA plans that were mailed surveys (all unique MA plans). Because we know the universe of approved, unique MA plans or contracts, the denominator of the response rate does not include ineligible plans or plans whose eligibility is unknown. Response rate calculations are based on standards established by the American Association for Public Opinion Research.
Based on previous surveys with similar populations, we anticipate achieving a minimum response rate of 80 percent on the survey. For non-respondents, we will construct a profile based on characteristics of the plans drawn from the HPMS and data collected through the pre-survey screening outreach.
A total of nine MA plans were selected to pretest the survey instrument. The plans were selected to represent a mix that varies by whether C/DM was offered. The pretest identified some items that were burdensome or difficult to respond to, and these items were removed or revised accordingly. An average response time estimate from the pretests was 55 minutes, which is used in our response burden estimate in Section A.12 above.
The following people have contributed to the design of the mail survey: Dr. Lisa Green, project director at L&M Policy Research (240-476-6663); Ms. Myra Tanamor of L&M (202) 230-9029; Ms. Julia Doherty of L&M (202) 291-2518; Ms. Jennifer Schore, an MPR senior researcher (609-275-2380); and Mr. Todd Ensor, an MPR senior survey researcher (609-275-2326). Ms. Noemi Rudolph (410-786-6662), Project Officer at CMS, Office of Research, Demonstrations, and Information, is supervising the study for the government.
| File Type | application/msword |
| File Title | MEMORANDUM |
| Author | Angela Merrill |
| Last Modified By | CMS |
| File Modified | 2008-07-24 |
| File Created | 2008-07-24 |