ATTACHMENT N.1 Pretest memo on 51 State Survey

Attachment N.1

pre-test Memo from 51-State Survey

Pretest Memo: 51-State Survey of Medicaid and CHIP Outreach, Enrollment, and Renewal Policies

A. Overview of the 51-State Survey Instrument Pretest

The survey instrument collects information about Medicaid and CHIP policies and experiences with outreach, enrollment and renewal. In February 2012, we invited three respondents to complete the survey and discuss their responses and their experience of the instrument with us. Two were former state Medicaid officials who had multiple years of experience with Medicaid and CHIP enrollment and renewal. One was from a state with a combined Medicaid and CHIP program and one from a state with a separate CHIP program. The third respondent is a current Medicaid official and a member of the project’s Technical Advisory Group. The purpose of the pretest was to test the language, question flow, respondent comprehension, and to determine the overall burden of completing the survey.

Each of the three respondents independently completed a paper and pencil copy and then participated in a telephone debriefing with HMA. The average time to complete the survey was 30 minutes (with a range of 25 to 35 minutes). This includes some time to read the skip pattern instructions, which will be eliminated during electronic administration of the actual survey, where skip patterns will be programmed. With less reading, and the likelihood of some questions being skipped automatically in the electronic version, we expect response times will range from 20 to 30 minutes.

All three agreed that their agency would have found the survey of interest and relevant, and that their agency would have responded to our request. Generally speaking, a survey requested by a research organization for policy purposes would be completed. All found the survey length acceptable. Two of three noted some repetition in questions (done deliberately to solicit information on both Medicaid and CHIP at both application and renewal). As shown below, we have restructured the questions with repeated concepts in a table, making it easier for respondents to indicate responses without re-reading answer choices.

Respondents noted that the Medicaid Director (or CHIP Director in a state with a separate CHIP program) is the best person to send the survey to, even though he or she may refer it on to someone else for completion. In one state, all surveys get funneled to one lower level staff person who seeks the appropriate assistance to complete it. For example, eligibility questions would go to the Office of Medicaid Policy and Planning, whereas application processing questions would go to the Division of Family Resources. The staff person would follow-up with all the responders until all the answers were back.

Respondents found all questions were comprehensible. One respondent said a question about adequacy of staffing would be skipped in their state due a law suit, and another state respondent said they would skip the response category that asks if political barriers prevented using ELE. The other respondents found those questions appropriate in their states. Given that we are interested in these responses to the extent states administrators can answer them, we opted to include these questions rather than omit them.

Additional clarifying and simplifying edits made in response to respondent feedback are noted below, and a complete copy of the revised survey can be found in the Part A Statement, Attachment F.

Feedback and How it Was Addressed: Survey of Medicaid and CHIP Outreach, Enrollment and Renewal Policies

1. FEEDBACK TO THE PREAMBLE: Two respondents suggested we elaborate on the purpose and source of funding for the evaluation as it would carry more weight to know that it was Congressionally-mandated. The highlighted text has been added to address their suggestion.

This survey is being conducted for the U.S. Department of Health and Human Services (HHS) by its contractor, Health Management Associates (HMA). The survey is part of a larger evaluation being conducted by HMA, Mathematica Policy Research, and the Urban Institute to understand the impact of states’ activities to enroll and retain eligible children in Medicaid and CHIP. Congress mandated this study be undertaken to examine the impact of Express Lane Eligibility provisions of the Children’s Health Insurance Program Reauthorization Act (CHIPRA), passed in 2009. The study includes ELE and non-ELE enrollment and renewal simplifications undertaken by all states and the District of Columbia. Your responses will be used in the report to Congress and will be attributed to your state (or D.C.), but not to you personally.

2. FEEDBACK ON RESPONSE CHOICES, QUESTION 4: One respondent suggested we add a response choice which is “Intend to use” which will be relevant to several states in the midst of the implementation process; this has now been inserted. Below we show the original question and responses, with the new language inserted in yellow highlight. In further edits, question 4 has been made into a table that also includes questions about respondents’ experiences with each data source. See the second version of Question 4 (next page) for the final formatting of the new question 4.

Initial version of question 4 with new language inserted:

4. Please indicate below if you are using or intend to use any of the following third-party data to identify potentially eligible children for outreach.

	Data sources being used for Medicaid outreach	Data sources being used for CHIP outreach
SNAP	Yes/No/Intend to Use/ Tried to use but could not	Yes/No/Intend to Use/ Tried to use but could not
Tax returns	Yes/No/Intend to Use/ Tried to use but could not	Yes/No/Intend to Use/ Tried to use but could not
WIC	Yes/No/Intend to Use/ Tried to use but could not	Yes/No/Intend to Use/ Tried to use but could not
National School Lunch Program	Yes/No/Intend to Use/ Tried to use but could not	Yes/No/Intend to Use/ Tried to use but could not
Other ________	Yes/No/Intend to Use/ Tried to use but could not	Yes/No/Intend to Use/ Tried to use but could not
Other ________	Yes/No/Intend to Use/ Tried to use but could not	Yes/No/Intend to Use/ Tried to use but could not

Revised version of question 4:

4. Please indicate below if you are using any of the following third-party data to identify potentially eligible children for outreach, and if you experienced any problems in doing so. (Questions about data matching for enrollment will come later.)

	Data sources being used for Medicaid outreach	Data sources being used for CHIP outreach	Problems your agency experienced in using this data source for outreach
SNAP	Yes No Tried to use but could not Intend to use	Yes No Tried to use but could not Intend to use	SNAP agency lacked the resources or systems to participate SNAP and Medicaid/CHIP had difficulty developing a data sharing process The policy/political environment was not supportive Medicaid/CHIP could not reach families identified through the match Medicaid/CHIP did not get enough responses back from families Other, please describe:_____________________
Tax returns	Yes No Tried to use but could not Intend to use	Yes No Tried to use but could not Intend to use	Tax agency lacked the resources or systems to participate Tax and Medicaid/CHIP had difficulty developing a data sharing process The policy/political environment was not supportive Medicaid/CHIP could not reach families identified through the match Medicaid/CHIP did not get enough responses back from families Other, please describe: ____________________
WIC	Yes No Tried to use but could not Intend to use	Yes No Tried to use but could not Intend to use	WIC agency lacked the resources or systems to participate WIC and Medicaid/CHIP had difficulty developing a data sharing process The policy/political environment was not supportive Medicaid/CHIP could not reach families identified through the match Medicaid/CHIP did not get enough responses back from families Other, please describe:__________________
National School Lunch Program (NSLP)	Yes No Tried to use but could not Intend to use	Yes No Tried to use but could not Intend to use	NSLP agency lacked the resources or systems to participate NSLP and Medicaid/CHIP had difficulty developing a data sharing process The policy/political environment was not supportive Medicaid/CHIP could not reach families identified through the match Medicaid/CHIP did not get enough responses back from families Other, please describe:_____________________
Other ________	Yes No Tried to use but could not Intend to use	Yes No Tried to use but could not Intend to use	The other agency lacked the resources or systems to participate The other agency and Medicaid/CHIP had difficulty developing a data sharing process The policy/political environment was not supportive Medicaid/CHIP could not reach families identified through the match Medicaid/CHIP did not get enough responses back from families Other, please describe:_____________________
Other ________	Yes No Tried to use but could not Intend to use	Yes No Tried to use but could not Intend to use	The other agency lacked the resources or systems to participate The other agency and Medicaid/CHIP had difficulty developing a data sharing process The policy/political environment was not supportive Medicaid/CHIP could not reach families identified through the match Medicaid/CHIP did not get enough responses back from families Other, please describe:____________________

3. FEEDBACK FOR QUESTIONS ABOUT BARRIERS, QUESTIONS 5-11: One respondent expressed his confusion about the meaning of “policy environment” and suggested “political environment” was clearer. When prompted, other respondents stated either word was clear to them. In the five questions with this response choice, we added “political” as shown in this example (inserted in all of the questions 5-11 of survey). Below we show the original questions and responses, with the new language inserted in yellow highlight. We further simplified questions 5-11 by including them in a column in question 4, above. In this way, a respondent would be able to answer all related questions at once, reducing the time it would take to re-read similar language. This suggestion, made by a reviewer, is also noted below.

Initial versions of questions with new text highlighted (revised final versions above in table):

5. When you tried using SNAP for outreach, please indicate whether you experienced any of the following problems: (select all that apply)

1. The agency administering SNAP lacked the resources or systems to participate

2. The SNAP and Medicaid agencies had difficulty developing a data sharing process

3. The policy/political environment was not supportive

4. The Medicaid agency could not reach families identified through the match

5. The Medicaid agency did not get enough responses back from families

6. Other/details: _________________________________

6. When you tried to use tax returns for outreach, please indicate whether you experienced any of the following problems: (select all that apply)

1. The tax agency lacked the resources or systems to participate

2. The tax and Medicaid agencies had difficulty developing a data sharing process

3. The policy/political environment was not supportive

4. The tax or Medicaid agency could not reach families identified through the match

5. The Medicaid agency did not get enough responses back from families

6. Other/details: _________________________________

6. When you tried to use WIC for outreach, please indicate whether you experienced any of the following problems: (select all that apply)

1. The WIC agency lacked the resources or systems to participate

b. The WIC and Medicaid agencies had difficulty developing a data sharing process

c. The policy/political environment was not supportive

d. The WIC or Medicaid agency could not reach families identified through the match

e. The Medicaid agency did not get enough responses back from families

f. Other/details: _________________________________

6. When you tried using the National School Lunch Program for outreach, please indicate whether you experienced any of the following problems: (select all that apply)

1. The NSLP agency, such as your state’s Department of Education, lacked the resources or systems to participate

2. The NSLP and Medicaid agencies had difficulty developing a data sharing process

3. The policy/political environment was not supportive

4. The NSLP or Medicaid agency could not reach families identified through the match

5. The Medicaid agency did not get enough responses back from families

6. Other/details: _________________________________

6. When you tried using another matching source (please specify: ______________) for outreach, please indicate whether you experienced any of the following problems: (select all that apply)

1. The other agency who had the data lacked the resources or systems to participate

2. The Medicaid agency and the other agency who had the data had difficulty developing a data sharing process

3. The policy/political environment was not supportive

4. The other agency or the Medicaid agency could not reach families identified through the match

5. The Medicaid agency did not get enough responses back from families

6. Other/details: _________________________________

6. When you tried using another matching source (please specify: ______________) for outreach, please indicate whether you experienced any of the following problems: (select all that apply)

7. The other agency who had the data lacked the resources or systems to participate

8. The Medicaid agency and the other agency who had the data had difficulty developing a data sharing process

9. The policy/political environment was not supportive

10. The other agency or the Medicaid agency could not reach families identified through the match

11. The Medicaid agency did not get enough responses back from families

12. Other/details: _________________________________

6. If you have not attempted to use third-party data matching to identify potentially eligible but uninsured children, why not? (select all that apply)

1. Current methods of outreach have been sufficiently successful.

2. We lack the computer systems to do so.

3. We lack the staff time to do so.

4. We think the program eligibility is too different to yield enough children.

5. We have concerns about the validity of income information (e.g., data being outdated)

6. The policy/political environment is not supportive of expanding enrollment

7. We experienced concerns about privacy

8. The other agency we sought to work with lacked the capacity to work with us

9. The other agency we chose to work was not interested

10. Other ______________________

4. FEEDBACK ABOUT WHAT CONSTITUTES “ANALYSIS”, QUESTIONS 14, 17, 40, 47 and 49: One respondent questioned what we meant by “analyzed” when we asked in these five questions if the state had analyzed the impact of their simplifications. This respondent thought that their approach, which was to review data patterns, was very useful to them in shaping policy and should be included explicitly. We made the following change each place this question was used. The change to the original question wording is shown in yellow. As with question 4, above, we found a way to simplify the survey by combining like questions into a table. Therefore, the final edits to questions 14, 17, 40, 47, and 49 appear in the final version of the survey as questions 8, 14, and 16, also provided below (some renumbering occurred because of simplifying into tables) (see Attachment F of Supporting Statement A for a complete version of the final survey).

Initial question versions with new text highlighted:

14. In the past few years, some application simplifications in CHIP and/or Medicaid enrollment were introduced in your state. Have you examined your data or done formal analyses to assess the impact of any application simplifications (such as reduced paper documentation) on enrollment?

17. In the past few years, some application simplifications in CHIP and/or Medicaid enrollment were introduced in your state. Have you examined your data or done formal analyses to assess the impact of any application simplifications (such as reduced paper documentation) on administrative efficiency or cost savings?

40. Have you examined your data or done formal analyses to assess the impact of any application processing simplifications on administrative efficiency or cost savings?

47. Many states have undertaken renewal simplifications since 2008. Have you examined your data or done formal analyses to assess the impact of renewal simplifications on retention of eligible children in Medicaid or CHIP?

49. Have you examined your data or done formal analysis to assess the impact of renewal simplifications on administrative efficiency or cost savings?

Revised versions of questions:

8. States have been undertaking a number of application and enrollment simplifications in the past few years. If you have examined your data or done formal analyses to assess the impact of any simplifications on enrollment, how effective have these changes been?

	How effective have your state’s simplifications been at improving:
Simplification	Medicaid Enrollment	CHIP Enrollment	Administrative Efficiency or Cost Savings
Elimination of face-to-face interview	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state
Elimination or liberalization of asset test	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state
Eligibility expansion for children	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not applicable
Elimination or reduction in premiums	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state
Joint Medicaid/CHIP application form	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state
Reduced or eliminated income documentation	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state
Using the Social Security Administration database for citizenship verification	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state
Acceptance of electronic applications	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state
Acceptance of electronic documentation	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state
Presumptive eligibility	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state
Express lane eligibility	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state
Expansion of coverage for parents (did covering more parents help you enroll more children?)	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state
Increasing consistency between parent and child eligibility and enrollment policies	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state
Other: _____________________________________	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state	Not at all effective Somewhat effective Effective Very effective Not sure or didn’t analyze Not used in our state

14. Which of the following application processing simplifications have you done? If you have examined your data or evaluated the impact, has each simplification been effective in increasing administrative efficiency, saving money, or reducing churn?

	Medicaid	CHIP	Effectiveness?
We have centralized application processing in fewer locations	Yes No In process Don’t know	Yes No In process Don’t know	Not at all effective Somewhat effective Effective Very effective Not sure Not used in our state
We have eliminated or significantly reduced data entry by using an electronic application	Yes No In process Don’t know	Yes No In process Don’t know	Not at all effective Somewhat effective Effective Very effective Not sure Not used in our state
We have reduced documentation handling by adding scanning capacity	Yes No In process Don’t know	Yes No In process Don’t know	Not at all effective Somewhat effective Effective Very effective Not sure Not used in our state
We have updated our eligibility system	Yes No In process Don’t know	Yes No In process Don’t know	Not at all effective Somewhat effective Effective Very effective Not sure Not used in our state
We have improved transfer of applications between Medicaid and CHIP	Yes No In process Don’t know	Yes No In process Don’t know	Not at all effective Somewhat effective Effective Very effective Not sure Not used in our state
We have reduced the need to track down documentation from families by using other data sources.	Yes No In process Don’t know	Yes No In process Don’t know	Not at all effective Somewhat effective Effective Very effective Not sure Not used in our state
Social security records	Yes No In process Don’t know	Yes No In process Don’t know	Not at all effective Somewhat effective Effective Very effective Not sure Not used in our state
Vital records	Yes No In process Don’t know	Yes No In process Don’t know	Not at all effective Somewhat effective Effective Very effective Not sure Not used in our state
Income verification, such as through the Department of Labor, a private data provider, or the IRS.	Yes No In process Don’t know	Yes No In process Don’t know	Not at all effective Somewhat effective Effective Very effective Not sure Not used in our state
Other source: ________________	Yes No In process Don’t know	Yes No In process Don’t know	Not at all effective Somewhat effective Effective Very effective Not sure Not used in our state
Other process changes: _______________________________	Yes No In process Don’t know	Yes No In process Don’t know	Not at all effective Somewhat effective Effective Very effective Not sure Not used in our state

16. States have been undertaking a number of renewal simplifications in the past few years to improve retention of eligible children in coverage. If you have examined your data or done formal analyses to assess the impact of any simplifications on retention, how effective have these changes been?

	Medicaid Retention	CHIP Retention	Improving Administrative Efficiency
Seeking income verification from a third party source prior to asking parents for documentation (include ELE, ex parte renewal, or administrative renewal)	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state
Sending pre-printed renewal forms to families	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state
Automatically renewing coverage unless a change in eligibility is reported	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state
Allowing self-declaration of income	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state
Continuous eligibility	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state
Seeking assistance from the same entity who assisted with the application (e.g., health plan, provider, or CBO)	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state
Improved letters to families	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state
Keeping contact information up to date If so, how was this done? ___________________	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state
Online or phone renewal	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state
Longer premium payment grace periods	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state
Shorter lockout period for late payment	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state
Roll forward the renewal date when a family self-reports changed circumstances	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state
Use ex parte renewal to reset the renewal date for Medicaid/CHIP	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state
Other	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state	Not at all effective Somewhat effective Effective Very effective Not sure Not used in my state

6. EDITS TO QUESTION WORDING, QUESTIONS 26-27: One respondent expressed her confusion about our questions related to web-based applications, possibly because they followed a question about the ability of the web-based application to help an applicant by pre-screening them. We added the yellow highlighting to clarify that the question relates to the more general topic of web-based applications, not explicitly to pre-screening. Below we show the original questions, with new language inserted in yellow highlight, and below that, the final layout, in table form, which is now question 10 (see Attachment F of Supporting Statement A for a complete version of the final survey).

Initial versions of questions with new language in highlighted text:

26. If you have a web-based application, do online applicants get feedback about their likely eligibility at the time they complete the web-based application?

1. Yes

2. No

26. If you have a web-based application, does your web-based application feed data directly into your eligibility system, or must the information from the application be re-entered?

1. There is a direct transfer of information into the eligibility system

2. No, data are entered by our staff

3. Other, please describe: ____________________________________

Revised final versions of questions:

10. Please answer the following questions about use of the internet in your program.

	Medicaid	CHIP
Do you have a web-based application for Medicaid and/or CHIP that allows applicants to pre-screen for eligibility?	Yes No	Yes No
If applicants use a web-based application, do they get feedback about their likely eligibility at the time they complete the web-based application?	Yes No Not applicable	Yes No Not applicable
If applicants use web-based application, does your web-based application feed data directly into your eligibility system, or must the information from the application be re-entered?	Yes, there is a direct transfer of information into the eligibility system No, data are entered by our staff Other_______________________________	Yes, there is a direct transfer of information into the eligibility system No, data are entered by our staff Other______________________________
If you require any documentation, can it be uploaded electronically, or must it be mailed?	Documentation can be uploaded electronically Documentation must be mailed Other_____________________________	Documentation can be uploaded electronically Documentation must be mailed Other_____________________________
If all information is complete when an application is submitted online, how long until a family receives a response?	Immediately Within 24 hours, electronically Longer, by mail Other_____________________	Immediately Within 24 hours, electronically Longer, by mail Other_____________________

7. NEW QUESTION SUGGESTED (AFTER QUESTION 27): This question relates to the emerging practice of electronic applications, and the ability to also submit documentation electronically. We added it based on reviewer feedback. It is now question 10c, shown in the Table above.

New question:

If you require any documentation, can it be uploaded electronically, or must it be mailed?

1. Electronic upload of documentation is an option

2. Documentation must be mailed

3. Other, please describe: ___________________________

8. QUESTION WORDING CHANGE, QUESTIONS 42, 44: One respondent pointed out that some states use contractors in the eligibility determination process, and we should be explicit if we want them to be considered in responses about staff capacity. New language is inserted in yellow highlight. In the final version, these are in question 15 (see Attachment F of Supporting Statement A for a complete version of the final survey).

Initial version of questions with new text highlighted:

42. How has the size of the eligibility staff (including contractors, if you use them) changed since 2008?

1. No change

2. Fewer people involved in eligibility determination

3. More people determining eligibility

44. What changes are you considering, if any, to prepare for increased applications and enrollment in 2014?

1. Adding staff or contractors

2. Centralizing application determinations

3. Contracting with a third party for application processing

4. Adding online applications

5. Adding kiosks or other self-service options

6. Other _________________

7. Don’t know

Revised final version of questions:

14. Please answer the following questions about application processing.

	Medicaid	CHIP
How has the size of the eligibility staff (including contractors, if you use them) changed in the last few years?	No change Fewer people involved in eligibility determination More people determining eligibility	No change Fewer people involved in eligibility determination More people determining eligibility
Do you have enough capacity to meet your goals for application processing?	Yes, resources match our needs. We are getting by We are short-handed We are stressed by the current workload or work processes, and need a change	Yes, resources match our needs. We are getting by We are short-handed We are stressed by the current workload or work processes, and need a change
What changes are you considering, if any, to prepare for increased applications and enrollment in 2014?	Adding staff or contractors Centralizing application determinations Contracting with a third party for application processing Adding online applications Adding kiosks or other self-service options Other _________________ Don’t know	Adding staff or contractors Centralizing application determinations Contracting with a third party for application processing Adding online applications Adding kiosks or other self-service options Other _________________ Don’t know

9. QUESTION WORDING CHANGE, QUESTION 46: Some of our questions gave a date cut-off as a point of reference. One respondent was skeptical she would know which changes were before or after this date. We had used the date to eliminate different interpretations of a more general phrase like “the past few years” but after reconsideration and discussion with pretesters, think it can work adequately. New language is in yellow highlight, eliminated language is in strikethrough font. In the final version, this is now question 16 (see Supporting Statement A, Attachment F).

Initial version of question with new text highlighted:

46. Many states have undertaken renewal simplifications in the past few years. since 2008 Have you evaluated the impact of renewal simplifications on retention?

1. Yes

2. No

Revised final version of question:

16. States have been undertaking a number of renewal simplifications in the past few years to improve retention of eligible children in coverage. If you have examined your data or done formal analyses to assess the impact of any simplifications on retention, how effective have these changes been?

10. DIFFICULTY ANSWERING QUESTION 52: One state indicated they do not track a churning rate, and another state indicated this is the one question in the entire survey that would require follow-up with a different department for an answer. They suggested asking a more general question about magnitude of the problem. We will replace the specific question as follows (new language in yellow highlight, eliminated language in strikethrough font) to allow states that do calculate a churn rate to provide it, while others estimate it. This is now question 18.

Initial version of question with new text highlighted, removed text struck:

52. How big a problem is churning in your Medicaid Program (i.e., children leaving the program while still eligible, and then coming back on the program in a short time)? What percentages of children who lose Medicaid or CHIP coverage at renewal regain coverage within six months? (If you use another metric to measure this, please tell us your version of this measure.)

1. We calculated our Medicaid churn rate to be:_______

2. We believe it happens rarely.

3. We believe it happens somewhat.

4. We believe it occurs for approximately half the children in the program.

Revised final version of question:

18. How big a problem is churning in your Medicaid Program (i.e., children leaving the program while still eligible, and then coming back on the program in a short time)?

1. We calculated our Medicaid churn rate to be:_______

2. We believe it happens rarely.

3. We believe it happens somewhat.

4. We believe it occurs for approximately half the children in the program.

11. QUESTION WORDING, QUESTION 55: States often have a problem reaching enrollees because of changes of address or phone number. With this question, we sought to quantify the magnitude of the problem. One respondent pointed out that they cannot know how many families they reach, only what proportion respond. Question 22 incorporates the clearer wording.

Initial version of question with new language highlighted, removed language struck:

55. What proportion of families responds to your communications? do you reach?

3. Few

4. Less than half

5. Half

6. More than half

7. Most

Revised final version of question:

22. Please answer the following questions about communicating with families.

	Medicaid	CHIP
How do you communicate with families at the time of renewal?	Letters Emails Phone Through the enrollment assistor who helped them apply Other______________________________	Letters Emails Phone Through the enrollment assistor who helped them apply Other__________________________
Approximately what proportion of families responds to your communications?	Few Less than half Half More than half Most	Few Less than half Half More than half Most
Are you working on changing the way you communicate with families at renewal?	No Yes. Please explain: _________________________________	No Yes. Please explain: ________________________________