Part A: Justification Mathematica Policy Research
Part A: Justification
Explain the circumstances that make the collection of information necessary. Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.
The Supplemental Nutrition Assistance Program (SNAP) (formerly known as the Food Stamp Program) is a critical source of support for many low-income families and individuals. In recent years, states have implemented new procedures and policies in order to reduce SNAP administrative costs while maintaining or improving program access. These changes—often referred to as modernization—incorporate technology, administrative restructuring, community partnering, and policy simplification.
Together, the Food, Conservation, and Energy Act of 2008, which amended Section 11 of the Food and Nutrition Act of 2008, 7 USC 2020, and the Food Nutrition Act of 2008, which amended Section 17 of 7 USC 2026, authorize FNS to develop standards for identifying major operational changes, require States to provide any information required by the US Department of Agriculture (USDA), and authorize FNS to undertake research that will help improve the administration and effectiveness of SNAP (see Appendix A). The goal of this study is to learn how states are modernizing their SNAP programs, to measure the extent to which they have met their modernization goals, and to examine how modernization affects key outcome measures: efficiency, access, and integrity. The level of modernization and specific approaches vary substantially by state, and the six states identified as having the most advanced modernization initiatives were selected and have agreed to participate in this study: Florida, Georgia, Massachusetts, Utah, Washington, and Wisconsin. Four of these states – Florida, Massachusetts, Utah, and Washington – will also serve as case study sites for another data collection effort that is studying the performance standards and measures of State SNAP modernization initiatives. This study is a revision of a currently approved collection that gathered information to develop a comprehensive, national inventory of SNAP modernization efforts undertaken in all the states. This study will yield a comprehensive picture of each of the six study state’s experiences with modernization and assess the potential impacts of modernization. Specifically, the study will identify the steps these six states have taken to implement modernization changes, the challenges they experienced, and the perceptions of SNAP staff and participants regarding the changes.
This collection effort builds on the descriptive study “Enhancing Food Stamp Certification: Food Stamp Modernization Efforts.” It will update FNS’ information on state modernizations to provide context for understanding how the initiatives are affecting key outcome measures.
Indicate how, by whom, how frequently, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.
Information for the In-depth Case Studies of Advanced Modernization Initiatives will be collected by Mathematica Policy Research on behalf of FNS. This study will build on the findings from a previous data collection effort, Enhancing Food Stamp Certification: Food Stamp Modernization Efforts. To obtain a detailed and comprehensive view of the implementation of SNAP modernization initiatives, data will be collected via in-person interviews, focus group discussions, and through administrative case records, application statistics, performance data, and other relevant materials.
The project has seven research objectives: (1) update the existing state profiles of modernization efforts and identify the geographic and caseload coverage affected by modernization changes; (2) describe how key certification, recertification, and case management functions have changed; (3) describe the current roles and responsibilities of state and local SNAP staff, vendors, and partners and how they have changed; (4) document the relationship between SNAP modernization initiatives and stakeholder satisfaction; (5) describe the current performance of each state’s modernization initiatives and the level of outcome variability within each state; (6) compare performance before, during, and after modernization1; and (7) document the main takeaway points for use by other states and for future study consideration.
The information from the currently approved data collection describe key features and outcomes associated with SNAP modernization; systematically describe and compare techniques states are using to modernize the SNAP; identify promising practices; and create a single, comprehensive information source on state modernization initiatives. The information is instructive to States about what other States are doing and how the States executed their modernization changes. In addition, it was useful in identifying the States that are most advanced in SNAP modernization initiatives that could participate in this follow-up study. Federal and state policymakers will be able to use the information produced from this study to understand the implications of modernization changes and identify effective modernization practices while avoiding implementation pitfalls.
In-person interviews (Appendix B) will be completed during site visits with either individuals or small groups with a time limit of 60 minutes. For these interviews, there will be four types of respondents: (1) state SNAP staff, (2) county and local SNAP staff, (3) Community Business Organization (CBO) partner staff, and (4) vendor staff. In general, the interviews will cover six main topics: (1) overview of modernization; (2) key modernization changes; (3) roles and responsibilities; (4) implementation and performance; (5) stakeholder satisfaction; and (6) successes, challenges, and lessons learned.
To assess client satisfaction and awareness of SNAP modernization initiatives, four focus groups (Appendix C) will be convened in each of the six states. Among the four focus groups in each state, two will be convened with SNAP participants and two will be convened with eligible non-participants. SNAP participants will fall into one of two categories: recent participants (individuals who applied for SNAP benefits in the preceding three months) or long-term participants (individuals who have been receiving benefits for more than two years). Eligible nonparticipants are individuals who are eligible for SNAP but are not participating in the program. In focus groups with SNAP participants, the discussion will address their experiences and satisfaction with SNAP. Focus groups with eligible nonparticipants, will discuss group members’ knowledge of SNAP, including eligibility requirements, and why people do not apply or participate. The focus groups will be conducted during the site visits and will last 90 minutes or less.
Extant data (Appendix D) will be used to help document the modernization changes made in each study state, identify any trends associated with the implementation of modernization initiatives, and examine whether the modernization changes are potentially driving changes in key program outcomes. Working closely with data managers from each of the six study states, the study team will collect 10 years of monthly state case record extract data in two batches: one containing case records from July 2000 to December 2010 and the other containing case records from January to June 2011. Although each state has its own timeline for modernization, the ten-year period will be sufficient to include data covering years before most modernization activities were implemented for all states.
Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also, describe any consideration of using information technology to reduce burden.
We are committed to compliance with the E-Government Act, 2002 to promote the use of technology. In requesting the extant data, FNS will minimize the burden on respondents by accepting those data in the format (such as excel file, text file, or mdb file) and delivery method (such as use of a secure file exchange site or email) that is most convenient for respondents.
Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purpose described in item 2 above.
Similar data collection efforts have been done but they do provide the information needed for this study. Most SNAP modernization changes are recent, thus the information to be collected in this study does not exist elsewhere. FNS does not require most states (unless certain waivers are in place) to report any information related to the implementation of their modernization efforts; therefore, this data collection does not duplicate state efforts. Additionally, these studies do not examine the implications of modernization initiatives in the depth that we plan.
Respondents from Florida have described their modernization activities in the collections for OMB No.: 0584-0537 expiration date 7/2009, Enhancing Food Stamp Certification: Food Stamp Modernization Efforts and Study of the Modernization of the FSP in Florida. Florida’s modernization initiative, called Automated Community Connection to Economic Self-Sufficiency (ACCESS) Florida, involved a variety of statewide changes to the SNAP eligibility system, including technological expansions such as online applications, document imaging, and a call center with an automated response system; and policy changes such as shortened interviews, expanded use of telephone interviews, and reduced documentation requirements. This study was conducted four years ago in 2006. At that time Florida’s modernization initiatives were still fairly new. A study investigating the longer term effects of modernization in that state has not yet been conducted.
We are also currently conducting the Performance Standards and Reporting for SNAP Modernization Initiatives study that is surveying all states including 150 county and local offices. The purpose of the Performance Standards study is to understand how States are assessing and measuring performance of their modernization initiatives. The Performance Standards study also involves site visits with 10 States, four of which are in this study. While the focus of this data collection is clearly different, similar information, such as descriptions about the modernization initiatives implemented by the states, may have been collected from States. Wherever possible, the project team will draw upon the information collected from the Performance Standards and Reporting study in an effort to reduce the burden on the respondents.
If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.
The information being requested has been held to the minimum required for the intended use. Out of 1,353 respondents, the study team will request participation from 74 small organizations that may, as SNAP partners or vendors, participate in this collection. We will minimize burden on these small organizations by tailoring interview questions to ask staff only the questions that pertain to their work with the state or county or local SNAP office and to exclude references to modernization initiatives in which the partner agency is not involved. We estimate these interviews will take 60 minutes.
Describe the consequence to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.
This is a one-time collection effort. The data collection plan described in this submission will help FNS understand the experiences of states with advanced modernization initiatives, investigate whether there is evidence that the initiatives have affected key program outcomes, and identify key themes and lessons learned from these states’ experiences. Without this detailed case study, FNS would need to rely on the states’ general statements regarding program operations and aggregate statistics on modernization initiatives, resulting in an incomplete understanding of how modernization has affected the implementation of SNAP. In particular, FNS would be missing the perspective of the local office staff and SNAP participants directly affected by modernization changes. Moreover, FNS would have an incomplete understanding of the specific factors that drive changes in modernization initiatives. Finally, FNS would not be able to compile a detailed list of modernization initiatives and combine it with lessons learned from states’ experiences. These limitations would significantly hinder FNS’ ability to make future policy decisions regarding similar modernization issues in other states.
Explain any special circumstances that would cause an information collection to be conducted in a manner:
requiring respondents to report information to the agency more often than quarterly;
requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;
requiring respondents to submit more than an original and two copies of any document;
requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than three years;
in connection with a statistical survey, that is not designed to produce valid and reliable results that can be generalized to the universe of study;
requiring the use of a statistical data classification that has not been reviewed and approved by OMB;
that includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or
requiring respondents to submit proprietary trade secret, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information's confidentiality to the extent permitted by law.
There are no special circumstances. Data collection will be conducted in a manner consistent with the guidelines in 5 CFR 1320.5.
If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the agency's notice, soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments.
Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting form, and on the data elements to be recorded, disclosed, or reported.
In accordance with 5 CFR 1320.8 (d) 1995, a notice of the proposed information collection and an invitation for public comment was published in the Federal Register, August 4, 2010, Volume 75, Number 149, Pages 46899-46901. No comments were received.
In addition to soliciting comments from the public and from National Agricultural Statistics Service (NASS), FNS sought input from three Mathematica senior technical staff about the availability of data, design, level of burden, and clarity of instructions for this collection:
Scott Cody: Associate Director of Research and Project Director 202-484-4523
Kevin Conway: Deputy Project Director 609-750-4083
Lara Hulsey: Researcher 609-936-2778
Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.
Given the time required to travel to and participate in a focus group, FNS requests permission to offer a financial incentive to promote cooperation and participation in the focus groups, as is customary. During screening calls, SNAP participants, applicants, and eligible nonparticipants recruited for the focus groups will be offered $25 (incentive disbursed upon completion of the discussion). All invited SNAP participants will be informed that these incentives will not affect the value of their SNAP benefits. Based on experience, this amount has been enough to encourage participation in focus groups conducted with this population. To further increase participation, a light meal and refreshments, in addition to child care and transportation stipends will be provided to those who express a need for them.
SNAP staff and community partners that are interviewed will not be offered financial incentives to participate.
Each of the six participating states will receive remuneration of up to $75,000 to offset the costs of participating in the study. In addition to the extant data collection costs detailed in Table A.12.2, these funds will compensate the states for time and expenses they have already incurred in support of the study. The six states have already given time to provide information on their modernization initiatives, which has contributed to a more efficient data collection plan. Prior to submission of this data collection for OMB approval, preliminary discussions were held with the states to explain the study to them and obtain some foundational information on the modernization activities they were conducting. The states also completed a short survey that summarized the different types of modernization initiatives and waivers they were implementing. The estimated burden hours to complete the policy questionnaire was approximately 8 hours and 1.125 hours to participate in a follow-up call for clarification or any questions regarding the data collected. The results of the surveys contributed to draft state profiles that will be used to help tailor interviews and focus group discussions to the specific circumstances in each state. After completion of the state profiles, the states attended an in-person orientation meeting in late March 2010. The states reviewed these profiles and provided feedback to ensure their accuracy. The states also worked with Mathematica staff to develop memoranda of understanding and data use agreements for this study.
Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy.
The extant data to be collected from state and local SNAP staff and their community partners and vendors will not include any personal data. A file exchange site will be available for states to transfer data in a secure way. The secure file exchange site uses a Secure Sockets Layer (SSL) certificate to encrypt the data transmission, which may be a requirement for states that have strict data security protocols and need assurance that all steps are being taken to protect SNAP data. Users will access the file exchange site using a site-specific user name and password. During the collection, the project team will retain respondents’ contact information and the details of their responses about state, county, local, or partner efforts in modernization, but this contact information will not be released. In accordance with the Privacy Act, Mathematica will safeguard all data, and only authorized users will have access to them.
Interview and focus group respondents will be notified that participation is voluntary and will not affect their benefits and that their individual responses will be kept secure not be disclosed to anyone apart from the members of the research team, except as required by law. Mathematica has a long history of protecting the security and privacy of records and considers it a critical aspect of any study’s scientific integrity and legality. During eligibility screening calls (Appendix E), staff will explain to potential focus group participants that information requested from them is for research purposes only and that their identities will not be disclosed to anyone outside the evaluation project, including SNAP staff. Focus group moderators will also notify recipients that their conversations in the focus groups will be audio-taped, that their recorded comments will be viewed by members of the research team only and saved only until transcribed, and specify that the transcription summaries will not reveal their identities.
Information obtained from this collection will only be shared with other organizations in aggregate form, without any personally identifying information.
Provide additional justification for any questions of a sensitive nature, such as sexual behavior or attitudes, religious beliefs, and other matters that are commonly considered private. This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.
FNS and Mathematica will comply with the Privacy Act of 1974. The interview questions for SNAP staff and community partners and vendors will primarily relate to program details and their opinions of effectiveness, and will not be sensitive. Additionally, members of the participant focus groups are not likely to view the discussion questions about their SNAP experiences as sensitive.
Some of the questions for nonparticipants focus group members, however, may be considered sensitive. The questions will address their participation in various means-tested programs such as TANF, Supplemental Security Income, Medicaid, and energy assistance. The questions will also explore what these clients do when they need assistance obtaining food. These questions are commonly used in surveys of low-income people, but these clients may be sensitive about answering them in a discussion group. These questions are important components of the nonparticipant discussions, which aim to find out why these individuals do not participate in SNAP and whether their reasons are related to the modernization initiatives that occurred in their state or whether the initiatives will help address perceived barriers to participation.
As described in Section A.10, all respondents will be notified during the screening call and at the outset of the focus group that they will not be personally identified. In addition, they will be informed that participation is voluntary and they need not answer any questions that make them uncomfortable. They will also be informed that there are no penalties if they decide not to respond, either to the information collection as a whole or to any particular question. All responses will be kept secure and will not be reported to SNAP staff or any other program, agency, or organization, except as otherwise required by law. Rather, all the responses will be combined so that no individuals will be identifiable.
Provide estimates of the hour burden of the collection of information. The statement should:
Indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated. If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens in Item 13 of OMB Form 83-I.
Provide estimates of annualized cost to respondents for the hour burdens for collections of information, identifying and using appropriate wage rate categories.
Staff of states participating in this study have participated in an orientation meeting and completed a policy questionnaire. Data collection in this study will include (1) interviews with 33 state-level SNAP staff, (2) 12 SNAP staff compiling and sending the extant data, (3) interviews with 84 district/county SNAP staff, (4) interviews with 21 SNAP call center staff or other centralized operation units staff, (5) interviews with 154 local office SNAP staff (6) interviews with 14 vendors, (7) interviews with 60 staff members from community partners, and (8) focus groups with 120 SNAP participants and 120 eligible nonparticipants (9) Preliminary Meeting and (10) Meeting Policy.
The estimated number of annual responses per respondent is one for all respondents except for the 12 respondents who will provide the extant data, who will have two annual responses. The estimated total annual burden on respondents is 1,801.85 hours. The burden estimates for both the interviews and focus groups outlined in Table A.12.1 reflect the expected average length of the interviews and discussions. The interview and focus group guides provided in Appendices B and C include a comprehensive list of potential questions and follow-up probes that researchers might pursue during the interviews and discussion groups. The particular questions asked of each respondent will depend on the modernization initiatives implemented in the state and the respondent’s particular expertise.
The actual time of response for the orientation meeting and policy questionnaire in which state staff have participated is 2 days (16 hours) and 8.125 hours respectively. The estimated time of response varies for the other study components, as shown in Table A.12.1.
Table A.12.1 Annual Burden Estimate
Affected Public |
Respondent Type |
Estimated # Respondents |
Annual Responses per Respondent |
Total Annual Responses |
Estimated Avg. # of Hours per Response |
Estimated Total Hours |
State, Local, and Tribal Agencies |
State SNAP staff - Preliminary Meeting - Policy Questionnaire - Interview - Extant Data Collectionb |
9
6
33 12 |
1
1
1 2 |
9
6
33 24 |
16.00
8.125a
1.50 25.00 |
144.00
48.75
49.50 600.00 |
District/County SNAP staff |
84 |
1 |
84 |
1.5 |
126.0 |
|
Call Center staff or centralized operation unit staff |
21 |
1 |
21 |
1.5 |
31.5 |
|
Local office SNAP staff |
154 |
1 |
154 |
1.5 |
231.0 |
|
Businesses (for-profit and not-for-profit) |
Vendor staff |
14 |
1 |
14 |
1.5 |
21.0 |
Community partner staff |
60 |
1 |
60 |
1.5 |
90.0 |
|
Individuals and Households |
SNAP participantsc |
120 |
1 |
120 |
1.667 |
200.0 |
SNAP eligible nonparticipantsd |
120 |
1 |
120 |
1.667 |
200.0 |
|
Nonresponders (focus group refusers)c |
720 |
1 |
720 |
0.0835 |
60.1 |
|
Total |
1,353 |
|
1,365 |
|
1,801.85 |
|
aIncludes 8 hours for completing the policy questionnaire and 1.125 hours to participate in a follow-up call for clarification or any questions regarding the data collected.
bAssumes two staff from each state will respond to data requests and spend 25 hours each.
cAssumes two focus groups in each state with 10 participants each. Focus group members will participate in a brief screening call, participate in the focus group, and receive a reminder call and letter prior to the focus group.
d The number of refusers is based on the assumption that, in order to have 240 respondents ultimately attend the focus groups, 480 persons will need to be recruited. And, in order for 480 persons to be recruited, twice as many persons, or 960, will need to be contacted initially. Focus group refusers will participate in a brief screening call or interview.
For interviews with state SNAP staff, district/county SNAP staff, SNAP call center staff or centralized operation units staff, local office SNAP staff, vendor staff, and community partner staff, the burden estimate is 1.5 hours—1 hour for the interview and 30 minutes for respondents to prepare for the interview. For compiling and sending the extant data, the annual burden estimate is 100 hours for each state (2 staff per State, two data submissions, each spending 25 hours per submission). For all focus groups participants, the burden estimate is 1.667 hours—1.5 hours to participate in the focus group and an additional 10 minutes for respondents to be screened, receive a reminder call, and read a reminder letter. For all persons who screen out or decline to participate in the focus groups, the burden estimate is .0835 hours (5 minutes) and includes the respondents’ time to be screened. Estimates for the interviews, record collection, and focus groups are all based on Mathematica’s prior corporate experience.
The total cost to respondents for their time in this collection is $33,096.37 (Table A.12.2). To calculate the annualized cost to state and local agencies and business respondents, we used the mean hourly wage rate categories determined by the Bureau of Labor Statistics, May 2009, National Industry-Specific Occupational Employment and Wage Estimates. To calculate the annualized cost to SNAP participants, we used the federal minimum wage as of July 24, 2009.
Table A.12.2 Annual Cost to Respondents
Respondent Type |
Instrument Type |
Average Hours per Response |
Number of Respondents |
Frequency of Response |
Mean Hourly Wage Rate |
Cost to Respondent |
State SNAP Staffa |
Preliminary Meeting Survey In-person interview |
16.00
3.25 1.50 |
9
15 33 |
1
1 1 |
$23.67
$23.67 $23.67 |
$3,408.48
$1,153.91 $1,171.67 |
Extant data collection |
25.00 |
12 |
2 |
$23.67 |
$14,202.00 |
|
District/County SNAP Staffb |
In-person interview |
1.50 |
84 |
1 |
$21.68 |
$2,731.68 |
Call Center Staff or Centralized Operation Unit Staffb |
In-person interview |
1.50 |
21 |
1 |
$21.68 |
$682.92 |
Local Office SNAP Staffb |
In-person interview |
1.50 |
154 |
1 |
$21.68 |
$5,008.08 |
SNAP Community Partners and Vendor Staffc |
|
|
|
|
|
|
In-person interview |
1.50 |
74 |
1 |
$16.55 |
$1,837.05 |
|
|
|
|
|
|
|
|
Focus Group Membersd |
Focus group |
1.667 |
240 |
1 |
$7.25 |
$2,900.58 |
Total |
$33,096.37 |
|||||
a North American Industry Classification System (NAICS) 999200: State Government
b NAICS 999300: Local Government
c NAICS 624200: Community Food and Housing, Emergency and Other Relief Services
d Federal minimum wage as of July 24, 2009
Provide estimates of the total annual cost burden to respondents or record keepers resulting from the collection of information, (do not include the cost of any hour burden shown in items 12 and 14). The cost estimates should be split into two components: (a) a total capital and start-up cost component annualized over its expected useful life; and (b) a total operation and maintenance and purchase of services component.
There are no capital/start-up or ongoing operation/maintenance costs associated with this information collection.
Provide estimates of annualized cost to the Federal government. Also, provide a description of the method used to estimate cost and any other expense that would not have been incurred without this collection of information.
The total costs of this study include a firm fixed price contract with Mathematica for $1,890,6902, which includes design of the study and development of data collection instruments, data collection, analysis, report writing, and payments to participating states, plus time spent by the federal project officer (GS 13-Step 10) to manage data collection ($4,000). Annual contract costs for the study are as follows:
Year 1: $410,282 (September 2009-August 2010)
Year 2: $981,477 (September 2010-August 2011)
Year 3: $498,931 (September 2011-May 2012)
Explain the reasons for any program changes or adjustments reported in Items 13 or 14 of the OMB Form 83-1.
This is a revision of a currently approved collection. The current collection includes 1,007 participants and in this phase we are requesting 1,353 participants and the number of responses decreased from 1,812 to 1,365 thus causing the burden hours to decrease from 1,870 to 1,802 resulting in a program change. The reason for this program change is that this phase of the study contains substantive revisions which will build on the current data collection for “Enhancing Food Stamp Certification: Food Stamp Modernization Efforts. In addition to changes in the instruments (adding and deleting question which decreased the burden per respondent).
For collections of information whose results are planned to be published, outline plans for tabulation and publication.
Products resulting from information obtained in this data collection will provide FNS with the following: 1) updated state modernization profiles for participating states; 2) an analysis of each state’s experience with modernization, including state-by-state descriptions of the changes made, the reasons for those changes, the implementation process, and the successes and challenges encountered; 3) an examination of the potential impact of the modernization changes in each state; and, 4) a cross-state analysis, exploring common themes in terms of the rationale for specific changes, the implementation experiences, lessons learned, and potential impacts on outcomes.
Mathematica will deliver analyses derived from this data collection to FNS via two key deliverables: a final report and a final briefing. Both the report and the briefing will present key findings of the study in clear, nontechnical language that makes them understandable by a large audience. Table A.16.1 presents the schedule for delivering these products to FNS. Mathematica will qualitatively analyze the information from the data collection to prepare these deliverables. No complex quantitative analytical techniques will be employed in this project. The final report and the briefing will inform FNS officials, state SNAP administrators, and other stakeholders about the process of developing and implementing modernization initiatives and will identify successful strategies to replicate, challenges to overcome, and problems to avoid.
Table A.16.1 Data Collection, Analysis, and Reporting Schedule
Data Collection* |
4/1/2011 – 9/2/2011 |
Send letter to respondents |
Immediately after receiving OMB clearance |
Pilot test questionnaires |
Shortly after receiving OMB clearance |
Conduct site visits |
Spring and summer 2011 |
Data Analysis |
|
Final table shells |
8/19/2011 |
Complete data analysis |
11/11/2011 |
Final Report |
|
Draft report |
2/10/2012 |
Revised draft report |
3/16/2012 |
Updated revised draft report |
4/20/2012 |
Final report |
5/18/2012 |
Final Briefing |
3/30/2012 |
Public Use Files |
|
Draft file documentation and codebooks |
4/15/2012 |
Final file documentation and codebooks |
5/15/2012 |
* Assumes receipt of OMB clearance on or about March 31, 2011.
The final report, to be published on the FNS website (http://www.fns.usda.gov/ora/), will include a stand-alone summary of the purpose, methodology, key findings, and policy implications, as well as a short executive summary. The chapters of the final report will directly address each of the research objectives and questions using text, graphs, and illustrations. We will provide highlights of the six study states’ modernization activities and implementation experiences in the text and support them with complete state profiles, including a summary table of each state’s activities and a graphic illustration of the implementation time line in the appendices.
These analyses will explore differences in SNAP operations and outcomes before, during, and after implementation of modernization initiatives. Although it is useful to discuss implementation in these three distinct phases, in reality the lines between phases are often blurred. Some initiatives are piloted in a few offices or rolled out gradually across the state. In addition, it may take some time between the official beginning of a new initiative in a given location and its full implementation there. The situation becomes more complex when considering that each state in the study has implemented a variety of modernization activities, some at different times and others grouped together for simultaneous implementation. We will use information from the updated state profiles and other data sources to establish a best estimate of the actual start date for each modernization initiative. As these implementation dates will vary by state, the state-by-state analyses will focus on different cut-off points for each state.
Analysis of State Modernization Changes. We will conduct a systematic study of the administrative processes associated with SNAP modernization in each of the six states. This study will examine the implementation of modernization initiatives, the changing roles and responsibilities of staff and partners, and the relationship between modernization and stakeholder satisfaction. We will incorporate data from all the data collection activities, drawing on data from the reviews of documentation; in-person interviews with SNAP staff, CBOs, and vendors; focus groups with SNAP participants and eligible nonparticipants; on-site observations; and the updated state profiles. Our analysis of the administrative process has four key steps: (1) code site visit notes and other documents, (2) summarize information by interview type and location within the state, (3) summarize information across data collection activities, and (4) describe the administrative process. The result will be a comprehensive description of each state’s modernization initiatives, covering the development and implementation process, changing roles and responsibilities, stakeholder satisfaction, key challenges, and successes.
Potential Impacts of Modernization Changes. Much of this quantitative analysis will draw on the administrative data sources (from the state data extractions). We will explore potential impacts of modernization by examining key SNAP outcomes before, during, and after implementation of modernization initiatives. This type of design—essentially a pre-post design—will lack the causal validity needed to conclude that any changes in outcomes were caused by the modernization changes (or that outcomes that have no change were unaffected by the modernization changes). Still, when viewed in light of these limitations, the analysis of potential impacts can prove informative when trying to understand the role that modernization potentially plays in participation, payment accuracy, and program costs. We will examine whether evidence of intended outcomes, such as cost reductions and increased program access, is apparent. Using qualitative information received from state and local staff and from the focus groups, we will examine whether evidence exists of unintended outcomes, such as reduced access, higher-than-anticipated costs, or increased application-processing time.
The specific outcomes we will examine include application approvals and denials, application processing time, participation levels, payment errors, program costs, and stakeholder satisfaction. We will examine other outcomes if state-level data are available. The analysis of potential impacts on client satisfaction will rely primarily on information obtained through focus groups, interviews with state staff, and interviews with community partners. And for all outcomes other than client satisfaction, we will examine trends in the outcome data to explore whether disruptions in the trends occur after key modernization changes.
Analysis of Cross-State Experiences. We will synthesize the analyses of the administrative processes and program outcomes in each state into a comprehensive cross-site analysis. This analysis will explore common themes in terms of the rationale for specific changes, implementation experiences, lessons learned, and the potential impacts on outcomes. When applicable, we will use findings from the analysis of outcomes to support discussions of the administrative process, and vice versa. The cross-site analysis will explicitly examine similarities and differences in program features across states. The primary analytic tool used to identify similarities and differences across states and to identify common lessons learned will be theme tables. For each modernization component that occurs in two or more states, we will construct a theme table in which the rows are key research questions. The columns of the table will represent each state with this component and the individual cells will summarize the response for that state. We will also explore how key outcomes changed across states in relation to the timing of modernization activities that are common across multiple states in the study.
If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.
The agency will display the OMB approval number and expiration date on all instruments.
Explain each exception to the certification statement identified in Item 19 “Certification for Paperwork Reduction Act.”
There are no exceptions to the certification statement.
1 The study will examine changes in application approvals and denials, application processing time, participation, payment accuracy, and program costs over the course of the modernization process.
2 This figure excludes an additional incentive amount of up to $13,000 over four of the contract tasks.
| File Type | application/msword |
| File Modified | 2011-06-28 |
| File Created | 2011-06-28 |