SS_Part B_Revised

Supporting Statement for the Evaluation of Alternatives to Improve Elderly Access to SNAP, Part B

Kameron Burt

Office of Policy Support

Food and Nutrition Service

U.S Department of Agriculture

3101 Park Center Drive

Alexandria, VA 22302

Revised Submission

Table of Contents

PART B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS 1

B1. Respondent universe and sampling methods 1

B2. Procedures for the collection of information 3

B3. Methods to maximize response rates and to deal with issues of nonresponse 14

B4. Tests of procedures or methods to be undertaken 16

B5. Individuals consulted on statistical aspects of the design and who will actually collect or analyze the information for the agency. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency. 17

ATTACHMENT A: LEGAL AUTHORITY Statutes And Regulations

Attachment B: RESEARCH DESIGN AND references

attachment C: exploratory study MEMOrandum

Attachment D: study of elderly participant perspectives REcruitment materials, screening, and consent forms

Attachment D.1: CBO Recruitment Email

Attachment D.2: Interview Recruitment Flyer

Attachment D.2.a: Interview Recruitment Flyer, Spanish

Attachment D.3: Interview Recruitment Phone Script

Attachment D.3.a: Interview Recruitment Phone Script, Spanish

Attachment D.4: Focus Group Recruitment Flyer

Attachment D.5: Interview Screening Form

Attachment D.5.a: Interview Screening Form, Spanish

Attachment D.6: Interview Consent Form

Attachment D.6a: Interview Consent Form, Spanish

Attachment D.7: Focus Group Consent Form

Attachment E: PUBLIC COMMENTS, NASS COMMENTS, AND FNS RESPONSES TO PUBLIC AND NASS COMMENTS

Attachment E.1: Academy of Nutrition and Dietetics Comments

Attachment E.1.a: FNS Response to Academy of Nutrition and Dietetics Comments

Attachment E.2: The State of Florida’s Department of Children and Families Comments

Attachment E.2.a: FNS Response to the State of Florida’s Department of Children and Families Comments

Attachment E.3: Meals on Wheels America Comments

Attachment E.3.a: FNS Response to Meals on Wheels America Comments

Attachment E.4: Jean Public’s Comments

Attachment E.4.a: FNS Response to Jean Public’s Comments

Attachment E.5: NASS Comments

Attachment E.5.a: FNS Response to NASS Comments

Attachment F: Sample Sizes, Estimated Burden, and Estimated Cost of Respondent Burden

ATTACHMENT G: Study of State Interventions Interview Protocols

Attachment G.1: State and Local SNAP and Partner Agency Administrator Interview Protocol

Attachment G.2: State and Local SNAP and Partner Agency Staff Interview Protocol

Attachment G.3: Community-Based Organization Interview Protocol

Attachment H: study of elderly participant perspectives interview and Focus group Protocols

Attachment H.1: Elderly Participant Interview Protocol Moderator Guide

Attachment H.1.a: Elderly Participant Interview Protocol Moderator Guide, Spanish

Attachment H.2: Elderly Participant Focus Group Protocol Moderator Guide

ATTACHMENT I: Study of State Interventions Recruitment Materials

Attachment I.1: Memorandum of Understanding

Attachment I.2: Study Description

Attachment I.3: Frequently Asked Questions

Attachment I.4: FNS Letter of Support

Attachment I.5: Initial Email

Attachment I.6: Follow-Up Email

Attachment I.7: Telephone Call Script: Follow-Up to Initial Email

Attachment I.8: Second Phone Call Topics

Attachment I.9: CBO Recruitment Email

Attachment J: Interview guide Test memo

ATTACHMENT K: STATE SELECTION MEmorandum

PART B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

B1. Respondent universe and sampling methods

Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The study will select respondents in two steps. First, ten States will be selected to participate in the study. Within each study State, key administrators of SNAP and partner agencies will be identified, as well as staff members and CBOs who can provide a range of perspectives on the State’s experience implementing interventions. In addition, within each study State, the study team will select elderly SNAP participants, non-participating applicants, and eligible non-participants for conducting interviews and focus groups.

The States proposed for inclusion in the study were selected, using a non-statistical approach to meet several major criteria. First, the States have adopted at least one of the interventions identified as being of special interest for improving elder SNAP access (as identified in Section A-1). Second, the study States were selected to vary in terms of SNAP elder participation rates (high participation States trending higher, high participation States staying constant, low participation States trending higher, and low participation States staying constant) to ensure that the outcome to be explained (elder SNAP enrollment) has a high degree of internal variation that supports meaningful research conclusions. Third, study States were selected to vary in how the program is administered (either at the State level or the county level). Fourth, States were selected to represent all FNS geographic regions (Attachment K, State Selection Memo).

Ten States that match the criteria set above will be invited to participate in the study: Alabama, Arkansas, Florida, Idaho, Massachusetts, Minnesota, Nebraska, New York, Pennsylvania, and Washington. Six alternate States will be considered for inclusion in the case that any of the primary States decline to participate in the study. If a State opts out, it will be replaced with an alternative State that matches its key selection criteria.

Even though a statistical process was not followed, the resulting list of potential study States is deemed to effectively capture essential dimensions of variation and to allow valid research conclusions about the implementation and effectiveness of interventions aimed at increasing elder access to SNAP.

For the Study of State Interventions, the study team will work collaboratively with States to identify respondents and select localities to visit. We will identify key administrators/directors of SNAP and partner agencies and the appropriate staff members and CBOs who can provide a range of perspectives on the State’s experience implementing interventions. We estimate that we will interview 23 State and local agency staff members and three CBO respondents in each of the 10 States in a one-time data collection activity (Attachment I).

For the Study of Elderly Participant Perspectives, the study team will randomly select 560 elderly SNAP participants, non-participating applicants, and eligible non-participants (56 in each of the ten participating States) from State administrative data and data purchased from a vendor. Potential respondents will be called two weeks in advance of the visit to be screened for interviews, until 280 interviews are scheduled. In each of the ten States, we will attempt to schedule fourteen SNAP participants, eight non-participating applicants, and six eligible non-participants per State to allow for an expected 30 percent attrition level. We will track interviews as they are scheduled to determine if the mix of respondents is becoming highly skewed compared to the universe along characteristics such as gender, age, ethnicity, and disability status. If it is, we will over-sample from subgroups that are underrepresented until the sample is more aligned with the universe in each local area.¹ If we are not successful in setting up enough interviews within a week of the planned visit, we will coordinate with CBOs to identify additional respondents to interview as a convenience sample. The focus group participants will be selected from a convenience sample due to resource constraints and the infeasibility of recruiting a random sample for focus groups in the short period time that the team will be on-site.

The administrative data collection associated with the Study of Intervention Effects will not employ any statistical methods for sampling. The data request will cover all SNAP participants during the selected time periods from each of the study States rather than a sample of participants.

B2. Procedures for the collection of information

Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedure,

• Degree of accuracy needed for the purpose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Study of State Interventions

The primary source of data for the Study of State Interventions will be the extensive information collected from respondents during three-day site visits to each of the selected States. Before the site visits, the study team will collect documents that may enhance our understanding of the interventions and their operations. The documents will serve as a key source of information on intervention design and on formal changes to policy and procedure. They also may provide critical information on outcomes. Examples of documents include the following: training manuals, policy guidance or directives issued for SNAP program staff, wavier applications, State documents outside of policy guidance such as relevant State legislation, cost neutrality or other reports required by FNS, other formal communication with FNS regarding implementation, and SNAP application forms.

The study team will work collaboratively with States to identify respondents, select localities, and schedule visits. Most of the interviews will be conducted with State or local SNAP staff members (Attachment G.1). To the extent we are able to garner their cooperation, we also will interview partner agency staff members in States with a CAP (where we will interview Social Security Administration [SSA] administrators involved in planning discussions and front-line staff members involved with the application and eligibility determination process) and in States that conduct data verification across agencies (where we will interview partner agency staff members involved in planning discussions about data sharing or staff members integral to the operations of the data systems). To obtain a variety of perspectives, the study team will interview staff members at various levels (including administrators, supervisors, and front-line staff members) and those responsible for the design, initial implementation, and operations of each intervention (Attachment G.2). The study team will work with the States to identify the administrators responsible for the design, implementation, and operations of each intervention and will work with local offices to identify supervisors and staff who are likely to be involved in the operations of the intervention and available to meet with us.

Researchers also will interview representatives of non-governmental community-based organizations (CBOs) that may refer elder individuals to SNAP or provide them with supplemental or alternative support such as meals or food baskets (Attachment G.3). Each CBO will select the representative that will be most appropriate to participate in the interviews based on their job function.

The study team will begin the process of analyzing data for individual States during production of the site visit summaries. The summaries will include key high-level take-away points, a timeline of key events directly related to each intervention and other events important for understanding the context in which the intervention is operating, details that support and supplement the key take-away points and timeline, and a list of documents obtained before and during the site visit.

The researchers will use the site visit summaries along with their raw notes to assess their State(s) on a number of constructs that describe the process, quality, and context of implementation.² Examples of analytic constructs include strength of leadership, structural support, SNAP staff buy-in and training, implementation process, quality assurance, policy context, and socioeconomic context. Within each construct, researchers will consider various indicators. For instance, within the construct addressing staff buy-in and training, indicators include staff satisfaction with the intervention and the frequency with which supervisors take corrective action on procedures related to the intervention.

Most indicators will require researchers to rank a State along a continuum. For instance, the staff satisfaction construct may require researchers to assign a value between 0 and 3, where 0 means that staff members were uniformly completely dissatisfied with the intervention and 3 means they were uniformly very satisfied. Other indicators may require a yes-or-no designation (using numeric values of 0 to represent a no response and 1 to represent a yes response). The value of the indicators within each construct will be summed to produce an overall ranking for the construct within each site. We do not intend to ask respondents questions about each indicator directly (though expect some will be discussed directly given the line of questioning in the protocols and as part of the natural flow of the conversations). We will analyze the rich qualitative data respondents provide throughout the interview (rather than based on a single response to a single question) to derive the indicators. These rankings will be for internal research team use and will not be shared with individual States.

To ensure that researchers are consistent in their assessments, the study team will develop clear definitions for each indicator and its associated rankings along with guidelines on what to consider in assigning values. All indicators will be assessed using a Likert scale (for instance, values for leadership engagement may include “very engaged,” “somewhat engaged,” and “not at all engaged”). The study team will review the definitions and guidelines during site visitor training. Each researcher will individually complete an assessment for each intervention in each State he or she visited. Site visit teams will then compare and contrast their individual responses and collaborate to come to consensus on each indicator and therefore each intervention assessment overall.

If the study team observes different effects in States with similar interventions, they will use this comparison to understand the potential reasons for the different effects. For example, if two States have a similarly structured intervention, and one of those States experienced an increase in the elder SNAP caseload after its implementation and the other did not, the comparative analysis in the Study of State Interventions will help to explain the discrepancy.

Study of Elderly Participant Perspectives

Our primary data sources will be interviews with elderly participants, focus groups with elderly participants, and field notes produced each day on-site to document observations and identify emerging themes. Two experienced interviewers will co-lead data collection activities, with each co-lead overseeing recruitment and data collection for interviews and focus groups in five of the selected States. Interviews will be conducted with 200 elderly individuals, and 8-10 focus groups will be held with approximately eight individuals each. Topics to be covered in the interviews include the following:

• Participant background;

• Application process (if applicable);

• Eligibility determination (if applicable);

• Receiving benefits (if applicable).

Topics to be covered in the focus groups include the following:

• Local sources of food and food assistance;

• Awareness of SNAP;

• Reputation of SNAP;

• Suggestions for improvement.

All data collection materials are included in Attachment H: Elderly Participant Perspectives Guides and Protocols.

Study of Intervention Effects

Discussions with States about administrative data will begin during the process of recruiting them into the study to ensure their complete understanding of what participation in the study will entail and to provide them with ample time to develop procedures to collect data (Attachment I.2 and I.3). The study team has developed standardized communication tools and templates for data sharing agreements that will guide these discussions (Attachment I.1). Data-sharing agreements will specify the roles of each party, the records and specific variables to be provided by the State, the schedule for acquisition, logistical information on file transfer, and procedures for ensuring data security.

Within two weeks of securing a data sharing agreement from a State, a study team member will (1) request that the State assign a data liaison to be the study’s key contact for all issues related to administrative data; (2) request documentation of the State’s data system; and (3) assign a senior-level programmer to work with the State. The study team will be flexible on the file format and will accept data extracts or complete records.

Two types of administrative data will be requested: (1) data on applicants and applications, and (2) data on participants and SNAP cases. The former will provide information about how many and which types of elderly individuals seek SNAP after implementation of an intervention relative to before, and the latter will shed light on how caseloads changed after implementation. The request will include data such as case number, application information, benefit level, duration on SNAP, amount and sources of income and deductions, household size and composition, participation in assistance programs (Temporary Assistance for Needy Families, Medicaid, Supplemental Security Income), and personal characteristics (age, gender, race, citizenship status, educational attainment, and marital status). Study team members will work with each State to resolve any problems in supplying specific data elements and how each variable is captured, leveraging their experience in collecting State administrative data for other studies, many of which included States proposed for this study (Rowe et al., 2015; Kauff et al., 2014; Sama-Miller et al., 2014).

A longitudinal file will be built by requesting data from each study State for a period beginning 12 months before the implementation of an intervention and continuing through 12 months after implementation. In States with multiple interventions, multiple extracts for the period relevant to each intervention will be obtained. We will collect up to 23 data files across the 10 States. However, in States where the time periods for each extract overlap (or where there are no or only a few months between time periods), it may be easier for the State to provide a single extract that includes all months of data required. The study team will explore this option with States in an effort to minimize their burden and maximize efficiency in the data extraction process. The analysis will include only those interventions that were implemented by January 1, 2017.

A separate CITS model will be estimated for each of six interventions. This analysis will indicate the overall effect of each intervention across the States using it. In addition, the study team will conduct a series of CITS models for each intervention within each State. This analysis will allow an examination of the variation in the effects of a given intervention across States. It will also enable the study team to assess the effects of specific interventions within specific States, given the environment in which they were implemented, using data from the Study of State Interventions.

For all interventions, CITS models will include households containing at least one elder member (those who may be affected directly by the interventions) as the treatment group. Ideally, a comparison group of similar households that were not directly affected by the interventions would be identified and used in the analysis. However, that approach may only be feasible for the interventions that were implemented in select localities within the study States. If an intervention is implemented statewide in the selected States, there is likely no other age subgroup within the caseload that mimics the characteristics and participation trends of those age 60 or older. Therefore, households with no elder members (those who would not be affected directly by the interventions) will be the comparison group for interventions adopted statewide.

The CITS models will compare for both the treatment and the comparison group the trend in the outcome in the pre-period to the trend in the outcome in the post-period. The deviation from the pre-period trend for the treatment group minus the deviation from the trend for the comparison group is the estimated effect of the intervention. Specifically, the following model will be run:

where 𝑌_𝑎𝑡 is the outcome (for example, the number of elder SNAP households) for group a at time t; t is the time period centered at the last pre-period (for example, if there were four pre-intervention months and four post-intervention months, t would range from -3 to +4); 𝑇_𝑎𝑡 equals 1 for the treatment group and 0 for the comparison group; POST_k equals 1 for post-period k and 0 otherwise (in the above example, k would range from 1 to 4 and POST_k would equal 1 for each of the four post-intervention months and 0 for all pre-intervention months); 𝛼 and 𝛽 equal the intercept and slope of the pre-intervention trend for the comparison group; (𝛼 + λ) and (𝛽 + 𝛾) equal the intercept and slope of the pre-intervention trend for the treatment group; 𝛿₁, 𝛿₂, 𝛿₃, … represent the deviation from the trend for the comparison group in post-periods 1, 2, 3, etc.; θ₁, θ₂, θ₃, … represent the estimated effects in post-periods 1, 2, 3, etc.—the deviation from the trend for the treatment group minus the deviation from the trend for the comparison group; X_𝑎𝑡 is an optional vector of other characteristics for which the study team might want to control (such as the employment ratio, poverty rate, racial make-up of the senior population, minimum and maximum SNAP benefit for a family size of two, or other economic indicators that might affect the number of SNAP participants in a given time period and that can be easily obtained from publicly available data sources such as the American Community Survey (ACS) or the Current Population Survey-Annual Social and Economic Supplement); ΣS_n is a vector of State fixed effects (that is, a set of 0/1 dummy variables indicating which State the observation came from, where n is the number of States included in the model); and ε_𝑎𝑡 is an error term. In the models of individual interventions in individual States, ΣS_n will be dropped from the model.

Effects and their statistical significance will be presented for each intervention across and within States in easy-to-read tables and figures. The tables will show the unadjusted effects, regression-adjusted effects, and explanatory variables. The data will also be presented by graphing the outcome measures for the treatment and comparison groups over time. With this approach, deviations in the pattern of outcomes for the treatment group from the pattern of outcomes for the comparison group can be easily detected and understood by a broad audience. To facilitate a comparative analysis across States, a table will be created that presents effects and their statistical significance across States by intervention. To the extent effects vary across States for a specific intervention, other concurrent interventions in operation will be considered. Findings from the study of State interventions and the study of participant perspectives will be deployed to explain why States may have seen different effects.

Another important objective of the study is to explore potential interactions between various interventions and assess whether they amplify each other’s effect or if, conversely, they have joint unintended consequences that might hinder, rather than promote, program access. To accomplish this objective, the main analytical approach will be to employ a series of panel regressions whereby program outcomes for each State and each period are regressed on SNAP interventions, State and time fixed effects, and a vector of time-varying unit characteristics (controls). Models will be estimated on a combined dataset that pools data from all available States and time periods, according to the model described below:

where i identifies States; t identifies time periods (months); Y represents the outcome of interest (number of new applications, elder caseloads, number of churners, number of applicants using the medical deduction); Intervention 1, 2… are interventions whose effect is being estimated (dummy variables that equal 1 for the presence of an intervention in a given month and 0 for the absence of an intervention); W_it is a vector that contains interactions between interventions of interest (for example, in a model with interaction effects calculated for Intervention1 and Intervention2, W_it would include the term Intervention1_it*Intervention2_it); S₁, S₂, … are fixed effects (dummy variables for n-1 States, where n equals the number of study States); X_it is an optional vector of controls; and µ_it is an error term. The coefficients of interest are β₁, β₂, … and

Interaction effects and their statistical significance for each type of interaction effect will be presented. The tables will show interaction effects expressed as percentage change in the numbers of new applications, elder caseloads, churning rates, and deduction usage.

In addition, the average characteristics of elder participants on the caseload and those of elders applying to SNAP before and after the implementation of each intervention will be tabulated. The study team will assess individual-level demographic characteristics—such as age, gender, race, educational attainment, and marital status—as well as household characteristics—such as benefit level, income, medical expenses and deduction, household size and composition, and participation in other assistance programs (e.g. Temporary Assistance for Needy Families, Medicaid, Social Security Insurance). The results in tables will be presented side by side to facilitate comparative analyses, though these simple descriptive statistics will not allow us to attribute any observed differences to the intervention.

Collection and processing of administrative data will begin immediately upon receiving signed MOUs from States, approximately 1 – 3 months after OMB clearance, and is expected to conclude 6 months after OMB clearance. A memo will report on the quantitative analyses of State administrative data. Findings from the Study of Intervention Effects will be incorporated into a final report, which will integrate findings from the different study components. For example, the details of how each State and county implemented the intended interventions has far-reaching implications for understanding both the results of the interviews with elder participants and the findings from the quantitative analysis of administrative data.

B3. Methods to maximize response rates and to deal with issues of nonresponse

Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

It is possible that one or several of the 10 States that will be initially invited to participate in the study (see section B.1 above) will decline to participate. In each case, an alternate State will be selected that matches the State that declined with respect to the key selection criteria. This strategy will ensure that the States that will be eventually included in the study reflect the experience of the wider group of States that have adopted interventions aimed at increasing elder access to SNAP.

In arranging the interviews for the Study of State Interventions, the study team will work with respondents to determine the most convenient times and formats (group versus individual; phone versus in-person) to convene the interviews. The study team also will limit the interviews to approximately one hour to ensure that the data collection imposes only a modest burden on respondents. The study team will use separate protocols for each potential respondent type so that respondents are not asked about activities or issues that are not applicable to them. In addition, the study team interviewer will meet with in-person interview respondents in their own offices or at locations of their choice. Although we do not anticipate that many agency staff members will refuse to participate in an interview, if this does occur we will select another staff member with similar expertise and hierarchical level to limit bias resulting from interview nonresponse.

In arranging for interviews and focus groups for the Study of Elderly Participant Perspectives, the study team will schedule enough participants to allow for attrition, which we estimate to be around 30 percent. For the interviews, we will select a random sample of individuals to contact and then will monitor responses to ascertain whether particular subgoups are over-represented and, if so, we will begin to over-sample from under-represented groups to reduce bias in the final sample. However, given that this is a qualitative study that has small sample sizes in each location to begin with, it will not be possible to ever claim that the sample is fully representative of the population (nor is representativeness the most important determinant of rigor in qualitative research). For the focus groups, we will use a convenience sample, because it will not be feasible within the scope of the available project resources to recruit focus group participants randomly in the short time our teams will be on-site. The final analysis of interview and focus group data will acknowledge these limitations in the sampling strategy. Having rich qualitative data will enable FNS to better understand why SNAP enrollment rates are lower for those 60 years and older than for other age groups, how elderly participants perceive the SNAP enrollment and recertification processes, and how the elderly make decisions about applying or using SNAP under different policy environments (i.e. different combinations of policy interventions).

Upon receiving administrative data for the Study of Intervention Effects, the study team will run quality-control programs to assess whether files are readable, complete, and contain the expected observations and variables, and to identify (and work with States to reconcile) issues with missing data or inconsistencies. In addition, study team members will work with States to ensure that the data elements are interpreted accurately and a standardized file can be created. Some States that are unable to provide certain data elements will have missing values for those variables in the standardized file. Any differences or inconsistencies with the data will be documented. In some States, it may be necessary to construct new variables from the raw data provided in the extracts. All the code that involves creating new variables as well as the basic code constructing the file will be internally reviewed. Once the quality of the data is confirmed and each file is cleaned, the data will be merged into an analysis file.

B4. Tests of procedures or methods to be undertaken

Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

No pre-tests of the interview protocols for the Study of State Interventions will be conducted. Based on the experiences of the early interviews conducted, the study team will, if necessary, make minor modifications to the data collection procedures and protocols.

For the Study of Elderly Participant Perspectives, the study team conducted pretesting on the interview guide and submitted a memo with the results. The memo can be found in Attachment J. Six test interviews were conducted March 8–10, 2017, at a Senior Wellness Center in Washington, DC. No significant changes were made to the interview guide as a result of the test interviews. The testing process helped inform planning for recruitment, training of recruiters, and data analysis.

The Study of Intervention Effects involves only the analysis of administrative data that have already been collected by States, so there are no pre-tests that need to be conducted. The study team will leverage the experience gained in other projects accessing and processing administrative SNAP data—which includes knowledge of State data systems, existing code, and MOU templates—to build data collection infrastructures for this project.

B5. Individuals consulted on statistical aspects of the design and who will actually collect or analyze the information for the agency. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

FNS has contracted with SPR and its subcontractor Mathematica to collect and analyze the data for this study. An interdisciplinary team of researchers from SPR and Mathematica contributed to the design of the study. All consulted individuals are listed in Table B.5.

Table B.5: Individuals Responsible for Statistical Aspects and Data Collection and Analysis

Name	Title (Project Role)	Organizational Affiliation and Address	Phone Number
Hannah Betesh	Senior Associate (research design, data collection and analysis)	Social Policy Research 1333 Broadway, Suite 310 Oakland, CA 94612	(510) 788-2469
Elizabeth Brown	Researcher (qualitative data collection and analysis)	Mathematica Policy Research 1100 First Street, NE, 12th Floor Washington, DC 20002-4221	(202) 484-4680
Kameron Burt	Social Science Policy Analyst, Contracting Officer's Representative	Food and Nutrition Service 3101 Park Center Drive Alexandria, VA	(703) 305-2572
Ronald J. D’Amico	Senior Advisor, study design	Social Policy Research 1333 Broadway, Suite 310 Oakland, CA 94612	(510) 788-2484
Lisa Dragoset	Senior Researcher (quantitative data collection and analysis)	Mathematica Policy Research P.O. Box 2393 Princeton, NJ 08543-2393	(609) 945-3348
Catherine Benvie	Program Analyst	Food and Nutrition Service 3101 Park Center Drive Alexandria, VA	(703) 605-3205
Irene Fan	Mathematical Statistician	Summary, Estimation and Disclosure Methodology Branch National Agricultural Statistics Service U.S. Department of Agriculture	(800) 727-9540
Annelies Goger	Senior Associate (research design, data collection and analysis)	Social Policy Research 1333 Broadway, Suite 310 Oakland, CA 94612	(510) 788-2490
Jacqueline Kauff	Principal Investigator	Mathematica Policy Research 1100 First Street, NE, 12th Floor Washington, DC 20002-4221	(202) 484-5266
Madeleine Levin	Senior Associate (data collection and analysis)	Social Policy Research 1333 Broadway, Suite 310 Oakland, CA 94612	(510) 768-8277
Melissa Mack	Project Director	Social Policy Research 1333 Broadway, Suite 310 Oakland, CA 94612	(510) 788-2478
Libby Makowsky	Researcher (qualitative data collection and analysis)	Mathematica Policy Research 1100 First Street, NE, 12th Floor Washington, DC 20002-4221	(734) 794-8026
Barbara Murphy	Chief, SNAP Analysis Branch	Food and Nutrition Service 3101 Park Center Drive Alexandria, VA	(703) 305-2532
Marian Negoita	Principal Investigator	Social Policy Research 1333 Broadway, Suite 310 Oakland, CA 94612	(510) 788 -2477
Anne Paprocki	Associate (research design, data collection and analysis)	Social Policy Research 1333 Broadway, Suite 310 Oakland, CA 94612	(510) 768-8499
Gretchen Rowe	Senior Advisor (research design, data collection and analysis)	Mathematica Policy Research 1100 First Street, NE, 12th Floor Washington, DC 20002-4221	(202) 484-4221

1 Although we chose this approach to minimize sample bias that could occur with a straightforward convenience sample, the sample sizes in each local area will be too low to make general claims about representativeness regardless. Low sample sizes are customary in qualitative research and rigor in this case requires taking reasonable steps to reduce obvious sources of bias and being clear about the limitations of the sample in the analysis process. Because the goal of qualitative research is meaning rather than generalizability, the representativeness of the sample is a lower priority than it would be in quantitative research. Source: Mason, M. (2010, August). Sample size and saturation in PhD studies using qualitative interviews. In Forum qualitative Sozialforschung/Forum: qualitative social research (Vol. 11, No. 3).

2 This approach draws on concepts from implementation science which suggests that the quality of implementation of an intervention typically is driven by various factors including aspects of the organization delivering the intervention (such as organizational culture, climate, attitudes, understanding of the purpose of the intervention, and experience providing similar services); engagement of leadership and decision-making structures; the staffing model, supervisory structure, and service delivery supports (including staff training, administrative support, staff development activities); and existing infrastructure—such as data systems and organizational policies (Fixsen et al., 2005; Glisson, 2008; Rapp et al., 2010; Duggan & Supplee, 2012).