Appendix A3 WIC NATS Sampling Approach

Appendix A3. WIC NATS Sampling Approach

WIC Nutrition Assessment and Tailoring Study (WIC NATS)

Sampling Approach

Background

The Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) Nutrition Assessment and Tailoring Study (WIC NATS) requires data collection from four respondent groups: (1) WIC State Agencies (SAs); (2) WIC Local Agencies (LAs); (3) WIC Clinics; and (4) WIC participants. The WIC NATS study team selected a non-probability national sample of SAs, LAs, and WIC clinics to participate in the original phase of remote site visits. We first selected 10 SAs, then 30 LAs within those 10 SAs, and ultimately 1 clinic site per LA.

WIC is administered in 89 SAs – which includes all 50 States, 33 Indian Tribal Organizations (ITOs), the District of Columbia, and five territories (the Commonwealth of the Northern Mariana Islands, American Samoa, Guam, Puerto Rico, and the U.S. Virgin Islands). The universe of WIC SAs for this study includes all SAs expected to meet the following eligibility criteria: having a fully-operational Electronic Benefit Transfer (EBT) system for at least six months, not engaging in any major overhauls of their Management Information System (MIS) during the study data collection window, and located within the contiguous United States. Across these estimated 54 SAs, there is a universe of an estimated 1,533 LAs and an estimated 3,669 clinics associated with those LAs.¹ The universe of eligible WIC participants is estimated at 5,355,980 participants.² Table 1 summarizes the respondent universe and estimated response rates.

Table 1. Respondent Universe by Respondent Category

Respondent Category	Size of Respondent Universe	Initial Sample1	Target completed cases	Response Rates
WIC SAs	54	13	10	77%
WIC LAs LA Survey Site Visits	1,533	370 36	306 30	83% 83%
WIC clinics2	3,669	36	30	83%
WIC participants Observation Interview	5,355,980	1,020 510	510 300	50% 59%
Overall Response Rate		1,439	856	59%

¹ Does not include pretest respondents.

² Initial sample for WIC clinics includes the approximately 36 clinics that will be contacted to successfully recruit 30 clinics into the study. It is expected that all 30 clinics will fully participate in all site director and staff interviews.

State Agencies

Using WIC participant caseload and State Plan data, the eligible SAs were stratified by four stratification variables to capture the primary variation in WIC nutrition risk assessments at the SA level, while maintaining geographic diversity. The SA stratification variables are:

1. FNS region: One SA from each of the seven FNS regions plus an additional SA from the three regions with the largest WIC participant caseload using the most recent annual participation data available from the FNS’s WIC data tables: https://www.fns.usda.gov/pd/wic-program.

2. SA participation level (high or low): High and low participation is defined using the most recent annual participation data available from the FNS’s WIC data tables: https://www.fns.usda.gov/pd/wic-program. SAs above the median will be considered to be high participation level SAs and those below the median to be low participation level SAs.

3. Flexibility allowed in tailoring of food packages (high or low): High or low flexibility allowed in tailoring food packages, will be based on a review of State Plans to determine whether the SA allows LAs to develop specific individual food package tailoring guidelines (State Plan section IIB, question 2d). SAs that do not allow LAs to develop tailoring guidelines will be considered low flexibility SAs; SAs that do allow this will be considered high flexibility SAs.

4. MIS complexity (high or low): MIS complexity will also be determined from State Plan data. In section IIIC of the State Plan, SAs use a checklist of 25 WIC systems functional requirements and indicate which functions the SA system currently performs. High complexity/low complexity will reflect the number of items checked based on a “cutoff” between high and low complexity systems. SAs with the most functionality in their MIS will be considered high MIS complexity SAs.

To systematically select a diverse set of 10 SAs, the study team performed the following steps:

Step 1: Place all eligible SAs into one of 8 stratum based on participation level (high/low), tailoring flexibility (high/low) and MIS complexity (high/low). The 8 strata are shown in Table 2.

Step 2: Order the strata by the number of SAs in each, lowest to highest.

Step 3: In the first stratum, randomly select one SA.

Step 4: If the selected SA’s region is a region from which we will select only one SA, remove the remaining SAs in that region from the sampling frame.

Step 5: In the next stratum, randomly select one SA.

Step 6: Remove the remaining SAs in that region if the region has met its desired quota.

Step 7: Continue until we have selected a SA from each strata. (We will have selected up to 8 SAs at this point.)

Step 8: Re-order strata that still have SAs in them by the number of SAs in each, lowest to highest.

Step 9: Continue selecting from each stratum until we have selected 10 SAs.

Table 2. Strata for Selecting State Agencies

Strata	Enrollment	Tailoring	MIS
1	Low	Low	Low
2	Low	Low	High
3	Low	High	Low
4	Low	High	High
5	High	Low	Low
6	High	Low	High
7	High	High	Low
8	High	High	High

Local Agencies and WIC Clinics

In order to capture important variation in the implementation of the nutrition risk assessment at the LA level, we requested information from all LAs within the 10 selected SAs about their nutrition risk assessment process through an online survey cleared under the original WIC NATS ICR. Survey responses informed the selection of 30 LAs that represent the variability observed based on the following key characteristics:

1. LA size (caseload), constructed from WIC Participant and Program Characteristics 2018 data, defined as large, medium, or small.

2. Urbanicity (from the survey), defined as primarily rural, primarily urban, and primarily suburban.

3. Language use (from the survey). Classified into two categories:

   1. Greater than 10% of participants are Spanish speakers (yes/no)

   2. Greater than 10% of participants speak an “other” language (yes/no)

4. Modes of nutrition education offered (from the survey), defined as offering more than X modes (yes/no), with X determined based on the distribution of responses to the associated survey question.

5. Use of technology for nutrition education (from the survey), (yes/no).

The study team took the following steps to systematically select the sample of LAs:

Step 1: For each of the five characteristics listed above (size, urbanicity, language use, modes of nutrition education, and technology use), the study team determined the number of LAs with the characteristic. If each characteristic is a bucket with one or more LAs in it, then, given the list of characteristics, LAs can be in up to 6 buckets.

Step 2: Sort the characteristics by number of LAs with each from lowest to highest.

Step 3: Divide the characteristics into quartiles based on number of LAs with each characteristic.

Step 4: Randomly select LAs from each characteristic as follows:

• From the characteristics with the 1st quartile (those having the smallest number of LAs) randomly select 1 LA from each.

• From the characteristics in the 2nd quartile, randomly select 2 LAs from each.

• From the characteristics in the 3rd quartile, randomly select 3 LAs from each.

• From the characteristics in the 4th quartile, randomly select 4 LAs from each.

Table 3 shows the total number of LAs selected from each quartile.

Table 3. The LA Selection Process

Quartile	Characteristics in Each	Number of LAs Selected from Each Characteristic	Total Number of LAs Selected
1st	3	1	3
2nd	3	2	6
3rd	3	3	9
4th	3	4	12
Total			30

Selected LAs then provided the following information about each WIC clinic site operated by the LA:

• Monthly caseload for reference month

• Zip code

• Language use

• Percent of high risk participants

• Percent of participants with only one documented risk

• Days of the week usually open for new enrollments and recertification appointments 

From this list, one clinic per LA was selected for a site visit. To ensure that the study team will be able to observe up to 17 nutrition risk assessments at each clinic site, only sites open at least three or more consecutive days a week were eligible for selection. Selection of clinic sites was based on similarity of their characteristics with those of their LA (i.e., for LAs with large caseloads, the clinic with the largest monthly caseload was recruited). This ensures we maintain the characteristics for which we selected the LA as we select clinic sites. Table 4 shows the decision criteria for the purposive selection of clinics with the LA selection based on size, urbanicity, and language use.

For LAs selected based on modes of education and use of technology in providing nutrition education, all their respective clinic sites are assumed to have the same characteristics. Therefore, within these LAs, clinics were selected with varying percentages of high-risk participants and participants with only one documented risk. We sorted clinics by percentage of high-risk participants and selected those at the high and low end for inclusion. We sorted clinics in the remaining LAs by the percentage of participants with only one documented risk and selected those at the high and low end for inclusion.

Table 4. Decision Criteria for Selection of Clinics

LA Selection Criteria	WIC Clinic Selected
Large caseload	Site with largest caseload in the LA
Medium caseload	Site with caseload closest to the median caseload across the LA
Small caseload	Site with smallest caseload in the LA
Primarily urban	Randomly selected urban site (if more than one site is in an urban area)
Primarily rural	Randomly selected rural site (if more than one site is in a rural area)
Primarily suburban	Randomly selected suburban site (if more than one site is in a suburban area)
Greater than 10% of participants are Spanish speakers	Site with the largest percentage of Spanish speakers
Greater than 10% of participants speak an “other” language	Site with the largest percentage of “other language” speakers

1 WICprograms.org, WIC Program and Office Directory. https://www.wicprograms.org/; accessed January 23, 2020.

2 U.S. Department of Agriculture, Food and Nutrition Service. WIC Data Tables, 2018 National level annual summary. https://www.fns.usda.gov/pd/wic-program; accessed January 23, 2020.

5