SSB - WIC NATS_In-PersonRev Final 2022.2.1

Supporting Statement for OMB Clearance for the WIC Nutrition Assessment and Tailoring Study In-Person Site Visit Data Collection

Part B

Revision to OMB Control # 0584-0663: WIC Nutrition Assessment and Tailoring Study

February 1, 2022

Alexander Bush

Social Science Research Analyst

Office of Policy Support

USDA, Food and Nutrition Service

1320 Braddock Place

Alexandria, VA 22314

Email: Alexander.Bush@usda.gov

Telephone: (703) 305-2127

Table of Contents

B.1 Respondent Universe and Sampling Methods 3

B.2 Procedures for the Collection of Information 6

B.3 Methods to Maximize Response Rates and to Deal With Nonresponse 7

B.4 Test of Procedures or Methods to be Undertaken 9

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 11

Tables

B1-1. Respondent Universe by Respondent Category 4

B2-1. Standard Error of the Estimates 7

B4-1. Pretested Instruments by Respondent 9

B5-1. Individuals Consulted on Data Collection or Analysis 12

Appendices

Appendix A: WIC NATS Supporting Documents

1. Authorizing Statutes

2. Relevant WIC Policy and Guidance Documents

3. WIC NATS Sampling Approach

Appendix B: WIC NATS In-Person Data Collection OMB Burden Table

Appendix C: Information Collections and Instruments

C1. Site Director Interview Guide

C2. Nutrition Services Observation Form

C2a. Identified Risks Data Collection Form

C3. Staff Interview Guide

C4. WIC Participant Interview Guide

C4a. WIC Participant Interview Guide-Spanish

C5. Clinic Site Observation Form

Appendix D: WIC Clinic Site Visit and Participant Recruitment

1. Site Visit Email to WIC Clinic and Schedule Call

2. Telephone Call with WIC Clinic

3. Informed Consent for Site Director Interview

4. Informed Consent for Observation and Staff Interview

5. Study Brochure for WIC Participant

D5a. Study Brochure for WIC Participant-Spanish

6. WIC Participant Screener

D6a. WIC Participant Screener-Spanish

7. Informed Consent for Observation and WIC Participant Interview

D7a. Informed Consent for Observation and WIC Participant Interview-Spanish

8. Reminder Text for WIC Participant Interview to be Conducted Over the Phone

D8a. Reminder Text for WIC Participant Interview to be Conducted Over the Phone-Spanish

9. Reminder Call for WIC Participant Interview to be Conducted Over the Phone

D9a. Reminder Call for WIC Participant Interview to be Conducted Over the Phone-Spanish

Appendix E: Pretest Memorandum

Appendix F: Expert Review Panel Comments and Responses

Appendix G: Confidentiality Agreements and IRB Documents

G1. Statement of Confidentiality and Non-Disclosure

G2. Westat IRB Approval Letter

Appendix H: Comments from NASS

Appendix I: FNS Response to Comments from NASS

B.1 Respondent Universe and Sampling Methods

Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

B.1.1 Respondent Universe

In the original WIC Nutrition Assessment and Tailoring Study (WIC NATS), data was collected from four respondent groups: (1) WIC State Agencies (SAs); (2) WIC Local Agencies (LAs); (3) WIC Clinics; and (4) WIC participants. Under this proposed revision, the United States Department of Agriculture’s (USDA) Food and Nutrition Service (FNS) will utilize the same diverse sample of 30 clinic sites that were recruited under the original WIC NATS collection (OMB Control Number 0584-0663 WIC Nutrition Assessment and Tailoring Study; expiration date 04/30/2024). The proposed revision adds data collection from in-person site visits, where the data collection activities planned for the currently approved remote site visits under WIC NATS will be replicated for use with 30 WIC clinic sites for in-person site visits once WIC clinic sites safely resume in-person operations. Data collection during in-person site visits under this revision will include observing nutrition risk assessments and interviewing participants and staff. In this revision, data will not be collected from the SAs and the LAs.

WIC is administered in 89 SAs – which includes all 50 States, 33 Indian Tribal Organizations (ITOs), the District of Columbia, and five territories (the Commonwealth of the Northern Mariana Islands, American Samoa, Guam, Puerto Rico, and the U.S. Virgin Islands). The universe of WIC SAs for this study includes all SAs expected to meet the following eligibility criteria: having a fully-operational Electronic Benefit Transfer (EBT) system for at least six months, not engaging in any major overhauls of their Management Information System (MIS) during the study data collection window, and located within the contiguous United States. At the time the WIC NATS sample was selected, it was estimated that 54 SAs met the eligibility criteria, and within those SAs there was a universe of an estimated 1,533 LAs and an estimated 3,669 clinics associated with those LAs. The universe of eligible WIC participants is estimated at 5,355,980 participants. Table B1-1 summarizes the respondent universe and estimated response rates.

The study team will take many strategic steps to maximize response rates across respondent groups. This includes providing clear information about the study and flexible scheduling to all participants, using incentives for WIC participants, and sending reminders to complete phone interviews. A full description of efforts to improve response rates can be found in Section B3 of this supporting statement.

Table B1-1. Respondent Universe by Respondent Category

Respondent Category	Size of Respondent Universe	Initial Sample	Target completed cases	Response Rates
WIC clinics1	3,669	30	30	100%
WIC participants Observation Interview	5,355,980	1,020 510	510 300	50% 59%
Overall Response Rate		1,050	540	51%

¹ It is expected that all 30 clinics will fully participate in all site director and staff interviews.

B.1.2 Sampling Methods

As mentioned above, the in-person site visits conducted under this revision will occur in the same diverse sample of 30 clinic sites that were recruited under the original WIC NATS collection (OMB Control Number 0584-0663 WIC Nutrition Assessment and Tailoring Study; expiration date 04/30/2024). This will allow for better comparisons between information collected during remote and in-person site visits and will not require additional sampling of State agencies, local agencies, or WIC clinics. Details on the sampling of State agencies, local agencies, and WIC clinics into the original WIC NATS sample is described in Appendix A3.

During the site visits, the field researchers will screen (Appendix D6, D6a) an estimated 1,020 WIC participants in order to recruit approximately 510 WIC participants (about 17 per site) to provide informed consent (Appendix D7, D7a) and participate in the observation (estimated 50 percent participation rate).¹ Field researchers will review the WIC clinic schedule for the week of the site visit with the site director. To ensure that a diverse sample of participants and nutrition risk assessments are observed, the team will aim to recruit participants across appointment types (certification and recertification), and participant categories (pregnant women, postpartum women, infants, and children).

All observed participants will be asked to complete the follow-up interview (Appendix C4, C4a). Of the 510 that participated in the nutrition assessment observations, we estimate that around 300 will complete the follow-up interview (estimated 59 percent response rate).¹ WIC participants will be given the option of either completing their interview immediately following their appointment in-person at the clinic site or scheduling a later date to complete the interview over the phone. We estimate that 150 participants will complete the interview onsite after their appointment. We expect 255 participants will agree to schedule phone interviews, but that only 150 of those will complete their scheduled phone interviews. All 255 of those that agree to be interviewed by phone will receive a text message reminder just before the call (Appendix D8, D8a). Those that don’t answer the study team’s initial interview call will receive a follow up reminder call (Appendix D9, D9a).

B.2 Procedures for the Collection of Information

Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedure,

• Degree of accuracy needed for the purpose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Information on how information will be collected is described in Section A.2 of this request.

B.2.1 Statistical Methodology for Stratification and Sample Selection, Estimation Procedure, and Degree of Accuracy Needed for the Purpose Described in the Justification

As discussed previously, we will recruit clinic staff and WIC participants from the same sample of 30 clinics that was used for the WIC NATS remote site visit collection. Methods for stratification and sample selection were approved under the original WIC NATS collection and are attached under Appendix A3 for reference. WIC staff and WIC participants at these sites will be recruited during the site visit. No sampling will be involved, as we are limited to recruiting WIC participants scheduled for certification or recertification visits during the site visit. Once a WIC participant agrees to participate in the study (Appendix D7, D7a), we will recruit the staff member conducting the assessment and/or benefit tailoring to participate in the study (Appendix D4).

We expect that 50 percent of the WIC participants approached will consent to participate in the observation, and that 59 percent of those who allow us to observe their appointment will complete the follow-up interview.

Table B.2-1 below shows the expected standard errors of the estimates for various levels of the proportion estimates (p) and sample sizes. These expected standard errors account for the nested design (i.e., the sampling of WIC participants within sites) and assume a small intraclass (within-site) correlation of 0.01². With 510 appointment observations and 300 participant interviews, the expected standard errors range from about 0.014 to 0.030. For example, for a proportion estimate of 90 percent and a sample size of 300, a 95 percent confidence interval is between 86.0 and 93.2 percent.

Table B2-1. Standard Error of the Estimates

Proportion Estimate (p)	Sample sizes
	300	450	600
0.1 (or 0.90)	0.018	0.015	0.013
0.2 (or 0.80)	0.024	0.020	0.018
0.3 (or 0.70)	0.030	0.025	0.022

B.2.2 Unusual Problems Requiring Specialized Sampling Procedures

There are no unusual problems that require specialized sampling procedures.

B.2.3 Use of Periodic Data Collection Cycles to Reduce Burden

This is a one-time study; concern regarding the periodicity of data collection cycles is not applicable.

B.3 Methods to Maximize Response Rates and to Deal With Nonresponse

Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.

We will contact each of the 30 clinic sites by email (Appendix D1) to schedule an informational call (Appendix D2). During this call, we will provide the clinic with an overview of the plans for in-person site visits and the data collection procedures and allow the site to ask any questions they may have about the study. We will also explain during the call that participating WIC clinics will be offered a gift (valued at $40) of children’s books and games for their waiting area as a thank you for time spent accommodating the study’s activities.

Across all site visits with participating WIC clinics, trained field data collectors in teams of two will screen 1,020 WIC participants by administering a screener (Appendix D6, D6a) and sharing the study brochure (Appendix D5, D5a). Given an expected response rate of 50 percent, we will identify 510 participants who consent (Appendix D7, D7a) to having the data collectors observe their nutrition risk assessment and benefit tailoring appointment. We believe the following steps will be sufficient to reach response rate targets by providing adequate information about the study, offering incentives to encourage participation, and allowing for flexibility of when and how the interview is conducted to meet participant schedule restraints. In order to ensure the study is accurately conveyed to participants in recruitment, field researchers will be trained in the details of the study as a whole and will be able to respond to participants’ questions or concerns. The study team will offer a $20 incentive to WIC participants that both allow the observation (Appendix C2) and complete the follow-up interview (Appendix C4, C4a). For flexibility, participants will have the option of completing the interview via telephone at a time of their choosing within a week of their observed nutrition assessment appointment. Participants who elect to complete the survey by phone will be provided with a disposable cell phone pre-loaded with enough minutes to complete the interview, should they not have access to a telephone for the call. For all scheduled telephone interviews, we will send the WIC participant a reminder by text message (Appendix D8, D8a) and, if needed, a reminder phone call (Appendix D9, D9a). We will attempt to complete the interview within one week following the observation. We expect to complete 300 interviews, both in-person and over the phone, out of the 510 WIC participants who allow their nutrition assessment appointment to be observed during the site visit (response rate of 59 percent).

B.4 Test of Procedures or Methods to be Undertaken

Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

All data collection instruments were pretested (Appendix E). The pretest for the instruments used in this revision involved a total of 6 individuals (2 clinic directors, 2 clinic staff members and 2 WIC participants), as seen in Table B4-1. Burden associated with the pretest was accounted for in the original ICR and no additional pretesting was conducted ahead of this revision.

Table B4-1. Pretested Instruments by Respondent

Respondent Number	Pretest Respondent	Instruments Pretested by Respondent
1	Clinic director (State #1)	Site Director Interview Guide (Appendix C1)
2	Clinic director (State #2)	Site Director Interview Guide (Appendix C1)
3	WIC clinic staff member (State #1)	Staff Interview Guide (Appendix C3) Nutrition Services Observation Form (Appendix C2)
4	WIC clinic staff member (State #2)	Staff Interview Guide (Appendix C3) Nutrition Services Observation Form (Appendix C2)
5	WIC participant (State #1)	WIC Participant Interview Guide (Appendix C4)
6	WIC participant (State #2)	WIC Participant Interview Guide (Appendix C4)

Two WIC clinics were nominated by their respective LAs and selected to pretest the site visit data collection instruments. At on-site visits with each of these clinics, the site visit staff pretested the Site Director Interview Guide (Appendix C1) with the clinic director. The site visit also included pretesting of the Staff Interview Guide (Appendix C3) and Nutrition Services Observation Form (Appendix C2) with two staff members who conduct nutrition risk assessment and benefit tailoring activities. Additionally, during the site visits the WIC Participant Interview Guide (Appendix C4) was pretested with two WIC participants, one from each pretest clinic, who were present for certification and recertification visits. The WIC participants were recruited at random during the pretest site visit.

The Site Director, Staff, and WIC Participant Interview Guides were tested to ensure that the respondents interpreted the questions as intended and could easily respond, that the interview guide was easy for the interviewer to administer, and to verify the burden estimates. Trained interviewers reviewed the instruments question by question with the respondents, observed and documented any issues that arose for either respondents or interviewers, and discussed any points of difficulty with respondents. The Nutrition Services Observation Form was tested to ensure that observers can complete the form as intended and were not missing any key elements of the visits and clinic flow.

Following the pretests, analysts reviewed notes from each interview and observation, and summarized themes and patterns within the data. In particular, analysts focused on problems with the instruments, including areas where the respondents demonstrated confusion, hesitation, or uncertainty. Themes and patterns were organized, evaluated, synthesized, and summarized into the WIC NATS Pretest Memorandum (Appendix E). The attached WIC NATS Pretest Memorandum also describes specific comments and recommendations from the pretest respondents as well as how these were used to refine the survey, interview guides, and observation forms. The pretest verified the initial burden estimates for the instruments. Note that the WIC NATS Pretest also includes the Local Agency Web Survey, Clinic Site Information Form, and the Clinic Observation Guide; these instruments were cleared under the currently approved WIC NATS collection, but additional burden for these is not needed in this revision.

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Table B5-1 presents a summary of individuals consulted on instrument design, data collection, and/or analysis. The protocol and instruments were developed and reviewed extensively by FNS and the Westat study team—which includes researchers that Westat sub-contracted with from Insight Policy Research, PHFE WIC, and Gabor and Associates. Westat, Insight Policy Research, PHFE WIC, and Gabor and Associates will also be responsible for the collection and analysis of the study’s data, in coordination with FNS. The three members of the Expert Review Panel reviewed the data collection procedures and the data analysis plan. The statistical procedures included in this information request have also been reviewed by Jennifer Rhorer with the USDA National Agricultual Statistics Service (NASS). The comment from NASS is included in Appendix H, and the FNS response isprovided in Appendix I.

Table B5-1. Individuals Consulted on Data Collection or Analysis

Staff	Title	Contact Information (phone or email)
Westat (contractor)
Laurie May, PhD	Vice President	301-517-4076
Mary Gabay, MS	Senior Study Director	301-294-2811
Christine Borger, PhD	Senior Study Director	301-294-2072
Thea Zimmerman, MS, RD	Senior Study Director	240-314-2413
Consultants and Sub-Contractors
Vivian Gabor, MPH	Principal, Gabor & Associates	202-841-3071
Shannon Whaley, PhD	Director of Research and Evaluation, PHFE WIC	626-233-0798
Betsy Thorn	Associate Director, Insight Policy Research	703-504-9488
Expert Review Panel
Joni Geppert, MPH, RD, LN	Senior Epidemiologist Minnesota Department of Health	651-201-3632
Angela Odoms-Young, PhD	Associate Professor, Kinesiology and Nutrition University of Illinois at Chicago	773-391-3358
Sarah Jane Schwarzenberg, MD	Professor, Department of Pediatrics University of Minnesota Masonic Children’s Hospital	612-624-1133
FNS Staff
Alexander Bush, MPH Courtney Paolicelli, DrPH	Social Science Research Analyst Lead Social Science Analyst	703-305-2127 703-605-4370
NASS Staff
Jennifer Rhorer	Mathematical Statistician, Methodology Division	202-720-3026

1 Based on response rates seen in the WIC Infant and Toddler Feeding Practices Study-2: Infant Report (OMB Control No: 0584-0580 WIC Infant and Toddler Feeding Practices Study-2 (WIC ITFPS-2); Expiration Date: 03/31/2022), available at https://fns-prod.azureedge.net/sites/default/files/ops/WIC-ITFPS2-Infant.pdf

2We assume a small intraclass correlation based on work previously conducted for the WIC program by Westat for the WIC Infant Toddler Feeding Practices Study 2 (OMB Control No: 0584-0580 WIC Infant and Toddler Feeding Practices Study-2 (WIC ITFPS-2); Expiration Date: 03/31/2022). In that study, intraclass correlation estimates of 0.01 were calculated for the variables WC21 (which asks if the mother made a change due to something she learned at WIC, a question asked at 3, 13, and 24 months) and I7_KA28 (which asked if WIC allows the mother to buy the types of food she would usually feed her child).