NSWP-III Pretest - SLT (IC 1 of 2)

Third National Survey of WIC Participants (NSWP-III)

Appendix A2

Revised Local WIC Agency Survey

Local WIC Agency Survey

Overview

The Local WIC Agency Survey will draw a sample of approximately 1,200 local agencies from the list of all local WIC agencies. The Local WIC Agency Survey will focus on the services that the local agency (LA) provides to the WIC participants, as well as the infrastructure of the WIC agency itself, including the (1) structure of the agency, (2) clinics and sites under the local agency, (3) income eligibility procedures, (4) certification procedures, and (5) food instrument or food distribution procedures. Local WIC agencies will also be asked about the policies and practices at their sites. These questions will characterize the heterogeneity in site-level policies and practices across the nation. The Local WIC Agency Survey was created by incorporating and modifying questions from NSWP-II. Some questions are new to the NSWP-III survey.

Protocol

The research team will email an invitation letter to all officials in the sample to request their participation in the study. The invitation email will let the officials know that a paper questionnaire will be mailed to them within a week, and that if they want to answer the survey sooner, they can access it online. A simple URL with a secure login and password will be provided.

Several days after the invitation is emailed, the research team will mail an invitation letter requesting participation in the study. The mail package will include a hard copy of the questionnaire, as well as instructions on how to access the survey online using the same secure login and password provided in the invitation email. Any respondent choosing the web version of the questionnaire will have the ability to complete the survey in more than one sitting, since their answers are saved automatically. In addition, the login and password provided will grant access to the survey for more than one person. The mail package will be sent using USPS priority mail so that the mail piece stands out among other mail items that the official may receive.

The survey may take approximately 43 minutes to complete. Respondent instructions are included with data collection document listed below.

Pretest Protocol

The research team will mail a hard copy of the invitation letter and two hard copies of the questionnaire (one to keep and one to return) to nine of the selected LA directors. Other details of the protocol including sampling, recruitment, data collection and analysis are described in the supporting statement for Generic OMB Clearance No. 0584-0606.

OMB No. 0584-0606

Exp. Date 3/31/19

A2. Local WIC Agency Survey

Instructions for Reviewers

The Local WIC Agency Survey will be delivered to local WIC agency representatives via an email link to a Web survey. This paper version approximates the layout of the survey and includes notes indicating how the Web survey will automatically route the respondent to the appropriate questions or data entry forms (these notes appear in the paper version in RED, CAPITALIZED text but will not be visible to the respondent in the Web version).

The NSWP-III version of the Local WIC Agency Survey is based on the research questions presented in the Performance Work Statement (PWS). Whenever possible, questions from NSWP-II are used for NSWP-III if they address the research questions from the PWS. This approach allows for more reliable comparisons between the two studies. The survey is organized into the following modules:

Table 1: Local WIC Agency Survey Sections	Page
Identity	3
Residency	3
Income	4
Breastfeeding	5
Denied applicants	5
Location of certification	8
Certification staffing	8
Operations	9
Administration	9
Physical space	9
Services information	10
Staffing	11
Caseload	12
Technology	13
Retention	13
End survey	14

Instructions for Respondents

INTRO: Thank you for participating in the pretest of the Third National Survey of WIC Participants. This survey is sponsored by the United States Department of Agriculture Food Nutrition Service and administered by 2M Research Services and Abt Associates. Please refer to the accompanying cover letter for full details of the research effort. If you have any questions, please contact Paul Ruggiere at 1-817-856-0871, or by email at pruggiere@2mresearch.com.

This survey—along with surveys of State agencies and participants—is designed to provide FNS with additional information on policies and program operations, beyond those available from existing program sources.

You have been provided two copies of this survey—one to mail back to us and one to keep for your notes. Indicate your responses on the copy you send back to us. Your notes on the copy you keep may include your responses or any feedback you have regarding the content or clarity of the questions you could share with us during our debriefing interview. We are particularly interested in how well the questions were understood by you or any other feedback we should consider as we finalize the questionnaire for use with all local agencies administering the WIC program.

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB number. The valid OMB control number for this information collection is 0584-0606. It will take you, on average, 43 minutes to login and complete this survey.

SCREENER

We are surveying a random sample of local agencies to better understand the services that the local agency provides to WIC participants, as well as the infrastructure, policies and practices of the agency itself. The results of this survey will assist FNS in program operations.

S1. Does your local agency conduct certifications of WIC applicants, or does it serve as a purely administrative office, overseeing these functions at the clinic level?

• Agency to which this survey was addressed does certifications. [CONTINUE TO SURVEY]

Please contact Paul Ruggiere at 2M Research Services to clarify if you should fill out this survey.

Phone: 817-856-0871
Email: pruggiere@2mresearch.com

Phone: 1.817.856.0871

Email: pruggiere@2mresearch.com

ICsurvey@2MResearch.com

• Agency serves as a purely administrative office

• Not sure

Certification Policies

Identity

Q1. What types of documentation does your local agency accept as proofs of identity for a WIC applicant? (CHECK ALL THAT APPLY)

 Letter from government agency (including WIC) w/name form/letter

 Driver’s license, State ID

 Work, school, or bus pass ID w/photo & name

 Military ID

 Social Security card

 Voter’s registration card

 Foster placement letter

 Passport or immigration records

 Marriage license

 Birth certificate

 Crib card, hospital discharge papers, or hospital ID bracelet

 Immunization record

 Other: PLEASE SPECIFY

 Other: PLEASE SPECIFY

 Other: PLEASE SPECIFY

 Other: PLEASE SPECIFY

Residency

Q2. What types of documentation does your local agency accept to verify the residency of a WIC applicant? (CHECK ALL THAT APPLY)

 Driver’s license

 Current utility/tax bill, rent receipt, mortgage receipt, or lease receipt with name and address on it

 Letter from government agency (including WIC) w/name and address

 State or Tribal-issued license or ID w/name and address

 Postmarked mail from reliable third party with name and address

 Checkbook, bank statement

 Signed statement by applicant that he/she is victim of loss or disaster, or is homeless, a migrant person, or military personnel.

 Other: PLEASE SPECIFY

 Other: PLEASE SPECIFY

 Other: PLEASE SPECIFY

 Other: PLEASE SPECIFY

 Other: PLEASE SPECIFY: __________________________

Income

Q3. Which of the following documents satisfy the income documentation requirements of your local agency? Among those documents, rank the top three most-often provided documents (where “1” is most often provided, “2” is the second most often provided and “3” is the third most often provided):

Document		Satisfies Document Requirement (CHECK ALL THAT APPLY)	Three Most Frequently Provided Documents (RANK 1, 2, or 3)
a.	1st Paystub/earnings statement	
b.	W-2 form	
c.	IRS tax return	
d.	Business records	
e.	Unemployment compensation (letter, check stub, copy of check)	
f.	Workers compensation (award statement, check stub, copy of check, statement from insurance company)	
g.	Social Security benefits (award letter, statement of benefits, check stub, copy of check)	
h.	State SSI or State disability insurance (notice of benefits, check stub, copy of check)	
i.	Public assistance or TANF (notice of benefits, check stub, copy of check)	
j.	Energy assistance (notice of benefits, check stub, copy of check)	
k.	Alimony or child support (copy of check, agreement, divorce/separation decrees, court order)	
l.	Any government or private pension, annuity, or survivor’s benefits (notice of benefits, check stub or copy of check)	
m.	Estate or trust earnings statement	
n.	Interest or dividends statement	
o.	Savings account earnings statement	
p.	Veteran’s payments (notice of benefits, check stub, copy of check)	
q.	Military pay (leave and earnings statement, check stub, copy of check)	
r.	Other documents____________________________	
s.	Other documents____________________________	
t.	Other documents____________________________	
u.	Other documents____________________________	

Q4. Which of the following satisfy the program participation documentation requirements for automatic or adjunctive eligibility of your local agency? Among those documents, rank the top three most-often method used (where “1” is most often, “2” is the second most often, and “3” is the third most often).

Document		Satisfies Documentation Requirement (CHECK ALL THAT APPLY)	Three Most Frequently Method Used (RANK 1, 2, or 3)
a.	Valid program or member ID card	
b.	Award letter or notice of benefits	
c.	Active program voucher	
d.	Electronic access	
e.	Other: PLEASE SPECIFY:________________________	

Breastfeeding

Q5. In your estimation, at what ages are infants being certified to receive “fully” (rather than “partially”) breastfeeding food packages?

Infant Age at Certification	Percentage of Infant Certifications in the past 12 months
1-3 months
4-6 months
7-9 months
10-12 months
Total	100%

Denied Applications

Q6. Does your local agency keep information on denied applications?

• Yes

• No: PLEASE EXPLAIN: _____________________ [GO TO Q8]

Q7. What information on denied applications do you retain and how is it retained? (ANSWER B. AND C. ONLY IF A. IS CHECKED.)

a. Information Retained (CHECK ALL THAT APPLY)		b. How Retained (CHECK ONE)		c. Where Retained (CHECK ALL THAT APPLY)
	Name of applicant		Paper copy only		WIC State Agency
			Electronic copy only		Your Local Agency
			Both paper and electronic		Sites/Clinics
	Address		Paper copy only		WIC State Agency
			Electronic copy only		Your Local Agency
			Both paper and electronic		Sites/Clinics
	Phone number		Paper copy only		WIC State Agency
			Electronic copy only		Your Local Agency
			Both paper and electronic		Sites/Clinics
	WIC applicant category		Paper copy only		WIC State Agency
			Electronic copy only		Your Local Agency
			Both paper and electronic		Sites/Clinics
	Reason for denial		Paper copy only		WIC State Agency
			Electronic copy only		Your Local Agency
			Both paper and electronic		Sites/Clinics
	Date of application		Paper copy only		WIC State Agency
			Electronic copy only		Your Local Agency
			Both paper and electronic		Sites/Clinics
	Date of denial		Paper copy only		WIC State Agency
			Electronic copy only		Your Local Agency
			Both paper and electronic		Sites/Clinics

Q8. Of applicants new to your WIC agency, what percentage in the past 12 months was denied certification? (SELECT ONE ANSWER IN EACH BOX)

• ≤10%

• 11 – 20%

• 21 – 30%

• 31 – 40%

• 41 – 50%

• 51 – 60%

• 61 – 70%

• 71 – 80%

• 81 – 90%

• 91 – 100%

Q8A. How confident are you in the range entered here?

 Very confident

 Somewhat confident

 Not very confident (i.e., a lot of guesswork involved)

Q9. Of WIC participants seeking certification at your WIC agency, what percentage in the past 12 months was denied certification? (SELECT ONE ANSWER IN EACH BOX)

• ≤10%

• 11 – 20%

• 21 – 30%

• 31 – 40%

• 41 – 50%

• 51 – 60%

• 61 – 70%

• 71 – 80%

• 81 – 90%

• 91 – 100%

Q9A. How confident are you in the range entered here?

 Very confident

 Somewhat confident

 Not very confident (i.e., a lot of guesswork involved)

Q10. Please specify the percentage of denials reported above that are attributable to the following eligibility problems. It is possible the percentages may sum to more than 100%, as applicants may be denied for more than one reason.

Reason for Denial		Percentage Distribution for New Applicants	Percentage Distribution for Certification
a.	Lack of documentation provided for identity
b.	Income ineligibility (over income limit)
c.	No nutritional risk
d.	Lack of documentation provided for residency
e.	Categorical ineligibility (i.e., not pregnant, child over 5 years, etc.)
f.	Other: PLEASE SPECIFY____________________________
Total		100%	100%

Q10A. How confident are you in the responses that were entered here?

 Very confident

 Somewhat confident

 Not very confident (i.e., a lot of guesswork involved)

Q11. Does your agency send an official letter of denial to applicants who are determined ineligible for WIC?

• Yes

• No

• Other: PLEASE SPECIFY: ____________________________

Q12. Can an applicant be screened and determined ineligible by telephone?

• Yes

• No [GO TO Q14]

Q13. IF Q11=YES AND Q12=YES

What is the percentage distribution of denials through screening phone calls versus formal, in-person applications in the past 12 months?

	Percentage of Certification	Percentage of Denials	Total
Screening phone calls
Formal in-person applications

Location of Certification

Q14. Does your agency offer provide certification at alternative sites (e.g., satellite or off-site clinics at a hospital, school, etc.)?

• Yes

• No [GO TO Q15]

Q14A. Which of the following WIC categories does your agency offer certification at alternative sites? (CHECK ALL THAT APPLY)

 Pregnant woman

 Postpartum woman

 Breastfeeding woman

 Infant

 Child

• None

Q15. Under what circumstances is certification provided at an alternative site? What is your agency’s policy toward providing certifications at an alternative site?

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

Certification Staffing

Q16. Does your staff regularly perform ALL of the certification-related tasks shown below?

• Assess categorical eligibility criteria

• Assess residential eligibility criteria

• Assess nutritional risk criteria

• Assess income/adjunctive income eligibility

• Issue benefits (vouchers/EBT cards)

• Yes, we have certification staff who perform ALL of those tasks.

• No, our certification staff specializes in one or more of those tasks. [GO TO Q17]

Q16A. What percentage of your certification staff performs ALL of those tasks?

__ __ __%

Q17. Among all full-time staff in your jurisdiction who conduct certification, what is their average monthly caseload?

______ participants

Operations

Administration

Q18. Is your local agency… (CHECK ONE)

• part of the State agency

• a local government entity administering the WIC program

• a tribal entity/organization administering the WIC program

• a non-profit organization that has been contracted to run the WIC program

• not a local agency, but rather a clinic under a local agency

• Other: PLEASE SPECIFY ___________________________

Physical Space

Q19. Which description most closely fits the structure in which your local agency is located? (CHECK ONE)

• Health department or medical clinic

• Social services office or agency

• Full service hospital

• School

• Head Start

• Community center

• Mobile clinic (van)

• Migrant health center and/or camp

• Indian Health Service facility

• Religious center

• Other: PLEASE SPECIFY ___________________________

Q20. Of the spaces available at your local WIC agency, excluding such things as hallways, bathrooms, kitchen, and storage closets, how adequate would you rate the following spaces for optimally delivering WIC services to your participants at this time? Please rate each type of room. [rooms marked “somewhat adequate” or “not at all adequate,” go to q20a]

Type of Room	Completely Adequate	Mostly Adequate	Somewhat Adequate	Not at All Adequate	N/A
Large waiting rooms/reception areas (greater than 15x15 feet)					
Small waiting rooms/reception areas (15x15 feet or smaller)					
Rooms, offices, or cubicles where clients are seen					
Large training/conference/multipurpose rooms					
Small training/conference/multipurpose rooms					
Administrative offices (no clients seen)					
Administrative cubicles (no clients seen)					
Laboratory (height/weight taking areas)					
Other: PLEASE SPECIFY ___________________________					

Q20A. Please explain why you selected [response from q20: “SOMEWHAT ADEQUATE” OR “NOT AT ALL ADEQUATE”] for the following rooms [type of room associated with “somewhat adequate” or “not at all adequate” response]:
_____________________________________________________________________

Q21. How would you rate the physical security of your local agency’s location (for example, protection from natural disasters such as fire, earthquake; burglary or vandalism of the site; unauthorized visitors; etc.)?

• Very safe (no incidents) [go to Q22]

• Safe (occasional minor incidents) [go to q22]

• Unsafe (occasional major incidents or frequent minor incidents)

• Very unsafe (frequent major incidents)

Q21A. Please explain why you selected [response from q21: “unsafe” or “very unsAfe”]:
_____________________________________________________________________

Services Information

Q22. Please enter the number of other WIC sites that operate under the authority of this local agency, by type.

Clinics (defined as a permanent location assigned to the WIC program; include main clinic)

Satellites (defined as a location such as a school, church or town hall that is only temporarily assigned the WIC program. WIC staff must carry their own files and equipment to the site each visit)

Mobile Units (a vehicle assigned to the WIC program that may make multiple stops to conduct certifications)

Q23. To what extent are the following services provided by your local agency at the various sites you specified in the previous question? [WEB SURVEY WILL SHOW CLINICS, SATELLITES AND/OR MOBILE UNITS COLUMN ONLY IF RESPONDENT HAS ANSWERED >0 IN Q22]

		Local Agency	Clinics			Satellites			Mobile Units
		Agency does this	All can do	Some can do	None can do	All can do	Some can do	None can do	All can do	Some can do	None can do
Conduct certifications											
Perform blood testing											
Take anthropometric measurements for height, weight, and body mass index (BMI)											
Conduct nutrition counseling											
Offer other educational seminars (e.g., on breastfeeding)											
Provide food instruments (vouchers/EBT cards)											
Provide referrals to other services											
Access WIC participant records electronically											

Q24. Across all of the clinics under your local agency, on average, how many days per week, is the clinic open to clients/applicants? DAYS

Q25. Across all of the clinics under your local agency, on average, how many hours per week, is the clinic open clients/applicants? HOURS

Q26. Across all of the clinics under your local agency, provide the opening and closing hours for a typical clinic in a typical week of operations in the table below.

Operating Hours	Monday	Tuesday	Wednesday	Thursday	Friday	Saturday
Opening
Closing

Q26A. Is the typical clinic in a typical week of operations open over the lunch hour?

• Yes

• No

Staffing

Q27. Across all of the clinics under your local agency, what is the number of full-time equivalent (FTE) staff who currently work at your local WIC agency or clinic? (IN CALCULATING, NOTE THAT IF THE STANDARD WORK WEEK IS 35-40 HOURS, FTE COULD BE COMPOSED OF 1 FULL TIME EMPLOYEE OR TWO OR MORE PART-TIME EMPLOYEES WHO, COMBINED, WORK THAT NUMBER OF HOURS.)

FTE Staff

Across all of the clinics under your local agency, please provide the following information for each position listed below. [PLEASE GIVE NUMBER]	Number of full-time staff (working more than 32 hours/wk)	Number of part-time staff (working less than 32 hours/wk)	Of the total combined full and part-time staff, what percentage have worked at one of your clinics 12 months or less?	Of all of the employees who have held this position in the past 12 months, what percentage have left your agency (are not working at any of your clinics)?	Across all of the clinics under your local agency, what percentage of these positions are currently vacant?
a. WIC Director or Clinic Supervisor			______%	______%	______%
b. Office Manager			______%	______%	______%
c. Administrative Support Staff			______%	______%	______%
d. Certification Specialist			______%	______%	______%
e. Registered Dietitian			______%	______%	______%
f. Degreed/Licensed Nutritionist			______%	______%	______%
g. Trained Nutrition Paraprofessional			______%	______%	______%
h. Registered Nurse/Physician Assistant			______%	______%	______%
i. Physician			______%	______%	______%
j. Social Worker/ Psychologist/ Therapist			______%	______%	______%
k. Other Professional (non-medical)			______%	______%	______%
l. Other: PLEASE SPECIFY___________			______%	______%	______%
TOTAL STAFF			______%	______%	______%

Q28. Across all clinics under your local agency, what percentage of all staff are bilingual or multilingual?
_____% of staff

Q29. What languages, other than English, are spoken by staff at one or more of the clinics under your local agency to assist in providing WIC services? (CHECK ALL THAT APPLY)

	NONE		Hmong		Spanish
	Arabic		Khmer		Swahili
	Cambodian		Korean		Tamil
	Cantonese/Mandarin		Laotian		Tagalog
	Farsi		Portuguese		Urdu
	French/Creole		Punjabi		Vietnamese
	Fulani		Russian		Other: SPECIFY
	Hindi		Somali

Q30. What difficulties does your local agency face in retaining, recruiting, and hiring staff? (CHECK ALL THAT APPLY)

• Salaries not competitive

• Salaries not commensurate with level of job duties

• Benefits not competitive

• Minimal training and job growth offered

• Workload too great

• Location of local agency unsafe

• Location of local agency hard to get to

• Physical space occupied by local agency crowded

• Low employee morale throughout agency

• Lack of support for WIC program from State

• Limited career path or opportunities for promotion

• Required skillset lacking in prospective employees

• Other: PLEASE SPECIFY __________________

• None of the above

Caseload

Q31. Currently, approximately how many clients are served by all of the clinics under your local agency combined per month?

CLIENTS/MONTH

Technology

Q32. Does the typical clinic under your local agency have on-site the necessary technology, equipment, supplies, etc., to do the following tasks?

	Yes	No	Don’t Know
Enter/access client certification information via a computer?			
b. Perform hematological tests?			
c. Take anthropometric measurements for weight and height, and to calculate BMI (body mass)?			

Retention

Q33 Does your local agency calculate retention rates for WIC participants?

• Yes

• No: PLEASE EXPLAIN: ___________________________[GO TO Q34]

Q33A. If your local agency does, please provide the formula or methodology you use to calculate that rate: __________________________________________________________________

Q33B. How often does your local agency calculate retention rates?

 Weekly

 Monthly

 Quarterly

 Annually

 Other: PLEASE SPECIFY _________________________________

Q33C. Please provide your local agency’s retention rates for the past five Federal fiscal years (FY).

FY 2012	FY 2013	FY 2014	FY 2015	FY 2016

Q34. Does your local agency use any alternative or estimate measure for retention (for any category of participant)?

• Yes

• No [GO TO Q35]

Q34A. Please describe this measure: _______________________________________

Q34B. What does your local agency use it for? _______________________________________

Q35. Across all clinics under your local agency, what are the average WIC participant retention rates of your agency and service delivery sites, by eligibility category for the most recent Federal FY 2016?

Categories	Clinics
Pregnant woman	___%
Postpartum woman	___%
Breastfeeding woman	___%
Infant	___%
Child	___%

Q36. Does your local agency attempt to contact pregnant women who miss their first appointment (to apply for participation in the program) in order to reschedule the appointment?

• Yes

• No
  
  

Q37. Does your local agency, or any of the clinics under your local agency, do any of the following to increase WIC participant retention rates? (CHECK ALL THAT APPLY)

• Advertise via traditional delivery channels (including on television, movie theaters, internet, print publication materials, radio, gas stations, etc.)

• Post social media advertisements (Facebook, Pinterest, Twitter, etc.)

• Send first birthday card to WIC caregivers on child’s first birthday

• Text message appointment reminders

• Provide transportation to and from sites

• Provide childcare onsite

• Encourage current participants to invite eligible family and friends to enroll and remain active

• Encourage healthcare professionals (doctors, nurses, midwives, etc.) to inform eligible women to enroll and remain active

• Decrease stigma associated with WIC participation: PLEASE EXPLAIN: ________________

• Other: PLEASE SPECIFY______________________________________________________

[SUBMIT]

End Survey

Thank you for participating in this survey!