OMB Supporting Statement section B 01-29-2009

PART B. COLLECTION OF INFORMATION USING STATISTICAL METHODS.

B.1 Respondent universe and any sampling methods.

1. Overview of Sampling Approach

There will be data collection from five distinct populations: State WIC Agencies; Local WIC Agencies; active WIC Participants; denied WIC Applicants (new applicants deemed ineligible); and denied WIC recertificants (participants deemed ineligible for renewal). The collection efforts are described below:

• State WIC Agency Data To Be Collected: Macro will conduct a mail survey of all 90 State WIC agencies about their certification guidance to local WIC agencies; rebate arrangements for infant formula; breastfeeding support to local agencies; prevalence of nutritional risk factors; and State plans to implement the new VENA nutritional guidelines and the revised WIC food packages.

• Local WIC Agency Data To Be Collected: Macro will survey 500 randomly-selected local WIC agencies within the continental United States to administer an online survey covering agency procedures, general set-up and WIC clientele. The survey will be directed to the Director of the local agency.

• WIC Participant Data To Be Collected: Nationally representative information will be collected on 2400 participants covering their individual and household characteristics, WIC program participation, perceptions, and satisfaction. Participants will be chosen at random from a single-month sample of WIC food coupon issuances in February-March 2009. Selected recipients will receive a telephone survey. Half of these respondents will be randomly selected for an in-home interview. The in-home interview is needed for looking at eligibility documents (residence, household unit, income). Both the telephone survey and in-home interview will be offered in Spanish, as required. Spanish translations are in progress and will be submitted upon completion. A $20 gratuity will be paid to each respondent--similar to NSWP I.

These participants will be selected in two local agencies for each of the sampled areas (some states will be sampled multiple times) within a state. As shown in the sampling process, this works out to 80 local agencies.

• Denied Applicants (Denials) Data To Be Collected: A specified number¹ of Denied Applicants will receive a short telephone survey (Appendix F) about the denial of WIC benefits approximately one month after being denied eligibility.

• Denied Recertificants (Terminations) Data To Be Collected: A specified number² of Denied Recertificants will receive a short telephone survey (Appendix G) about the denial of WIC benefits approximately one to three months after being denied renewal of their eligibility.

Table A.16.1 summarizes the data collection approach.

2. Sampling Design

This design incorporates constructive features from the previous National Survey of WIC Participants (NSWP I)* with improvements in precision and lower cost. (*OMB Control #: 0584-0484; expiration date: 10/31/2000).

The current study will differ from NSWP I in that the previous study used certifications and certification periods as the major unit of analysis. This design measures active participants at a given point in time (participants issued food instruments for a given month) as the sampling unit. There is also a related component that samples Denied Applicants and Denied Recertificants who could not, of course, be sampled via food issuances since they were denied them. (See Appendices F and G).

The primary advantage of this approach is that it allows for an estimate of the proportion of redemptions nationwide that are in error and the proportion of unredeemed issuances. It relies on figures (the number of participants) for which there are official reports on a monthly basis and figures which are more likely to be accessible to local agencies and sites.

3. Sample Allocations and Precision

The sample sizes required for this study are driven by the required estimates of case error and improper payments. These requirements drive the sample size allocations and design.

Table 3a: Estimates of Error and Proposed Data Collection Mode

Parameter	Population	Assumed Estimate	Probability	Confidence Interval	Data Collection Mode
Error Rates	Combined Categories	10%	90%	2%	In-home
Error Rates	Separate Categories	10%	90%	5%	In-home
Other Estimates	Combined Categories	50%	95%	4%	Telephone
Other Estimates	Separate Categories	50%	95%	7.5%	Telephone

In general, the variance and, therefore, the standard deviation (σ) of a single dichotomous event (e.g., flip a coin and get “heads” or “tails”) reaches its maximum when the probability of the occurrence of the event is 0.5, but for the erroneous certification estimates we would use .10.

We express this as follows:

= 1.22 percentage points = 0.0122

^{= 0.0122}

^{= 0.0122}

n = .09/(.0122)²

n = 609

So we would need a sample size of 609 WIC participants, drawn from an infinitely large population of participants, for us to express 90 percent confidence that our estimate of misclassification will be within plus or minus 2.0 percentage points of the true value, no matter what value the estimate might take. Therefore, if we were sampling randomly without replacement from a very large number of participants, we would need a sample size of 609 to achieve an estimate with a power of 90 percent and a precision of plus or minus 2.0 percentage points.

However, it is not practical to draw participants randomly from around the contiguous states. This would mean visiting the homes of participants around the country and getting every agency to provide a list of participants. This is why clusters are being designed at different levels. States will be selected multiple times, two Local WIC agencies will be selected from a state for each time the state is selected, two clinics are selected from each local WIC agency and participants will be selected from each of five categories for each clinic. This means that the estimates will be affected by two factors—unequal weights and clustering. Factoring in a design effect, created by the necessity to do cluster sampling, the following sample requirements are produced:

Table 3b: Sample Requirements

Parameter	Population	Random Sample	With Design Effect Per Cell	Total Needed	Data Collection Mode
Error Rates	Combined Categories	609	1199	1199	In-home
Error Rates	Separate Categories	97	159	795	In-home
Other Estimates	Combined Categories	600	1798	1798	Telephone
Other Estimates	Separate Categories	171	409	2045	Telephone

The one estimate for which the confidence cannot be estimated is dollar error, since there is no source of a standard deviation. However, given the projected confidence interval for certification errors and the fact that variances for continuous variables tend to have lower CVs than those for associated dichotomous variables, it is assumed that the sample size will be more than sufficient for the intended purposes.

These numbers inform the design. At a minimum, 1,200 home interviews would be needed and over 2,000 telephone interviews. If one follows the procedure of NSWP I and selects 80 local WIC agencies, and conduct home-interviews with one-third of the cases sampled, it would make sense to sample 2,400 participants for the telephone interview and 1,200 of these for the in-home interview. That comes to 30 telephone interviews, 15 of which will be home interviews per Local WIC agency.

The above numbers refer to active enrollees as of the target month who will respond to the survey. A procedure is in place for replacement of Participants, Denials and Terminations who refuse to respond or cannot be located.

The table describing an initial sample assumes a form of order sampling. Individuals in each WIC category will be arranged in a random order list and approached in that order, replaced only when the data collector fails to locate that individual. Initial refusals will be followed-up, and once attempts are exhausted, the next individual in the list will be selected. This guarantees a random replacement without having to over-sample and obtain a surplus of respondents. Hence, the final sample is fixed, and the initial sample is the estimate of the number that will be contacted to produce the desired sample.

Applying the formula for sample size, we can calculate the sample size needed for each of the six estimate categories, as shown in the Exhibit below.

Table 3c: Sample Size by Category

Participant Category	WIC Participant Sample Sizes			In-Home Sample Sizes
	Number Selected	Assumed Response Rate	Number of Completes	Number Selected	Assumed Response Rate	Number of Completes
Pregnant	640	3/4	480	360	2/3	240
Breast-feeding	640	3/4	480	360	2/3	240
Postpartum	640	3/4	480	360	2/3	240
Infant	640	3/4	480	360	2/3	240
Child	640	3/4	480	360	2/3	240
Denied or terminated	432	3/4	324	243	2/3	162
Total	3,632	3/4	2,724	2,043	2/3	1,362

4. Sampling Procedures

States selected for the research were sampled using Probability Proportional to Size with Probability Minimum Replacement: Under this approach, States may be sampled more than once if their size (as measured by number of participants) is greater than the sampling interval. A total of 23 States were sampled; seven of them were sampled multiple times for a total number of 40 “hits”. In each hit, we will sample 2 local agencies for a total of 80; for purposes of the sample Indian Tribal Organizations will be treated as local agencies in the state in which they are located.

Table 4: Selected States, Regions, and Hits

Sampled State Agency	Region	Hits
Massachusetts	1	1
New York	1	3
Maryland	2	1
New Jersey	2	1
Pennsylvania	2	1
Virginia	2	1
Alabama	3	1
Florida	3	3
Georgia	3	2
North Carolina	3	1
Tennessee	3	1
Illinois	4	2
Indiana	4	1
Michigan	4	1
Ohio	4	2
Louisiana	5	1
Texas	5	5
Colorado	6	1
Kansas	6	1
Missouri	6	1
Arizona	7	1
California	7	7
Washington	7	1
Total 23 States	90	40

5. Estimation Procedures

For categorical variables, one can be 95 percent certain that survey responses are within 5 percentage points of the estimate in each cell. Any cell may be compared with another cell to detect significant differences. If the differences are of medium size (half a standard deviation or 5 points), the sample sizes are large enough to detect effects at the .01 level with power of .95, and most are sufficiently large to detect small effects (.2 standard deviations or 2 points) at the .05 level with power of .7. Small effects will not be detected in cells that use the finite population correction factor.

B.2 Procedures for the collection of information.

FNS envisions a distinct approach for each of the groups to be surveyed. Due to the complex and interwoven nature of this research pursuit, tasks are sometimes described for one group that pertains to the data collection of another.

State WIC Agency Survey

Macro will survey – by mail (or telephone, if preferred) -- the extant 90 “State” WIC organizations. These State agencies are located in all 50 U.S. states, Washington D.C., the 34 Indian Tribal Organizations (ITOs), Guam, Samoa, the Northern Marianas, Puerto Rico, and the Virgin Islands. The survey is attached in Appendix C.

FNS will notify its seven Regional Offices who, in turn, will inform State agencies about the research. In October/November 2008 State Agency Directors will be mailed a packet containing an introductory letter on FNS letterhead, a 10-page survey, and a pre-stamped, self-addressed envelope. Directors will receive follow-up telephone calls and emails to see if they received the packet or have any questions; they will be given the option to complete the survey by telephone, if it so conveniences them.

Letters Of Communication: In total, State WIC Directors will receive two letters (Appendix B). The first informs them of the overall study and asks for the names of all local agencies and the local agencies’ WIC participation numbers, broken out by category.

The second letter will be sent to States where a local agency has been selected for the Local Agency survey (n=500), and/or a local agency has been selected to be part of the WIC Participant survey (n=23). The letter, addressed to the State WIC Director, will advise them which agencies have been selected for the two data collections. We anticipate that a number of States will have local agencies in both categories. The letter also asks them to complete the State Agency survey enclosed.

Local WIC Agencies

Macro will randomly select 500 local WIC agencies to administer an online survey covering agency procedures, general set-up and WIC clientele. A letter informing them of their selection in the survey will be mailed in November 2008 to the Director of the local agency. It will contain the URL and a unique password that will save their answers as they go along—an important feature should they have to exit the survey and return later. Directors will receive follow-up telephone calls and emails to see if they received the letter; they will be given the option to complete the survey by telephone, if they prefer. A copy of the survey is provided in Appendix D.

Letters Of Communication: Two letters to local WIC Directors are included in Appendix B. The first (n=500) explains the study and gives them directions for completing the local agency survey online.

The second letter (n=80), to be mailed out the same day, will be sent to local agencies that have been selected to be part of the Participant/Denials portion of the survey³. The letter will ask for their assistance in assuring their WIC clients about the legitimacy and confidentiality of the research; and providing a list of food issuances during the month of October 2008. It will also give agencies instructions for keeping a list of Denied Applicants over the same time period.

Sometimes, a local agency may be chosen for both letters and tasks above, based on random selection.

WIC Participants, Denied Applicants (Denials) and Denied Recertificants (Terminations)

The three client populations—Participants, Denials and Terminations—will all be interviewed via telephone interviews. In addition, half of the Participants will be randomly chosen for an in-home audit. They will receive $20 compensation for the latter.

The first step will be to randomly select the 80 local agencies from State-provided lists who will be involved in this part of the research. Once the agencies are identified, FNS/Macro will ask the State agency—or the local agency if the State is unable—to extract a list of all food issuances and related participants for February-March 2009 for the 80 agencies. While only one month is required for the food issuances/participants sample, the previous month of food issuances is necessary in order to ascertain terminations, that is, participants whose certifications were ended.

Following a review of the WIC participant data provided by the State, Macro will conduct interviews as follows.

1. A telephone interview of 2400 active Participants (who received food instruments in February-March 2009). It will cover program eligibility and participation, individual and household characteristics, drivers of behavior, and perceptions of the WIC program. From these interviews, half (1200) of the Participants will be asked to take part in an in-home audit, for which they will be offered $20. (Survey instrument for both parts is in Appendix E).

The in-home audit will solicit documentation from them to support their WIC eligibility status—namely, valid identification, proofs of income (or adjunctive participation in qualified programs), and proof of residence. If respondents are unable to produce suitable documentation, they will be asked to sign a release form allowing the data collector to obtain that information from appropriate third party sources.

2. A very brief telephone interview of Denied Recertificants to verify if their denial was correct, using WIC’s own manual of procedures and eligibility guidelines. (See Appendix G for survey instrument).

3. A very brief telephone interview of Denied Applicants to verify if their termination was correct, using WIC’s own manual of procedures and eligibility guidelines. (See Appendix F for survey instrument).

Exhibit: Summary of Data Collection Approach

	1. Survey of State WIC Agencies	2. Survey of Local WIC Agencies	WIC Applicants
			3. Survey of Participants (Non-Expired)		4. Survey of Denied Certificants (Terminations)	5. Survey of Denied Applicants (Denials)
			Telephone Only	Telephone and In-Home
Sample frame, explained	Universe of WIC agen­cies and Indian Tribal Organization WIC offices	Mainland USA WIC offices	Active food issuances to WIC participants in mainland USA	Active food issuances to WIC participants in mainland USA	February 2009 food issuances to WIC participants that were cut off in March 2009	New applicants denied WIC certification in May 2009
Sample size	90	500	About 240 each of 5 categories= 1200 TOTAL To be replaced if necessary	About 240 each of 5 categories= 1200 TOTAL To be replaced if necessary	To be determined. Will be sampled at twice the rate of Participants in population	To be deter-mined. Will be sampled at twice the rate of Participants in population
Plans to communi­cate with source	Mail, email and telephone	Mail, email and telephone	Telephone	Telephone	Telephone	Telephone
Data to be collected	State guidance to local WIC agencies; and initiatives	Local agency operating procedures and set-up	Participant characteristics and behaviors	Participant characteristics and behaviors. Program eligibility	Program eligibility and demographics	Program eligibility and demographics
Number of respondents	90	500	1200	1200
Method of collection	Mail & Telephone	Online (or telephone/ mail)	Telephone	Telephone and In-person home audit	Telephone	Telephone
Special equipment needed	None	CAPI computers	CAPI computers	CAPI computers	CAPI computers	CAPI computers
Data collectors	None	None	Interviewers	Interviewers	Interviewers	Interviewers
Expected response burden	65 minutes	40 minutes	24 minutes	24 minutes, then 30-35 minutes	5 minutes	5 minutes
Incentive offered	None	None	None	$20	None	None
Fieldwork Period	May-June 2009	May-June 2009	July-September 2009	July-September 2009	July-September 2009	July-September 2009
Plans for non-English speakers	N/A	N/A	Trained interviewers in 3 languages	Trained interviewers in 3 languages	Trained interviewers in 3 languages	Trained interviewers in 3 languages
Quality control of data	Review submitted questionnaires for completeness		Interviewer training; QC checks on interviews and data submitted	Interviewer training; QC checks on interviews and data submitted	Interviewer training; Supervisor listening in	Interviewer training; Supervisor listening in

B.3 Methods to maximize response rates and to deal with non-response.Maximizing Participation

Researchers will endeavor to maximize response rates through conversations with the local WIC agencies and clinics. These agencies have earned the trust of WIC participants and are critical to the survey’s success. Their knowledge of the population they serve and barriers to participation as well as their active endorsement of the legitimacy and worthiness of the research effort will be critical; for it is important to implement the study without causing fear or program attrition. An informational/promotional flyer is already being planned.

Other methods include the hiring and training of highly skilled interviewers; having interviewers that speak Spanish and other foreign languages; assurances to potential respondents of the confidentiality of their individual answers; and multiple callbacks to reach respondents, varying the days and hours. (Callback script included in Appendix H.)

As a final measure, data collectors in the field will follow up with participants who do not respond to the telephone survey by trying to meet with them at their WIC clinic in cooperation with local WIC agencies when they return to pick up their food instruments.

Reducing Nonresponse

Many of the tactics outlined for maximizing participation will also be useful for reducing nonresponse. Of particular importance will be the assistance of local WIC agency staff in promoting the survey and handing out flyer referenced above, the emphasis on confidentiality, and the use of multi-lingual data collectors. As also noted, data collectors may try to intercept non-responders by trying to meet up with them at their WIC clinic during a return visit.

B.4 Tests of procedures or methods to be undertaken.

State WIC Agency Survey: The State Survey was pretested in two non-sampled states, the District of Columbia and West Virginia.

Local WIC Agency Survey: The Local WIC Agency Survey was pretested with two agencies in the same non-sampled states as the State Survey. The online version of the survey will be pretested with 1-2 non-sampled agencies prior to its release to sampled agencies.

Surveys of WIC Participants, Denied Applicants and Denied Recertificants: Macro has conducted pretests of the telephone survey with 6 participants (3 each from West Virginia and the District of Columbia) in various program benefit categories (2 pregnant, 1 breastfeeding, 1 postpartum/non-breastfeeding, 1 child) and the in-home interviews with 5 of the same participants (2 in West Virginia and 3 in the District of Columbia). These pretests included one person with limited English speaking skills who had a household member help as required. By choosing West Virginia and the District of Columbia both urban and rural areas were included. The average length of the telephone survey was 24 minutes, the predicted average. The length of time of the in-home interviews was 19 minutes, appreciably below the estimate of 36 minutes. The shorter length of the in-home interviewer was facilitated by a decision to mention to people when scheduling the in-home interviewer that they would be asked to show copies of the same documents they used when applying for benefits. Half of the people so warned had the documents ready when the interviewer arrived.

B.5 Names and telephone numbers of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Macro International Staff—Instrument Design, Data Collection and Interviewer

Dr. Daniel Geller, Project Director, (301) 572-0250

Dr. Pedro Saavedra, Senior Sampling Statistician, (301) 572-0273

Dr. Gary Huang, Technical Director (301) 572-0347

Staff from the Food and Nutrition Service, U.S.D.A. who will analyze the data:

Dr. Sheku G. Kamara, Project Officer, Office of Research and Analysis: (703) 305-2130

Dr. Theodore Macaluso, Branch Chief, Office of Research and Analysis: (703) 305-2121

Jay Hirschman, SNS Director, Office of Research and Analysis: (703) 305-2119.

Macro will retain the services of a temporary staffing agency (Aspen of DC, www.aspenofdc.com), with whom it has worked before, to recruit the data collectors nationwide. Macro will conduct the training and directly manage the interviewers.

1 They will be sampled at twice the incident rate of WIC Participants sampled.

2 They, too, will be sampled at twice the incident rate of WIC Participants sampled.

3 To be described in the next section

20