072212 WIC BPI-Part B only, double-spaced (final v2)

WIC Local Agency Breastfeeding Policy and Practices Inventory

OMB Part B

June 8, 2012

Contract Number: AG-3198-B-10-0015 Mathematica Reference Number: 06934.501 Submitted to: Office of Research and Analysis USDA Food and Nutrition Service 3101 Park Center Drive, Room 1014 Alexandria, VA 22302 Project Officer: Grant Lovellette Submitted by: Mathematica Policy Research 955 Massachusetts Avenue, Suite 801 Cambridge, MA 02139 Telephone: (617) 491-7900 Facsimile: (617) 491-8044 Project Director: James Mabli Survey Director: Sarah Forrestal

WIC Local Agency Breastfeeding Policy and Practices Inventory OMB Part B June 8, 2012

CONTENTS

B COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS 1

B.1. Respondent Universe and Sampling Methods 1

B.2. Procedures for the Collection of Information 2

B.3. Methods to Maximize Response Rates and to Deal with Nonresponse 5

B.4. Test of Procedures or Methods to be Undertaken 10

B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 13

TABLES

B.3.1 WIC BPI Fielding Plan and Predicted Response Rate 17

APPENDIX A: web survey

APPENDIX B: EXample screenshots of web survey

APPENDIX C: regional field office advance letter

APPENDIX D: respondent advance letter

APPENDIX E: Respondent frequently asked questions

APPENDIX F: respondent email invitations

APPENDIX G: Respondent Reminder email

APPENDIX H: Respondent reminder post card

APPENDIX I: Respondent reminder telephone call

APPENDIX J: respondent thank you Email

Appendix K: NASS Comments

This page has been left blank for double-sided copying.

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

Part B of the justification for this information collection activity, the WIC Local Agency Breastfeeding Policy and Practices Inventory (WIC BPI), addresses the five points outlined in Part B of the OMB guidelines.

B.1. Respondent Universe and Sampling Methods

To meet the objectives outlined in Supporting Statement Part A, Section A.2, we will conduct a national census of all 90 State WIC agencies and 2,000 local WIC agencies. The 90 State agencies consist of agencies in 50 States, the District of Columbia, five territories, and 34 Indian Tribal Organizations (ITOs). All agencies will be contacted, meaning that we will not use sampling methods. The survey will be administered via web in two parts, separated by three months. Respondents will consist of WIC agency staff. Typically, the survey will be completed by a single staff member, although multiple respondents may participate at some agencies. At both the State and local WIC agency levels, respondents are likely to be agency directors, breastfeeding coordinators, database managers, or staff members with equivalent titles.

To create a current, accurate list of the 90 State, territorial, and ITO WIC agencies and the 2,000 local agencies, we will use two FNS lists as our primary sources. First, we will use the list of WIC State agencies on the FNS website. This list identifies agency directors and nutrition and breastfeeding coordinators, along with detailed contact information, such as telephone and fax numbers, postal and email addresses, and websites. Second, to create a list of local agencies, we will use the WIC Local Agency Directory (LAD). States update the LAD with agency names and mailing addresses regularly, so we will have current information available when we are ready to field the survey. We will supplement the agency name and mailing address from the LAD with contact information for the agency director, as provided by State WIC agencies.

We expect at least an 80 percent response rate. This will consist of 72 completed surveys by State agencies and 1,600 completed surveys by local WIC agencies for a total of 1,672 completed surveys out of a possible 2,090. This may be a conservative estimate, though, as members of our study’s 11-person panel of WIC and breastfeeding experts (including State and local WIC agency directors and breastfeeding policy coordinators) have suggested that WIC agencies will be enthusiastic to participate in the study in order to learn more about the policies and practices that are promoting breastfeeding locally, at the state level, and nationally.

As discussed in greater detail in Section B.3, if nonresponse is at all higher than desired in the current study, we will conduct a nonresponse analysis and construct a nonresponse adjustment that will lay the basis for ensuring the national representativeness of the study population. Although the agency frame only contains agency names and addresses, we will obtain data with which to conduct a nonresponse analysis by linking those records to an extensive set of population characteristics from the American Community Survey (ACS) that describe the local areas in which agencies are located. We are confident that the ACS variables will be substantially correlated with nonresponse and will enable us to construct adjustment factors that will ensure national representativeness of the study population.

B.2. Procedures for the Collection of Information

Agencies will be asked to provide information in four major domains: (1) agency characteristics, (2) agency policies and practices, (3) breastfeeding measurement and outcomes, and (4) reporting procedures. The first two domains will be fielded in Part 1 of the survey and the remaining two domains will be fielded in Part 2. Agency characteristics include, for example, whether State agencies provide direct services to participants and the number of full-time equivalents in different job positions. A comprehensive set of agency policies and practices will be assessed, including those related to peer counseling, prenatal and postpartum contact, staff roles and staff training, breastfeeding education, coordination and referrals, and breastfeeding aids. Breastfeeding measures will include the process by which the agency collects information about initiation, duration, exclusivity, and intensity (such as wording of questions asked of WIC mothers), and the most recent breastfeeding estimates for breastfeeding exclusivity and intensity (initiation and duration data will be obtained using an existing data source). Questions about breastfeeding reporting procedures will focus on the constituencies to which agencies report breastfeeding information and agencies’ data quality control procedures.

Statistical Methodology. As noted in Section B.1, the study will consist of a census of all State and local WIC agencies, rather than a sample. Thus, there will be no stratification or sample selection methodologies. The survey will be administered via web.

Estimation Procedures. Mathematica Policy Research will be responsible for assigning survey weights to State and local WIC agencies that respond to the survey. Separate sets of weights will be developed for State and local WIC agencies. Weights will be solely based on an adjustment for nonresponse should the characteristics of responding and nonresponding agencies differ statistically. This is described further in Section B.3.

As described in Supporting Statement Part A, Section A.16, the analysis will consist of presenting tabulations and cross-tabulations of breastfeeding outcomes, measurement, data storage, reporting, and policies and practices. In addition, averages will be used for enumeration questions and other distributional characteristics will be estimated for numeric variables related to breastfeeding outcomes, such as the distribution across agencies of the percentage of WIC mothers who initiate breastfeeding. Most of the analyses will be descriptive.

Degree of Accuracy Needed for the Purpose Described in the Justification. Because we are conducting a census of agencies, rather than selecting a sample, there is no sampling error associated with any statistics that would be generated. Nonsampling error due to nonresponse – which we expect to be low – will be dealt with in the weighting process.

Unusual Problems Requiring Specialized Sampling Procedures. No specialized sampling procedures are involved.

Use of Periodic Data Collection Cycles to Reduce Burden. This is a one-time survey data collection effort.

B.3. Methods to Maximize Response Rates and to Deal with Nonresponse

As noted earlier in Section B.1, the expected response rate is 80 percent for both State and local agencies. A variety of efforts will be undertaken to maximize response and minimize nonresponse bias. We are hopeful that these efforts, which we next describe, will enable us to achieve response rates higher than expected. However, should they not, we will conduct nonresponse bias analyses, described in detail below.

We will field the survey in two parts separated by three months. Each part will have a 10-week period and will aim for a minimum 80 percent response rate using procedures that have worked well in web surveys on other studies. We will take several steps to encourage participation in order to minimize nonresponse while ensuring high-quality data. Specifically, splitting the survey into two parts spreads out the burden placed on agency staff over time. We will use early outreach and tailored recruitment materials that highlight how burden was minimized, an attractive, nonmonetary incentive (a customized local report) to encourage participation, and trained staff to interact with respondents. We have summarized the fielding plan in Table B.3.1 and describe key features of the plan following the table.

Table B.3.1. WIC BPI Fielding Plan and Predicted Response Rate

Week	Contact	Cumulative Response Rate (Percentage)
-2	Mail informational letter to FNS regional offices1	0
-1	Mail advance letter to State and local WIC agency directors1	0
0	Email invitation to agency directors	20
2	First email reminder	30
3	Postcard reminder	40
4	Second email reminder	55
6	Telephone reminder	70
9	Third email reminder	75
10	Fourth and final email reminder	80

¹ The informational and advance letters will only be sent before fielding the first part of the survey. Otherwise the fielding plan is identical for parts 1 and 2.

Early Outreach. In the two weeks before launching the first part of the survey, we will mail advance letters to the seven FNS regional offices and to all State and local WIC agency directors. The FNS regional offices will receive an informational letter and list of frequently asked questions (FAQs). Copies of this letter and FAQs are included in Appendix C and Appendix E, respectively. Regional office staff may receive calls from State and local WIC agencies; informing regional staff of the survey and its significance will enable them to respond to inquiries and encourage participation by state and local agencies. FNS regional offices will also be invited to contact the contractor, Mathematica, by telephone or email if they have questions.

State and local WIC agency directors will receive an advance letter, printed on FNS letterhead, inviting them to participate in the survey, along with the FAQs (Appendices D and E). The mailing will introduce the survey, its goals, and its content; encourage participation by highlighting the low burden and the customized agency summary report (a nonmonetary incentive; the report will compare the agency’s breastfeeding outcomes, policies, and practices to those found in its State, FNS region, and/or the U.S.); provide the telephone number and email address; and include each agency’s log-in information.

The advance letter will promote a successful survey in many ways. First, agency directors are likely to read mail from FNS, so the letter will prepare them to receive an email invitation and encourage them not to overlook it. Second, FNS endorsement will also lend credibility to the survey, a factor known to boost response rates. Third, sending the letter before the first field period will enable Mathematica to update contact information for any returned mailings quickly, which will give agencies ample time to participate.

Tailored Recruitment. Both of the 10-week field periods will begin with an email invitation sent to agency directors by the contractor. We will take several steps throughout the fielding period to ensure high participation and data quality. A mix of regular mail, email, and telephone reminders to agency directors will encourage them to complete the survey (Table B.3.1; see Appendices F through I for copies of these materials). Shortly after inviting agencies to participate, Mathematica will examine data from early respondents to verify that the skip logic is working and that data are being recorded accurately. Mathematica will also check agencies’ uploaded data and reports to confirm the files’ readability and relevance so that agencies can be contacted right away to resend any reports or data if problems are found.

We anticipate that no more than 10 percent of agencies will lack Internet access or be unable to complete the survey online. However, for any that do not have Internet access, we will mail them a pencil-and-paper WIC BPI and postage-paid return envelope. In the second field period, we will also include a blank CD with the mailing so that agencies have the option of copying data or reports to the CD instead of mailing back hard copies. If necessary, the contractor and FNS will decide whether to use the same steps with nonrespondents in week 8 of either field period.

Nonmonetary Incentive. Achieving the highest possible response rate in the census is essential not only for reliable baseline data, but also to create a foundation for success in any potential future monitoring of WIC agency breastfeeding policies, practices, and outcomes that FNS may wish to conduct. Because offering an incentive will motivate participants to provide higher-quality data, after Mathematica processes and analyzes the data and submits a draft final report to FNS, Mathematica will send a thank-you email (Appendix J) to each participating agency and attach a short Agency Summary Report comparing its breastfeeding outcomes, policies, and practices to those found in its State, FNS region, and/or the U.S. The reports will also include information on how the numbers were compiled and how to interpret them. Agency Summary Reports are a valuable alternative to monetary incentives. State and local agency directors can use the Agency Summary Report as an information source or decision tool by comparing their policies, practices, and outcomes against those of other WIC agencies.

Trained Staff. Although we will not use telephone interviewing for the study, and thus do not need to train interviewers to collect the data, Mathematica’s highly skilled interviewers will be trained to staff a toll-free number to field questions and offer assistance during both field periods. The trained interviewers, acting as technical assistants, will also conduct telephone reminder calls with nonrespondents, helping them log in and complete the survey online. To fulfill their role as technical assistants, interviewers will complete a two-hour training before fielding each part of the survey that will emphasize the goals and content of each portion of the WIC BPI. The training will include details about the project’s significance and objectives, a question-by-question review of the instrument examining common questions respondents are likely to have and how to answer them, and a review of the reminder call script. The interactive training will include practice scenarios and role-playing.

Nonresponse Analysis and Adjustment. We expect to meet or exceed our response rate target of 80 percent, given Mathematica’s experience conducting web surveys of agencies, the study’s incentives in the form of Agency Summary Reports, and our belief (supported by members of the study’s expert panel of WIC breastfeeding directors, staff, policymakers, and researchers) that the agencies will be excited to be a part of this study. If the response rate is lower than anticipated, however, we have a contingency plan for evaluating whether responding and nonresponding agencies differ and, if they do, for adjusting census weights to minimize the potential for nonresponse bias. We describe a contingency plan for local agencies only, but a similar assessment and adjustment will be followed for State agencies.

In general, the most useful nonresponse adjustments are derived from comparing factors that are strongly associated with outcome measures across survey respondents and nonrespondents (in this case, local agencies). Because our census frame will consist only of lists of agency names and addresses, we will use information from the American Community Survey (ACS) to assess differential nonresponse and estimate an adjustment factor. Specifically, we will append local population characteristics of local agencies to the current study’s census frame using the agency street address, city, State, and zip code. We then will examine differential nonresponse by factors that have been well documented as associated with breastfeeding behavior, such as measures of income, age, race and ethnicity, household resources (for example, owning a vehicle), and household composition.¹

Another potential source of information for the nonresponse adjustment is the WIC Participant Characteristics (WIC PC) data which contain many characteristics specific to breastfeeding, especially among WIC mothers. The usefulness of the data in this application is weakened, however, by the fact that they will not be available for all local agencies in the WIC BPI census frame. This contrasts with availability of the ACS data for each street address of responding and nonresponding agencies. If the 2010 WIC PC data contain most of the agencies in the WIC BPI frame, however, we will consider an adjustment factor using these data as well.

The nonresponse adjustment will consist of two steps. First, we will use information from the ACS to statistically compare responding and nonresponding agencies. Then, we will use those factors on which the agencies differ to define adjustment cells that enumerate agencies of each type (for example, agencies in higher-income versus lower-income areas). This will be a multiway cross-tabulation of responding and nonresponding agencies by all the factors. The adjustment factor for responding agencies in a cell will equal the ratio of responders plus nonresponders to the number of responders. Thus, all responding agencies in a cell will receive the same adjustment factor. Weights for each responding agency will be equal to the adjustment factor and will be used in descriptive analysis of breastfeeding measures and policies and practices by State, FNS region, and nationally.

B.4. Test of Procedures or Methods to Be Undertaken

In July and August 2011, Mathematica conducted a preliminary investigation with eight WIC agencies, consisting of five State agencies, one ITO agency, and two local WIC agencies, to inform the WIC BPI instrument development. Based on the investigation’s findings, survey questions were added, modified, and deleted. The refined instrument was shared with an 11-member independent panel of national and local area breastfeeding experts convened for this project. Experts were asked to review the survey between September and October 2011, and provide feedback on the survey items. The instrument was further revised based on recommendations from the expert panel.

A formal pretest of the survey was conducted in November and December 2011 with a total of nine State and local WIC agencies to test survey length as well as respondents’ understanding of survey items. The instrument was pretested in two steps to allow for sequential refinements. In the first step, the instrument was pretested with two State and two local WIC agency representatives and their feedback was used to guide an initial round of revisions. In the second step, the instrument was pretested with an additional two State and three local WIC agency representatives. This approach helped to ensure that any changes made to the instrument following the first round of pretesting did not create other, unforeseen problems.

The pretests were administered as pencil-and-paper questionnaires followed by a telephone debriefing interview. The questionnaires were customized for State and local agency respondents to minimize the burden of following skip instructions. Participants were offered a $35 incentive to participate in the pretest. After completing the questionnaire, pretest respondents were asked for their feedback during a 60-minute telephone interview. Respondents were asked about question order, comprehensibility, comprehensiveness, and burden, including the amount of time required to complete the survey and whether respondents had to look up information or seek input from other staff. Respondents also were asked to evaluate a draft Agency Summary Report to determine whether the data are presented in a clear, useful format.

Staff from four State and five local WIC agencies participated in the pretest. Participants reported that agency directors were the primary individuals completing the survey, with assistance from breastfeeding or nutrition coordinators as well as information technology or data staff. Participants generally agreed that the survey was comprehensive and easy to complete, and addressed an important topic. They found the terminology to be clear in several specific instances where we probed for their feedback. Participants identified specific questions that were confusing, but of greater concern to them was the high burden, which will be described separately below. The revisions to the instrument are summarized here:

• We revised question text in several questions throughout the instrument to provide clarification. Specifically, we updated the materials and survey text for State agency respondents to clarify that State agencies should report only on behalf of the agency itself and not for all local WIC agencies within the State. We refined definitions for staff positions and added a definition for full-time equivalents. We added clarifying text to several questions indicating that peer counselors should be included in response to questions about policies and practices. We changed the term “data set” to “data system” throughout the breastfeeding measurement section.

• Pretest participants felt that a question about in-hospital prenatal classes or postpartum visits did not belong in a module about breastfeeding referrals. We moved the question to a more appropriate module.

• Participants suggested we add an additional example of breastfeeding duration so that the examples of measures provided to respondents include those that include ongoing feeding as well as those that are based only on when breastfeeding has stopped.

Pretest participants were concerned about the effort it took to complete the breastfeeding estimate questions in the survey. Providing recent estimates of initiation, duration, and exclusivity often required consulting with information technology staff who could run queries on the data system. As a result of these findings, we revised the survey to no longer collect breastfeeding initiation and duration estimates. Instead, we will instead draw those estimates from the 2010 WIC PC data (or the 2012 data if they are available), as described in Section A.16. We also split the survey into two parts to spread the total burden over a longer period of time.

Prior to deployment of the web survey, Mathematica’s programming staff will conduct rigorous testing to ensure that respondents are routed through the instrument properly. The web testers will use different scenarios to test the skip patterns (for example, omitting modules for local agencies if the respondent is from a State WIC agency), question wording, and order to ensure the accuracy of the web survey compared with the hard-copy instrument. Testers will also ensure that partially completed cases route to the next unanswered question upon reentry to the survey.

B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Mathematica staff, FNS staff, and an 11-person work group of WIC breastfeeding experts, including State and local WIC agency directors and breastfeeding policy coordinators, contributed to the planning for the survey and other aspects of the collection. There were no comments from the public. Comments from NASS were received (responses are contained in Appendix K). The survey procedures and analysis plans were developed by Sarah Forrestal, James Mabli, and Ronette Briefel of Mathematica Policy Research (617-301-8997). Sarah Forrestal will be closely involved in planning and overseeing the data collection for the web survey. The members of the study’s expert panel (listed below) reviewed the interview protocol from the preliminary investigation and reviewed early drafts of the instrument. The panel consisted of:

• Tracy Erickson—WIC Breastfeeding Coordinator, Texas WIC Program

• Karen Farley—Program Manager, California WIC Association

• Larry Grummer-Strawn—Chief of Nutrition Branch, Division of Nutrition, Physical Activity, and Obesity, Centers for Disease Control and Prevention

• Jean O’Leary—Breastfeeding Coordinator, Washington State WIC Nutrition Program

• Jeannette Panchula—Senior public health nurse, Solano County Health and Social Services and California Maternal Child and Adolescent Health department

• Carole Peterson—Chair, National WIC Association Breastfeeding Committee

• Cheryl Richardson—Nutrition and Breastfeeding Coordinator, Citizen Potawatomi Nation WIC

• Ursuline Singleton—Public Health Analyst, Department of Health and Human Services, Office of Women’s Health

• Amy Spangler—Author and President, Baby Gooroo

• Laurie True—Executive Director, California WIC Association

• Kelly Whipker—Breastfeeding Coordinator, Davidson County WIC Program, Tennessee

1 The ACS does not contain information for Guam and the Virgin Islands. We will either impute local area characteristics for these areas using data from States and Puerto Rico, or we will use 2010 Census data, which has basic demographic information for all U.S. territories.