K1. MAINLAND STUDY DATA COLLECTION PLANS
This page has been left blank for double-sided copying.
The sections that follow describe the data collection plans for the SNMCS-II mainland study, which include:
Menu Survey with SNMs (Appendices E1 and E2)
SFA Director, Principal, and SNM Surveys (Appendices F1-F9)
Cost study data collection (Appendices G1-G14)
Competitive Foods Checklists and Cafeteria Observation Guide (Appendices H1 and H2)
Student and parent data collection (Appendices I1-I15)
Plate waste observations (Appendix J1)
Reimbursable Meal Sale Data Request (Appendix J2)
SNMs in Groups 2 and 3 will be asked to complete a Menu Survey for one school week (referred to as the “target week”). The Menu Survey collects detailed information on foods prepared and served in NSLP lunches and SBP breakfasts, and foods offered in afterschool snacks. Group 2 respondents will complete a Basic Menu Survey (Appendix E1.1), which will collect all the information needed to (1) estimate the nutrient and food group content of school meals and afterschool snacks and (2) assess the extent to which daily and weekly menus for school meals meet the nutrition standards (the “compliance assessment”). Group 3 respondents will complete an Expanded Menu Survey (Appendix E2.1), which will collect all of the above information plus information needed to estimate the cost of school meals, including information on nonreimbursable foods.
The Menu Survey collects data required to address key study research questions. The information collected will be used to provide estimates of the nutrient and USDA Food Pattern food group content of NSLP lunches, SBP breakfasts, and afterschool snacks. These data, as well as additional information collected on nonreimbursable foods, will be matched to food price documentation to estimate food costs in Group 3 schools. Finally, the Menu Survey data will provide key data elements needed to examine the amount of available nutrients and USDA Food Pattern food groups wasted in school meals (Group 3 only).
As in SNMCS-I, we will administer the Menu Survey as a self-administered web instrument called the Electronic Menu Survey (EMS). Highly trained technical assistants (TAs) will provide respondents with intensive support to ensure prompt completion of the Menu Survey and provision of complete and accurate data. Before the target week, TAs will introduce respondents to the Menu Survey, describe the EMS training videos (described below), and answer initial questions. The TA will also administer the questions in the Menu Survey Screener, which will determine which forms the respondent will complete. Instructions on how to complete the EMS will be provided via brief modular, online training videos. Each form in the EMS will also include written instructions and help text. TAs will also follow up to answer any questions and discuss options for submitting recipes and procedures for special circumstances, such as how to complete the EMS for a sampled school that shares a cafeteria with another school.
After the target week, nutrition coders will review completed forms in the EMS and follow up with SNMs on critical missing data. The nutrition coders will also generate the Fruit and Vegetable Questions and Meal Pattern Crediting Report (Appendix E3), which will list all unique foods offered in each sampled school’s reimbursable lunches and breakfasts during the target week. This report will be sent to the menu planner to indicate the maximum number of fruit and vegetable items students may take as part of a reimbursable meal, and how foods served in lunch and breakfast during the target week contributed to each meal pattern food group.
We will field surveys to SFA directors in Groups 1, 2, and 3. In addition, we will field surveys to SNMs and principals in Groups 2 and 3. These instruments will provide data needed to characterize the SFA and school environment, food service operations, school meal program participation, and student population characteristics. We will collect these data using self-administered web surveys.
We will mail Group 1 SFA directors an introductory letter about the study (Appendix F1) and send an email invitation (Appendix F2) to complete the SFA Director Survey on the web (Appendix F3). We will use a combination of email (Appendix F4) and telephone follow-up (Appendix F5) to encourage participation and will offer assistance through a toll-free help line and project email address.
Collection of data from SFA- and school-level staff in each sampled Group 2 and 3 SFA will take place in stages. The SNM Survey (Appendix F6) will be integrated into the EMS for respondent ease and can be completed before the target week. For the SFA Director Survey (Appendix F3) and Principal Survey (Appendix F7) in Groups 2 and 3, we will send invitations (Appendix F8) after the target week so these respondents can focus on the on-site data collection activities occurring during the target week. Follow-up efforts with these groups will replicate the procedures planned for Group 1 SFA directors; we will use email and telephone reminders (Appendices F4, F5, and F9).
Staff in the Group 3 sample will participate in cost interviews and provide administrative data needed to estimate meal costs and school food service revenues. Cost data collection in these samples will proceed in stages: (1) data collection before the target week, (2) on-site or remote data collection during the target week, and (3) follow-up data collection after the end of the school year.
Data collection before the target week. Concurrent with SFA recruitment, trained study staff will contact State Education and CN Agency finance officers to complete the State Agency Indirect Cost Survey (Appendix G1). This survey will be fielded to all States with sampled SFAs. We will mail an invitation letter (Appendix G2) and study overview (Appendix C8) to respondents. The survey will gather information on indirect cost rate-setting and what districts’ indirect cost rates include. This information is important for determining what SFAs’ reported costs include and whether unreported costs are direct or indirect. The survey is fielded to State staff because SFA-level respondents may not know this information. Another advantage of collecting this information from States is that if the SFA uses the State’s calculated and approved indirect cost rate or cost allocation plan, the SFA director and district business manager do not need to complete the section on SFA indirect costs included in the SFA On-Site Cost Interview (Appendix G3) during the target week, which reduces burden on these respondents.
Mainland study data collection during the target week. Trained field interviewers (FIs) will visit Group 3 SFAs and schools during the target week to conduct in-person interviews. FIs will administer the SFA On-Site Cost Interview (Appendix G3), SNM Cost Interview (Appendix G4), and Principal Cost Interview (Appendix G5) with the respective staff. Food prices and the assigned value of USDA Foods (donated commodities) are needed to determine the costs of food served during the target week in Group 3 schools. FIs will use the Food Cost Worksheet (Appendix G6) to review for completeness the food price documents SFA directors were asked to compile in advance (Appendix C19). The worksheet will identify standard categories of foods for which SFAs might have different price lists, such as fresh produce, bakery products, dairy, and frozen foods. FIs will indicate whether price information was obtained, is not applicable, or requires follow-up. During data processing, price coders will review the worksheet and the documentation and contact the SFA director if any needed documents are missing.
In schools with self-serve or made-to-order bars available on visit days, FIs will complete the On-Site Self-Serve/Made-to-Order Bar Form (Appendix G7) to record the types and amounts of foods used on each bar during the meal period. FIs will use paper forms because the format is more flexible for completing in a cafeteria setting. FIs will scan and securely transmit completed forms to the Menu Survey TAs to be uploaded into the EMS and linked to the school’s record.
The follow-up data collection for Group 3 occurs in two phases: (1) a web survey plus submission of final expense and revenue statements, and (2) a telephone interview with screen sharing, so that the interviewer and respondent can review financial data together in real time. The timing of the data collection is determined by when each SFA’s final financial statements are available, as reported to the study team during the spring data collection.
In the first phase, we will invite SFA directors to complete the SFA Follow-Up Web Survey to report food service operations summary data for SY 2019–2020, including the number of school meals and snacks that were claimed; the number of operating days; the number of meals prepared for non-school facilities; and cost information for the Fresh Fruit and Vegetable Program, if applicable (Appendix G11). The invitation will also request scheduling availability for the interview and instructions for gathering and transmitting the necessary financial records to be reviewed during the interviews (Appendix G12).
In the second phase, trained contractor staff will review and abstract financial information from the SFA expense and revenue statements and schedule the SFA Follow-Up Cost Interview (Appendix G13). Interviewers will conduct these interviews with Group 3 SFAs over the telephone using screen-sharing technology (when feasible) to ensure shared understanding of the revenues and expenses (and their subcomponents) and final indirect cost rates for SY 2019–2020. We will provide respondents with a user-friendly reference guide (Appendix G14) to facilitate the interview.
Trained contractor staff will collect data on the types of foods available in competitive foods venues and characteristics of the cafeteria environment in schools in Groups 2 and 3 using three observational tools: (1) the A la Carte Foods Checklist (Appendix E1.1 and E2.1); (2) the Vending Machine and Other Sources of Foods and Beverages Checklists (Appendix H1); and (3) the Cafeteria Observation Guide (COG) (Appendix H2). Table K.4 summarizes the data to be collected in these instruments and the respondents.
Table K.4. Competitive Foods Checklists and Cafeteria Observations
Instrument |
Data to be collected |
Respondent |
A la Carte Foods Checklist |
List of all foods and beverages the school food service department sold a la carte on a selected day |
School nutrition manager |
Vending Machine and Other Sources of Foods and Beverages Checklists |
Competitive foods and beverages available for sale to students in vending machines and in other venues, such as school stores, snack bars, kiosks, or fundraisers |
Field interviewer |
Cafeteria Observation Guide |
Characteristics of cafeteria facilities |
Field interviewer |
The A la Carte Foods Checklist is administered to SNMs as part of the EMS. FIs will complete the Vending Machines and Other Sources of Foods and Beverages Checklists and the COG during their time on-site. All three instruments will be fielded electronically. FIs will observe all non–a la carte competitive foods venues, and the COG will be completed for one randomly selected breakfast period (or up to 30 minutes of breakfast service, if there is no designated breakfast period) and one randomly selected lunch period.
In Group 2 schools, we will conduct Student (Appendices I1 and I2) and Parent Interviews (Appendices I3 and I4) to collect information on student characteristics and participation in and satisfaction with school meals. In addition, trained FIs will conduct 24-hour dietary recall interviews to collect information on students’ dietary intakes (Appendix I5) and measure students’ heights and weights (included in the Student Interview as part of Appendices I1 and I2).
Instruments will be available in English and Spanish. Data collection activities will differ for elementary school students and middle and high school students.
24-hour dietary recalls. SNMCS-II will examine the contributions of school meals to students’ dietary intakes, assess the adequacy and overall nutritional quality of students’ diets, and assess the impact of school meal participation on these and other nutrition-related outcomes. The approach to conducting the 24-hour dietary recall interviews will largely replicate the approach used in SNMCS-I, which is tailored to children’s cognitive abilities and is designed to maximize response rates and efficiency. To maintain consistency with other national data sets, including SNMCS-I and the National Health and Nutrition Examination Survey (NHANES), we will use USDA’s Automated Multiple-Pass Method (AMPM) software to collect the 24-hour recalls (Appendix I5). Students will report detailed information on foods and beverages consumed on the target day using a two-dimensional food model booklet and measuring cups and spoons to help with portion size estimation.
Before conducting the recall interviews in a school, FIs will complete three activities:
Complete a Point-of-Sale (POS) Form (Appendix I6) to document the physical locations where students can obtain foods and beverages, and assign codes to each location using a common coding structure to distinguish between locations that sell reimbursable and nonreimbursable items. (Although this form will be completed by FIs, it requires some interaction with the SNM to ensure that the correct information is being captured and coded.) During the recall, AMPM software will prompt the FI to ask the respondent for the specific source of each food obtained at school, and the FI will record the corresponding code from the POS Form.
Complete a Milk Form (Appendix I7), which documents the type, flavor, fat content, and carton or bottle/cap color of all milks available to students. This information will be used to help students accurately identify the type of milk consumed.
Collect printed school menus that list the items served on the target day. The menus will be used during the 24-hour recalls to help students accurately recall and describe foods eaten as part of a school meal, which facilitates matching these foods to items recorded in the school’s Menu Survey. The menus are used only as a fallback when a student cannot describe or recall an item eaten in a school meal after the standard “passes” used in the AMPM software to identify items consumed. Interviewers will be trained to probe neutrally to avoid leading respondents.
Middle/high school students will complete the 24-hour recall in one in-person interview at school, reporting their dietary intakes for the prior school day, midnight to midnight. Elementary school students will complete the interview in two parts. The reporting period will also be midnight to midnight, but the first part of the interview will take place on the target day. Elementary school students will be interviewed in person on the target day as soon as possible after lunch to report on foods consumed from the time they awoke through lunch. Conducting the recall immediately after lunch will maximize accurate recall—research has shown that younger children report their intakes more accurately when interviewed shortly after a meal.1 The rest of the interview, which will capture information about foods consumed the rest of the day, will be completed with parental assistance on the following day.2 For example, if the reference day for a 24-hour recall is a Monday from midnight to midnight, a middle/high school student will be interviewed once during the school day on Tuesday. An elementary school student will begin the interview on Monday after lunch and will report all foods and beverages consumed from midnight to the time of the interview after Monday’s lunch. The 24-hour recall will be completed on Tuesday, when the student and parent will be interviewed together and will report all foods and beverages consumed from the time of the Monday interview until midnight. The first part of interviews with elementary school students will be completed on Monday through Friday, while a subset of interviews with middle/high school students will be completed on Saturday to capture dietary intakes on Fridays.
The parent-assisted part of the 24-hour recall for elementary school students will be conducted by telephone to maximize completion rates and reduce burden on parents, compared to that of in-person interviews. Collection of 24-hour recalls by telephone is common—for example, NHANES 2015–2016 conducted 24-hour recalls by phone.3 Research has shown that 24-hour recalls conducted in person and by telephone yield comparable results.4 After completing the in-person interview, FIs will provide elementary school students with portion reference aids and a nonquantitative food diary (Appendices I8 and I9) to bring home to parents to serve as a memory aid.
To estimate distributions of usual nutrient and food group intakes, a second 24-hour recall is needed for a subsample of students. As in SNMCS-I, we will conduct these second recalls by telephone 3 to 10 days after the initial recall. Middle/high school students selected to complete the second recalls will be given portion reference aids to help in reporting portion sizes over the telephone. Parents of elementary school students selected to complete the second recalls will be provided with a second food diary. We will remind parents and students about the dietary recall interviews using a combination of email and text messages, if parents gave permission to text middle/high school students (Appendices I10 and I11).
Student and Parent Interviews. Both sampled students and their parents will complete an interview to collect information on their reasons for participating or not in school meals, perceived stigma of receiving free or reduced-price meals, and satisfaction with the meals. All sampled students will complete a Student Interview (Appendices I1 and I2), which will be administered in school, with computer-assisted personal interview (CAPI) software, in conjunction with the 24-hour dietary recall.
We will also collect students’ height and weight measurements following the procedures used in SNMCS-I. FIs will measure standing height using a portable stadiometer (height measuring board) modeled after the procedure developed for NHANES.5 Measurements will be completed after the 24-hour recalls and Student Interviews are completed, and values will be recorded in the Student Interview. To increase cooperation and minimize potential measurement problems associated with students’ clothing or hairstyles, school liaisons will send flyers home with sampled students the day before height and weight measurements (Appendices I12 and I13). The flyer will encourage students to dress in layers or avoid heavy outer clothing and give notice that they will be asked to take off shoes, as well as particular accessories such as hats or hair ornaments.
A parent of each sampled student will complete the Parent Interview in English or Spanish (Appendices I3 and I4). Parents will be offered a choice of completing the interview either on the web or by telephone. The Parent Interview includes questions on demographics and participation in other nutrition assistance programs that would not be appropriate to ask children. Parents will also be asked questions about their general satisfaction with school meals and specific issues such as whether they receive enough information about the meals, whether they believe the meals are healthy, and what perceptions they have about the school food environment (for example, the types and availability of competitive foods). We will remind parents to complete the interview on the web using a combination of email and texts (Appendices I14 and I15). If the Parent Interview is not complete by the time of the parent-assisted portion of the dietary recall interview, telephone interviewers will complete the Parent Interview by telephone. Telephone interviewers will also follow up to complete the Parent Interview with parents of middle/high school students, if it was not completed by the time the Student Interview is completed. Time to respond to calls is included in the burden estimates for the Parent Interview, as it is part of the time for reviewing instructions needed to complete the information collection.
To measure levels of plate waste in school meals, FIs will observe 3,900 lunch trays and 2,000 breakfast trays that include reimbursable meals. Observations, averaging 30 lunch and 15 breakfast observations per Group 3 school selected for the plate waste study, will document the types and amounts of food selected by students and the amounts left over after the student has eaten his or her meal. These plate waste observations will allow us to estimate the amounts and proportion of foods wasted by students and assess the relationship between plate waste and key characteristics of students, school food environments, school food service operations, and the meals themselves (for example, the nutritional quality and cost of meals).
FIs will pre-fill information about the length and timing of meal periods in the Plate Waste Observation Booklet (Appendix J1) before the visit and verify the information when they are on-site. (Although this booklet is completed by FIs, it requires some interaction with the SNM to confirm the foods offered and what constitutes a reimbursable meal.) In collecting plate waste data, we will use visual observation of solid foods and measurement of liquids wasted. We will enhance the visual observation by using portioned servings of foods purchased by FIs as a point of reference. We will measure liquids wasted with a measuring cup.
Foods and beverages observed in plate waste data will be linked with the corresponding foods reported in the Menu Survey on the observation day. This linkage is essential to obtaining nutrient and food group information for foods and beverages selected and wasted. The Menu Survey TA will remind the SNM of the plate waste observations and encourage the SNM to complete the listing of foods offered in reimbursable meals that day, even if the SNM does not complete the rest of the Menu Survey for that day until later.
The Reimbursable Meal Sale (RMS) Data Request (Appendix J2) will solicit, for all sampled students in Group 2 schools, information on certification status for free or reduced-price meals and whether they participated in the NSLP/SBP on the day referenced in the 24-hour dietary recall. FIs will collect this information from the school POS system or directly from the SNM while on-site.
1 Baxter et al. 2004
2 Most of the parent-assisted recall interviews will be completed on the following day. To allow parents greater flexibility, we will continue to attempt the interview up to two days after the target day. For example, if the elementary school student is interviewed on Monday, we will attempt to complete the parent-assisted portion on Tuesday or Wednesday. To preserve data quality, recalls will not be completed after the two-day period has ended.
3 Centers for Disease Control and Prevention (CDC). “National Health and Nutrition Examination Survey (NHANES) Phone Follow-Up Dietary Interviewer Procedures Manual.” Atlanta, GA: CDC, January 2016a. Available at https://wwwn.cdc.gov/nchs/data/nhanes/2015-2016/manuals/2016_Phone_Follow-Up_Dietary_Interviewer_Procedures_Manual.pdf. Accessed November 3, 2017.
4 Tran, K. M., R. K. Johnson, R. B. Soultanakis, and D. E. Matthews. “In-Person vs. Telephone-Administered Multiple-Pass 24-Hour Recalls in Women: Validation with Doubly Labeled Water.” Journal of the American Dietetic Association, vol. 100, 2000, pp. 777–783.
5 Centers for Disease Control and Prevention (CDC). “National Health and Nutrition Examination Survey (NHANES) Anthropometry Procedures Manual.” Atlanta, GA: CDC, January 2016b. Available at: https://wwwn.cdc.gov/nchs/data/nhanes/2015-2016/manuals/2016_Anthropometry_Procedures_Manual.pdf. Accessed June 28, 2017.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Mindy Hu |
| File Modified | 0000-00-00 |
| File Created | 2021-01-22 |