Appendix D3. Pretest Findings
Appendix D3. Pretest Findings.docx
School Food Purchase Study-IV (Reinstatement)
Appendix D3. Pretest Findings
OMB: 0584-0471
Appendix D3. Pretest Findings
Introduction and Methods |
1 |
The U.S. Department of Agriculture’s Food and Nutrition Service (FNS) has contracted with Westat to conduct the School Food Purchase Study IV (SFPS-IV). The purpose of the study is to provide updated national estimates of school food authority (SFA) food acquisitions (both purchased foods and USDA Foods) and describe food purchase practices, as well as changes in purchases and purchase practices since the prior study in SY 2009-2010.
In preparation for study launch, Westat tested two survey instruments: the Initial Vendor List and the Survey of Food Purchase Practices. The team also tested the school food authority (SFA) email to vendors, which includes the food purchase data elements. The Initial Vendor List and the Survey of Food Purchase Practices were developed using the Survey of Food Purchase Practices used in SFPS III. For this reason, testing was limited to questions that were revised or new, or where there was a question about the current applicability of response options. Testing ensures that we have accurately estimated the level of burden, and it provides an understanding of the potential response errors and challenges to providing the necessary data.
1.1 Data Collection
The following four SFA Directors participated in the telephone pretest:
Sandra Baxter, ISD Director of Child Nutrition, Brenham Independent Schools, Texas
Lynn Petrowski, Food Service Director, Hanover Schools, Massachusetts
Jeff Kavalek, Food Service Director, Clearfield Area School District, Pennsylvania
Cynthia Schrader, Branch Manager, Leavenworth Public Schools, Kansas
Trained senior qualitative researchers led the telephone interviews, assisted by a note taker. The interview sessions lasted up to 90 minutes and included the following:
The interviewer administered a study introduction, explaining the study purpose and the respondent’s rights as a research subject.
Respondents were asked for their verbal consent after interviewers explained the voluntary nature of their participation and confidentiality of their responses.
The respondents were provided opportunities to offer additional feedback or reactions at various points in the interview.
After the end of the session, the respondent was thanked for participating.
Respondents were e-mailed the instruments in advance and were asked to review the materials before the telephone discussion, but were told not to spend time completing the surveys. At the end of each call, interviewers asked respondents about their willingness to share any additional materials, such as vendor summaries, to assist the team in understanding the formats and fields contained in their vendor summaries.
1.2 Data Analysis
The senior qualitative researcher led the data analysis. All interview notes and audio recordings were reviewed and synthesized. The insights provided by pretest participants were used to edit and revise the surveys. In particular, staff focused on areas where the respondents demonstrated confusion, hesitation, and/or uncertainty, and on suggested changes in terminology. All feedback was discussed with project leadership who made final decisions about revisions to the data collection instruments.
1.3 Findings and Recommendations
The remainder of this report summarizes the issues raised during the pretests and provides recommendations for addressing these issues for the two surveys and the email to vendors. Wording changes to the questions are noted in red. Track changes versions of the three instruments are attached to this report.
Summary of Recommendations |
2 |
2.1 Initial Vendor List
Question number |
Findings |
Recommendations |
||||||||||||||||||||||||
|
Global Issues. There was a request to include a PDF copy of the survey for reference when the email with the link to the survey is sent as people may want to know all the questions they will be asked before they click through so that they can prepare. |
Include a PDF attachment and enhanced instructions when sending the link to complete the instrument. |
||||||||||||||||||||||||
6 |
Item: Do you belong to a buying cooperative?
Respondents indicated that one SFA could belong to several cooperatives, and not all purchase food, so it is best to clarify that this question only refers to a food-buying cooperative. |
Revise by adding the term “food” to the question:
Do you belong to a food buying cooperative?
|
||||||||||||||||||||||||
8 |
Item: We will be requesting detailed data on food purchases (hyperlink) for your district from <Month – Month>. For each food item listed below, list the vendors (hyperlink) you currently purchase foods from and provide their contact information. If you do not purchase foods from any vendors outside of the FSMC, check here.
Respondents indicated that the definitions of some food items were unclear and could lead to different products being within two items. |
Revise by adding parenthetical examples for some food categories and provide definitions for food items:
We will be requesting detailed data on food purchases (hyperlink) for your district from <Month – Month>. For each food item listed below, list the vendors (hyperlink) you currently purchase foods from and provide their contact information. If you do not purchase foods from any vendors outside of the FSMC, check here.
Dairy:
Fresh milk, cheese, butter, yogurt, and other milk-related
products; fresh eggs; substitute dairy products. Ice cream is
not included; it is treated separately. Ice
Cream:
Include all ice cream, ice milk products, and frozen yogurt. Bread:
Bread, rolls, buns, cakes, cookies, crackers, donuts. Do not
include snack items such as pretzels and snack cookies (see
below). Fresh
Produce:
Fresh fruits and vegetables, including fresh-cut i.e. chopped
lettuce, salad mix, apple slices. Canned/Staples:
All canned foods including canned fruits, vegetables, and meat;
staple foods such as flour, sugar, rice, cereals, and cooking
oils. Frozen
Foods:
All frozen foods including frozen fruits and vegetables, frozen
meats and frozen pizza, and other frozen entrees or side dishes.
Ice cream is not included. Fresh
Meats:
All fresh meat, poultry, and fish. Canned and frozen meat,
poultry, and fish are not included. Snack
Items:
Potato chips, pretzels, individual packs of cookies. Other
Beverages:
Include fruit juice, water and sparkling beverages. |
||||||||||||||||||||||||
9 |
Item: We need to identify the easiest way for you to provide us with a record of your food purchases (hyperlink) from all sources for <month1 to month3>. How will you provide us with this detailed information for all foods purchased between <Month 1 to Month 3> for the following vendors?
Respondents stated that vendor summaries are oftentimes referred to as “velocity reports.” Thus this option should be added to reduce confusion.
|
Revise drop down option by adding “velocity report” to vendor summaries option:
We need to identify the easiest way for you to provide us with a record of your food purchases (hyperlink) from all sources for <month1 to month3>. How will you provide us with this detailed information for all foods purchased between <Month 1 to Month 3> for the following vendors?
|
||||||||||||||||||||||||
10 |
Item: How are USDA Foods delivered to you? Select all that apply.
Name of Distributor ____________________________________________
Respondents said that USDA Foods could come from distributors and processors. |
Provide instructions on what to include when answering this question and clarifying response options:
How are USDA Foods delivered to you? This includes all USDA Foods, whether received through direct delivery (i.e., brown box) or diverted for further processing.
Select all that apply.
Name of Distributor ___________________________________________
Name of Processor [programmer: allow for multiple processors]_________________________________
|
||||||||||||||||||||||||
13 |
Item: Which value pass-through methods does your SFA use to obtain credit for the value of the USDA Foods in processed end products? (Section III, Q5, SFA Procurement Practices Web Survey) Select all that apply:
Including direct shipment and invoicing from the processor to the recipient agency, fee-for-service through a distributor and modified fee-for-service, when the recipient agency has an authorized agent bill them for the total case price.
Although “Direct Discount” wasn’t a familiar term for one respondent, she was able to select the method her district uses. We expect the same will be true for other respondents in that they may not recognize the terms that don’t apply to them, but will still be able to select the response option that captures their methods. |
No revisions needed as Westat expects that all respondents will be able to select the option that fits them. |
||||||||||||||||||||||||
14 |
Item: Please provide the following information for processed end products containing USDA Foods.
Respondents wanted clarification on which USDA Foods to include in their responses. |
Add instructions on what to exclude when responding to this question and add alternative wording for indirect discount (i.e. net off invoice): Please provide the following information for processed end products containing USDA Foods. Do not include direct delivery USDA Foods (i.e., brown box).
|
||||||||||||||||||||||||
15 |
What is the easiest way of providing us with information on the volume of USDA Foods received by your SFA for <month1 to month3>?
Respondents noted that velocity reports/vendor summaries, and SFA reports are commonly used by SFAs. Since the study prefers not to receive delivery slips it was suggested to move this option to near the end. |
Revise response options to reflect relevant forms of providing data:
What is the easiest way of providing us with information on the volume of USDA Foods received by your SFA for <month1 to month3>?
|
2.2 Survey of Food Purchase Practices
Question number |
Findings |
Recommendations |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Global Issues. There was a request to include a PDF copy of the survey for reference when the email with the link to the survey is sent as people may want to know all the questions they’ll be asked before they click through so that they can prepare.
Respondents also noted this instrument would likely take 1 ½ to 2 hours to complete. |
Send the survey as a PDF attachment to the email containing the survey link, and enhance survey instructions to provide additional details on what information SFAs should gather prior to logging in.
To minimize respondent burden, we will combine the initial vendor list with the survey of food purchase practices. We will also administer the combined survey at the beginning of the study for all SFAs, regardless of their assigned quarter for reporting food purchase data. This will spread out the burden of the study. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
1.3 |
Item: Number of serving days/number of meals served. Record the number of serving days and the number of student lunches and student breakfasts served, indicating whether they were full price, reduced price, or free. If your district operates the Community Eligibility Provision (CEP) or provisions 2 or 3, indicate the number of meals claimed in each category. Please provide this information for School Year 2019-2020 and for the period <Quarter date range, year>.
Respondents noted that severe need breakfasts and free breakfasts fall under the same category and rather than have them on two lines, it would be best to have severe needs as a subset of free breakfasts.
|
Indent response option so it is clear that severe need breakfasts are a subset of total number of free breakfasts. Also clarify that the footnote is about serving days.
Number of serving days/number of meals served. Record the number of serving days and the number of student lunches and student breakfasts served, indicating whether they were full price, reduced price, or free. If your district operates the Community Eligibility Provision (CEP) or provisions 2 or 3, indicate the number of meals claimed in each category. Please provide this information for School Year 2019-2020 and for the period <Quarter date range, year>.
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
1.5 |
Item: Meal Prices. As of October 31, 2020, what prices were charged to students for full price and for reduced price lunches and breakfasts in your school district by level of school?
One respondent, whose entire district is CEP, suggested providing a way to reflect that all meals are free. |
Introduce a skip pattern so that districts that serve only free meals, or just free breakfasts, can skip out of reporting meal prices.
Meal Prices. As of October 31, 2020, what prices were charged to students for full price and for reduced price lunches and breakfasts in your school district by level of school?
For full price lunches and breakfasts, we have provided space for more than one price if multiple prices were offered. If you indicate more than one charge for full price meals, please indicate the share of meals sold at each price.
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
1.7 |
Item: A la carte food (“nonprogram foods”) sales. Do any of the schools in your school district offer foods on an a la carte basis? A la carte foods are those that are priced and sold on an individual item basis rather than as a unit or complete meal. This includes items from a reimbursable meal if sold separately. It also includes sales through vending machines and at school stores using foods or beverages purchased through the nonprofit school food service account.
Respondents suggested defining a la carte/nonprogram foods would help avoid confusion. When the interviewer asked how difficult it would be to provide the top ten a la carte items, one respondent noted an SFA’s ability to respond would depend on how detailed their Point of Sale (POS) system is. A second respondent preferred providing a full list of a la carte item, and another said to keep it to just the top ten. |
Add clarifying language to describe nonprogram foods:
Nonprogram food sales (i.e., a la carte or competitive foods). Do any of the schools in your school district offer nonprogram foods? Nonprogram foods are those that are priced and sold on an individual item basis rather than as a unit or complete meal. This includes items from a reimbursable meal if sold separately. It also includes sales through vending machines and at school stores using foods or beverages purchased through the nonprofit school food service account. Do not include Special Milk Program or adult meals.
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
1.8 |
Item: Number of schools offering food service options. How many of the schools in your school district currently offer the following options to your students?
Respondents noted that their SFA had a snack bar and school store as the same entity and it would be best to combine these answer choices. Respondents also suggested changing “Electronic debit cards” to “Electronic payment methods” to capture a wider variety of responses. |
Revise two response options to clarify:
Number of schools offering food service options. How many of the schools in your school district currently offer the following options to your students?
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
2.1, 2.2 |
Item: 2.1 Vendor selections. Who in your school district has primary responsibility for determining where foods are purchased, i.e., which vendors are selected, whether by bid or other method? (If this person has more than one position, please select the position that best describes the person’s duties.)
2.2 Food selection. Who in your school district has primary responsibility for determining which foods are purchased? (Again, if this person has more than one position, select the one that best describes the person’s duties.)
Respondents noted that there may be some confusion between District food service manager and kitchen manager and therefore it would be best to remove “manager” from the district food service director option.
|
Delete the term “manager” from the first response option:
2.1 Vendor selections. Who in your school district has primary responsibility for determining how foods are purchased, i.e., which vendors are selected, whether by bid or other method? (If this person has more than one position, please select the position that best describes the person’s duties.)
2.2 Food selection. Who in your school district has primary responsibility for determining which foods are purchased? (Again, if this person has more than one position, select the one that best describes the person’s duties.)
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
2.4 |
Item: Level of purchasing. Are food purchase decisions made at the level of the school district (centralized), at the level of the individual school (decentralized), or some combination of the two? Check () one space for each type of food.
Dairy and non-dairy beverage fields were updated to provide greater clarity and to align with the Initial Vendor List survey. |
Provide examples of product types to include for the dairy and other beverages categories:
Level of purchasing. Are food purchase decisions made at the level of the school district (centralized), at the level of the individual school (decentralized), or some combination of the two? Check () one space for each type of food.
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
2.8c |
Item: What were the top 5 locally grown/produced items by value that you purchased in School Year 2019-2020?
Respondents noted that “value” could mean several different things. |
Revise to:
What were the top 5 locally grown/produced items by dollar value that you purchased in School Year 2019-2020? |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
3.1-3.5 |
Item:
3.1 USDA Foods. Do you know the dollar amount of your USDA Foods entitlement for School Year 2020-2021?
a) If YES, what is your USDA Foods entitlement for School Year 2020-2021? AMOUNT $__________
b) What percentage of your USDA Foods entitlement for School Year 2019-2020 was spent on processed end products containing USDA Foods? If the percentage you provide is an estimate please check () the box.
c) Did you fully utilize your USDA Foods entitlement in School Year 2019-2020? [NEW]
One respondent noted that in Kansas there is no commodity buying so these questions are not applicable. |
Use alternate wording for the following questions and skip Questions 3.2-3.5 for SFAs in Kansas:
a) If YES, what is your USDA Foods entitlement for School Year 2020-2021? [Programmer: Different wording for SFAs in Kansas: If YES, what is the amount of your USDA cash-in-lieu of commodities entitlement for School Year 2020-2021?] AMOUNT $__________
b) What percentage of your USDA Foods entitlement for School Year 2019-2020 was spent on processed end products containing USDA Foods? If the percentage you provide is an estimate please check () the box. [Programmer: SKIP for SFAs in Kansas]
c) Did you fully utilize your USDA Foods entitlement in School Year 2019-2020? [Programmer: SKIP for SFAs in Kansas]
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
4.2 |
Item: Indicate the principal method you use to purchase each type of food. Check () one for each food type
Respondents suggested removing “sales representative visits.” |
Remove “sales representative visits” and provide examples of product types to include for the dairy and other beverages categories. Also provide definitions/examples of products in each category:
Item: Indicate the principal method you use to purchase each type of food. Check () one for each food type
Dairy:
Fresh milk, cheese, butter, yogurt, and other milk-related
products; fresh eggs; substitute dairy products. Ice cream is
not included; it is treated separately. Ice
Cream:
Include all ice cream, ice milk products, and frozen yogurt. Bread:
Bread, rolls, buns, cakes, cookies, crackers, donuts. Do not
include snack items such as pretzels and snack cookies (see
below). Fresh
Produce:
Fresh fruits and vegetables, including fresh-cut i.e. chopped
lettuce, salad mix, apple slices. Canned
and Staple Foods:
All canned foods including canned fruits, vegetables, and meat;
staple foods such as flour, sugar, rice, cereals, and cooking
oils. Frozen
Foods:
All frozen foods including frozen fruits and vegetables, frozen
meats and frozen pizza, and other frozen entrees or side dishes.
Ice cream is not included. Fresh
Meats:
All fresh meat, poultry, and fish. Canned and frozen meat,
poultry, and fish are not included. Snack
Items:
Potato chips, pretzels, individual packs of cookies. Other
Beverages:
Include fruit juice, water and sparkling beverages. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
4.3d |
Item: If YES, indicate with a check () the foods you used in School Year 2019-2020 that were purchased through a cooperative buying program.
Dairy and non-dairy beverages were updated with the new text. |
Provide examples of product types to include for the dairy and other beverages categories:
If YES, indicate with a check () the foods you used in School Year 2019-2020 that were purchased through a cooperative buying program.
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
4.4 |
Item: Product pricing. For each food type below, indicate whether you have a formal agreement (contract) or an informal agreement with your major vendor. Within the category selected, check () the one approach to pricing that best describes how your food purchase prices are determined.
Dairy and non-dairy beverages were updated with the new text. We also include the list of food categories and definitions in a box below to provide additional clarity. |
Provide examples of product types to include for the dairy and other beverages categories. Also provide definitions/examples of products in each category:
Product pricing. For each food type below, indicate whether you have a formal agreement (contract) or an informal agreement with your major vendor. Within the category selected, check () the one approach to pricing that best describes how your food purchase prices are determined.
Dairy:
Fresh milk, cheese, butter, yogurt, and other milk-related
products; fresh eggs; substitute dairy products. Ice cream is
not included; it is treated separately. Ice
Cream:
Include all ice cream, ice milk products, and frozen yogurt. Bread:
Bread, rolls, buns, cakes, cookies, crackers, donuts. Do not
include snack items such as pretzels and snack cookies (see
below). Fresh
Produce:
Fresh fruits and vegetables, including fresh-cut i.e. chopped
lettuce, salad mix, apple slices. Canned
and Staple Foods:
All canned foods including canned fruits, vegetables, and meat;
staple foods such as flour, sugar, rice, cereals, and cooking
oils. Frozen
Foods:
All frozen foods including frozen fruits and vegetables, frozen
meats and frozen pizza, and other frozen entrees or side dishes.
Ice cream is not included. Fresh
Meats:
All fresh meat, poultry, and fish. Canned and frozen meat,
poultry, and fish are not included. Snack
Items:
Potato chips, pretzels, individual packs of cookies. Other
Beverages:
Include fruit juice, water and sparkling beverages. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
2.3 SFA Email to Vendors
Question number |
Findings |
Recommendations |
|
General Feedback on Data Collection. Respondents were asked if SFAs have all of these data elements for their purchase data readily available in-house or if they had to ask their vendors. Based on the responses it appears many SFAs will compile what they can themselves, and ask vendors to fill in any gaps.
Respondents were also asked in what format they will provide the data (Excel, pdf, hard copy). None of the respondents said they would provide a hard copy, but Excel or PDF files seem likely.
Timing of Data Request Respondents suggested waiting two weeks after the end of a quarter before sending this email, and then giving respondents 1 month to provide the requested data. However, for the April-June quarter, respondents suggested notifying SFAs in early May (i.e., before schools close for the summer) that this request is coming, and to give the SFAs providing data for that quarter an extra month (for a total of 2 months) to provide the data. |
No changes needed.
Time data collection request for purchase data to avoid contacting respondents during claim preparation time and provide one full month to submit the data. |
|
Respondents were asked about the data elements to provide, specifically:
Respondents asked for definitions of each data element desired. |
Provide a food purchase data template that lists all variables/data elements and an explanation for each data element.
We have attached an Excel template that lists the data elements and provides an explanation of each. You may use this template to collate and submit data for all vendors, or simply as a reference guide as you compile and submit documentation that contains the information (e.g., velocity reports/vendor summaries). |
|
The concluding paragraph:
Please let me know when you will be able to provide the requested information. If you have any questions about the study, you can visit the study website at www.PurchaseStudy.com or email the study team at PurchaseStudy@westat.com.
Respondents said to not to leave the data collection period open-ended, but to give a deadline. |
Include a definite submission data in request. Provide respondents 4 weeks after the close of the quarter to submit the data before sending a reminder email.
Please provide the requested food purchase information by [DATE]. If you have any questions about the study or this particular data collection, please visit the study website at www.PurchaseStudy.com or email the study team at PurchaseStudy@westat.com.
|
|
|
|
|
School Food Purchase Study-IV– Pretest Findings Report |
|
|
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Chantell Atere |
| File Modified | 0000-00-00 |
| File Created | 2021-01-13 |
© 2024 OMB.report | Privacy Policy