The U.S. Department of Agriculture’s (USDA’s) Food Safety and Inspection Service (FSIS) has been delegated the authority to exercise the functions of the Secretary of Agriculture (7 CFR 2.18, 2.53) as specified in the Federal Meat Inspection Act and the Poultry Products Inspection Act (21 U.S.C. 453 et. seq., 601 et seq.). FSIS protects the public by verifying that meat, poultry, and processed egg products are wholesome; not adulterated; and properly marked, labeled, and packaged.
USDA FSIS’s Office of Public Affairs and Consumer Education (OPACE) ensures that all segments of the farm-to-table chain receive valuable food safety information. The consumer education programs developed by OPACE’s Food Safety Education Staff inform the public on how to safely handle, prepare, and store meat, poultry, and egg products to minimize incidence of foodborne illness.
OPACE strives to continuously increase consumer awareness of recommended food safety practices with the intent to improve food-handling behaviors at home. OPACE shares its messages through The Food Safe Families campaign (a cooperative effort of USDA, the Food and Drug Administration (FDA), and the Centers for Disease Control and Prevention), earned media outreach; social media, AskKaren (an online database of frequently asked food safety questions), the FSIS web site, the Meat and Poultry Hotline, publications, and events. These messages are focused on the four core food safety behaviors: clean, separate, cook, and chill. Additionally, OPACE’s public education and outreach initiatives reach vulnerable and underserved populations.
By testing new consumer messaging and tailoring existing messaging, FSIS can help ensure that it is effectively communicating with the public and promoting behavior change with a goal of improving consumer food safety practices. This behavioral research will provide insight into the effect FSIS consumer outreach campaigns have on consumers’ food safety behaviors. The results of this research will be used to enhance messaging and accompanying materials to improve food safety behaviors of consumers. Additionally, this research will provide useful information for tracking progress toward the goals outlined in the FSIS Fiscal Years 2017–2021 Strategic Plan (USDA, FSIS, 2016).
To inform the development of food safety communication products and to evaluate public health education and communication activities, FSIS is requesting approval for a new information collection to conduct observational studies using an experimental design. Previous research suggests that self-reported data (e.g., surveys) on consumers’ food safety practices are unreliable because consumers tend to overreport their behavior (e.g., simply rinsing their hands instead of washing with soap and water for 20 seconds as recommended); thus, observational studies are a preferred approach for collecting information on consumers’ actual food safety practices (Mazengia et al., 2015; Sneed et al., 2015; Bruhn 2014; Phang & Bruhn, 2011; Anderson et al., 2004). These observational studies will help FSIS assess adherence to the four recommended food safety behaviors of clean, separate, cook, and chill; determine whether food safety messaging focused on those behaviors affects consumer food safety handling behaviors; and determine whether consumers introduce cross-contamination during food preparation. For this 3-year study, FSIS plans to conduct an observational study each year and focus on a different behavior, food and food preparation task, and food safety communication product each year. FSIS may decide to continue to conduct these studies annually and, if so, will request a renewal to extend the expiration date for the information collection request.
FSIS has contracted with RTI International to conduct the annual observational study. FSIS will use the findings from this study to address Objective 1.2.3 of the FSIS Fiscal Years 2017–2021 Strategic Plan (USDA, FSIS, 2016), which is to “increase public awareness of recalls, foodborne illness, and safe food handling practices.” This objective includes two measures: (1) “% increase in public awareness of safe food handling guidance and recalls through communication channels” and (2) “% increase of consumers identified who follow safe food handling behaviors.” The research findings will help to inform the development of strategic communication and outreach efforts and to evaluate the effect of these activities. By testing new consumer messaging and tailoring existing messaging, FSIS can help ensure that it is effectively communicating with the American public and promoting behavior change with a goal of increasing public awareness of foodborne illness and safe food handling practices, and ultimately increasing consumer adherence to the recommended safe food handling behaviors of clean, separate, cook, and chill. The remainder of this section provides an overview of the procedures for the annual observation study.
An observational study will be conducted each year following the approach described below. In each iteration of the observational study, participants will be assigned to a control group (no exposure to food safety messaging) or an intervention (treatment) group. Participants will be recruited and asked to come to the test kitchen location and provided with the recipes and ingredients needed to prepare the specific meat and/or poultry dish and a ready-to-eat dish. Each iteration of the study will focus on one of the four food safety behaviors (clean, separate, cook, or chill) and will also collect data on other food safety behaviors (e.g., washing hands before food preparation, washing hands after handling raw meat/poultry, using a separate or clean cutting board for raw meat/poultry and ready-to-eat foods) and whether cross-contamination occurred during meal preparation. The initial study will examine participants’ use of a food thermometer to determine if meat and poultry products are cooked to the proper temperatures (see Appendix A for a detailed description of the first study). The year 2 study is expected to examine the “clean” message, specifically the recommendation for consumers to not rinse poultry because of the potential to cause cross-contamination, and the year 3 study is expected to examine the “separate” message (keeping raw meat/poultry products separate from ready-to-eat products). The study materials contained in the appendices called out below are for the first study. These materials will be revised and customized for each year of the study and will be submitted to the Office of Management and Budget (OMB) for review before data collection in years 2 and 3.
Study participants will be recruited from the Raleigh-Durham area of North Carolina using convenience sampling via social media outlets (see Appendix B) and by sending emails to Expanded Food and Nutrition Education Program participants to reach low-income consumers (see Appendix C). Notices about the study will also be posted in approximately 100 grocery stores and food retailers of various sizes and price points throughout the Raleigh-Durham area of North Carolina (see Appendix D). To ensure enough adults with a high school/GED education or less are recruited, we may work with a local market research company in the Raleigh-Durham area and use convenience sampling and the OMB approved screening questionnaire to recruit adults with a high school/GED education or less from their database. The market research company will send a list of eligible individuals to the study team, who will contact eligible participants by phone to invite them to participate in the study and schedule an appointment. Recruitment materials (available in English and Spanish) will direct prospective participants to either call or email a study team member to be screened for eligibility or access a web link that will host the screening questionnaire (see Appendix E). As part of the screening process, data will be collected on participant and household demographics to ensure that the demographics of recruited participants are similar to those of the U.S. population based on Census data. Study enrollment will include contact by phone (see Appendix F) to schedule an appointment with individuals who meet the eligibility criteria followed by a confirmation email or letter (see Appendix G) and a reminder call 1 or 2 days before the scheduled appointment (see Appendix H). Participants will be told that study participation involves preparing several recipes and participating in a short interview. English-speaking and Spanish-speaking adults can take part in the study.
Before the observation and food preparation begin, the meat/poultry products will be inoculated with a harmless, realistic, and known amount of tracer bacteria or tracer virus such as bacteriophage MS2. Under video observation, participants will be asked to prepare two recipes: one made with a raw poultry product and one made with a ready-to-eat product. After receiving the appropriately assigned messaging (the treatment group will receive messaging on food safety specific to the behavior of interest), participants will receive instruction to cook the recipes in the order they would do so at home (see Appendix L).
The study will be conducted in North Carolina State University (NCSU) test kitchens (located in Raleigh, North Carolina) specifically designed for observation studies. Video recording equipment will be set up to record meal preparation. Trained research staff will conduct the video recording. Following the meal preparation and before clean-up, trained sample collectors will conduct surface swab sampling, and study staff will transport the samples to an NCSU testing laboratory.
Recording of food handling and meal preparation will begin as soon as the participant enters the test kitchen and will end after the participant leaves. Participants’ cleaning and sanitizing of equipment and environment before and after preparation of the recipe will also be recorded to evaluate intra-meal and inter-meal contamination risks (Redmond et al., 2004). Following the observation portion of the study, trained sample collectors will take surface swab samples from kitchen surfaces, utensils, food containers, appliance handles, kitchen towels, cutting boards, any devices (e.g., cell phones) that were touched during meal preparation, and the ready-to-eat dish (at least 15 sites in total). The swabs will be delivered to an NCSU testing laboratory and plated to determine the presence and concentration of the tracer. The presence of the tracer will indicate that cross-contamination occurred during food preparation. The level of cross-contamination will be compared across the sampling sites to determine the highest risk areas. Kitchen surfaces, appliances, and other potentially contaminated sites will be cleaned and sanitized after each participant uses the test kitchen to ensure that any bacterial or virus samples collected were from the participants’ behaviors.
Supplementing the observations, post-observation interviews (see Appendix I) will be conducted to provide insight into participants’ views, opinions, and experiences of their preparation practices of these products and to collect information on behaviors that could not be observed (e.g., storage of leftovers or thawing). Collecting qualitative data will allow the project team to connect the knowledge, attitude, and perceived behavior with actual observed practices, allowing for more targeted intervention refinement or development. A bilingual interviewer will conduct the interviews with Spanish-speaking participants.
Trained coders will use a coding rubric to evaluate the video observations based on the four food safety handling behaviors of clean, separate, cook, and chill. This rubric will be used to consistently define when a “good” (i.e., the recommended) behavior occurred (e.g., use of a food thermometer) or when one did not occur when it should have (e.g., did not wash hands after handling raw meat). Trained coders will watch the video observations and conduct the coding using the video observation rubric (see Appendix M). A sample of 10% of the observations will be coded by two coders to help ensure inter- and intra-coder reliability. Additionally, the coders will use notational analysis to assess recorded actions and their frequencies. Notational analysis is a generic tool used to collect observed events and place them in an ordered sequence and has been previously used in food safety research (Clayton & Griffith, 2004).
The Agency will use the findings of the observational studies to help FSIS assess adherence to the four recommended food safety behaviors of clean, separate, cook, and chill (based on actual, not self-reported behavior) and to determine whether food safety messaging focused on those behaviors affects consumer food safety handling behaviors and whether consumers introduce cross-contamination during food preparation. The findings from the observational studies will be used to inform the development of new and refined communication materials on food safety and foodborne illness prevention, thus helping to reduce the burden of foodborne illness in the United States.
Offering prospective participants the option to complete a web-based questionnaire for screening will be less burdensome and more cost-effective than requiring all prospective participants to call research staff to be screened for eligibility. Prospective participants who complete the web-based questionnaire and who meet the eligibility requirements for study participation will still need to be contacted via phone by research staff to schedule an appointment for completing the study.
FSIS reviewed existing research and concluded that the proposed data collection will not duplicate any similar study and the existing knowledge base and literature do not meet the Agency’s informational needs.
No small businesses will be involved in this collection.
FSIS will conduct an observational study each year to allow the evaluation of a specific behavior and related food safety communication material each year (i.e., year 1 will focus on the “cook” message, year 2 on the “clean” message, and year 3 on the “separate” message). A different set of individuals will participate in the study each year. If the data collection is conducted less frequently, then FSIS would be limited to evaluating only one specific behavior and related food safety communication. Proper safe food handling encompasses a variety of behaviors as evidenced by the number of behaviors asked about in the Food Safety Survey, an ongoing survey conducted by the FDA with support from FSIS to characterize consumers’ food safety practices when cooking at home. The most recent iteration of the survey (conducted in 2015–2016) found that some behaviors, such as handwashing, have remained constant or decreased in recent years after increasing for several years (FDA, 2016). Thus, there is the continued need to educate consumers about recommended food safety practices to increase knowledge and adoption of recommended behaviors. By conducting the annual observation study, FSIS will have a better understanding of whether food safety messaging focused on specific behaviors affects consumer behaviors.
requiring respondents to report information to the agency more often than quarterly;
requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;
requiring respondents to submit more than an original and two copies of any document;
requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records, for more than 3 years;
in connection with a statistical survey, that is not designed to produce valid and reliable results that can be generalized to the universe of study;
requiring the use of a statistical data classification that has not been reviewed and approved by OMB;
that includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or that unnecessarily impedes sharing of data with other agencies for compatible confidential use; or
requiring respondents to submit proprietary trade secret or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information’s confidentiality to the extent permitted by law.
This information collection fully complies with 5 CFR 1320.5(d) (2). There are no special circumstances associated with this information collection that would be inconsistent with the regulation.
In accordance with the Paperwork Reduction Act, FSIS published a 60-day notice requesting comments regarding this information collection request (81 FR 92772; 12/20/2016). The Agency received one comment from Ms. Jeanne Blankenship and Mr. Pepin Tuma of the Academy of Nutrition and Dietetics (the Academy). In addition to fully supporting the information collection, the Academy suggested six items FSIS should consider for the information collection. In the first three items, The Academy suggested we collect information from participants regarding thermometer ownership, literacy, and usage, including the calibration, cleaning, and storage of thermometers. In our response, we indicated that we will be able to collect most of this data via the observational study. The Academy also suggested that we include individuals from specific vulnerable populations (e.g., older adults, parents of young children, immunocompromised persons, etc.) in the study. In our response, we indicated that the sampling plan employs convenience sampling with quotas to ensure that study participants reflect the demographic characteristics of the U.S. population with regard to race, ethnicity, age, education, and household size; as such, individuals from vulnerable populations will most likely be included in the study. The Academy also suggested consumers be educated on the use of thermometers to improve safety and the culinary quality of food. In response, we stated that participants in the treatment group will watch a video with information on thermometer usage and that all participants will receive a free food thermometer.. In their final comment, the Academy suggested using technologies, such as Skype, to collect information from geographically dispersed participants. In our response, we stated they we will be unable to include the use of such technologies at this time because the study will be conducted in a test kitchen, and not in people’s homes.
The supporting statement was reviewed by the National Agricultural Statistics Service. Several revisions were made to address their comments, including clarifying the inclusion and exclusion criteria for recruiting study participants.
We understand that the OMB guidance about incentives for participation in research is based on the principles of the 2006 memo “Guidance on Agency Survey and Statistical Information Collections.” We propose providing each participant a $75 cash incentive and a small gift (food thermometer valued at $5.38) to ensure a high show rate for the observational studies and to improve data quality. Additionally, participation in observational studies requires substantial commitment and investment of time on the part of the participant, in that they must make a commitment to attend the study at a certain time on a specific date. Participation also requires participants to travel to a designated location, with the average commute in the United States metropolitan areas estimated at about 25.1 minutes (McKenzie & Rapino, 2011) and may also require that the participant obtain child care for a fee. Thus, providing incentives has long been considered a standard practice in conducting research such as observational studies.
Table A-1 provides a breakdown of the cost to participate in the observational study by subpopulation. Although the cost to participate varies depending on whether child care is needed (from $27.11 to $77.06), we propose to offer all participants the same incentive amount ($75) to avoid introducing selection bias that might occur by offering different incentive amounts to individuals with and without children in their households.
The proposed $75 incentive amount is in line with the industry standard. These industry-standard stipends help ensure that respondents can be recruited efficiently and ensure their arrival and participation in the study. These standards also exist to provide fair compensation for costs incurred by participants while participating in the study (i.e., travel and child care expenses). In addition to covering reasonable costs of participation, payment to participants is necessary to ensure that a sufficient number of respondents from the target population participate in the study. Payment to participants must encourage potential participants to agree to allocate their time to the study and maintain that commitment on the day of the research.
Offering no incentive or a smaller incentive could potentially exclude sections of the population who cannot participate in the study, either due to the cost of child care and/or travel or the cost of missing work. Excluding sections of the population would limit the information that would be gained through the observational studies and potentially bias the information needed to address the research questions of interest, thus negatively affecting data quality.
Moreover, the $75 incentive payment proposed is consistent with what OMB has approved for studies with a participant burden of 90 to 120 minutes, for example, OMB No. 0583-0166: Professional Services to Support Requirements Gathering Sessions for Safe Food Handling Instructions (SHI); OMB No. 0583-0141: Consumer Research, Assessing the Effectiveness and Application of Public Health Messages Affecting Consumer Behavior Regarding Food Safety; OMB No. 0920-0910: Perceptions of Health Risk from Smokeless Tobacco Products and Nicotine Replacement Therapy among Pregnant Women and Women Planning a Pregnancy; and OMB No. 0584-0561: Healthy Incentives Pilot Evaluation.
We anticipate that without the cash incentive and gift, we would need to screen more people to achieve the desired cooperation rate. The current estimated annualized burden for the participant screening is about 111 hours for each iteration of the study. Without any incentive, we expect that twice the number of individuals would need to be screened so that the total burden for screening would be about 222 hours for each iteration of the study. The cost to respondents and the federal government would increase accordingly.
Table A-1. Estimated Cost to Participants of Taking Part in the Observational Study by Households With and Without Children
Households with children |
|||
Cost Component |
Estimated Number of Units |
Unit Cost |
Total Cost |
Cost to travel to/from test kitchen |
50.2 milesa |
$0.54 /mileb |
$27.11 |
Cost of child care during travel time (1 hour round trip) and attending study (15 minutes before appointment to park and check-in, 2 hours for study, 15 minutes after group to check-out and receive incentive) |
3.5 hours |
$14.27/hourc |
$49.95 |
Total |
|
|
$77.06 |
Households without children |
|
|
|
Cost Component |
Estimated Number of Units |
Unit Cost |
Total Cost |
Cost to travel to/from test kitchen |
50.2 milesa |
$0.54/mileb |
$27.11 |
Total |
|
|
$27.11 |
a The average commute in a U.S. metropolitan areas is an estimated 25.1 minutes to the designated location (McKenzie & Rapino, 2011) for a total of 50.2 minutes. Assuming participants travel 60 miles an hour, the total number of miles is 50.2 miles.
b Source: http://www.gsa.gov/portal/content/100715
c Source: https://www.care.com/c/stories/2423/how-much-does-child-care-cost/
The privacy of the study participants will be assured by using an independent contractor to collect the information, by enacting procedures to prevent unauthorized access to respondent data, and by preventing the public disclosure of the responses of individual participants.
The only Information in Identifiable Form (IIF) that will be obtained are the participants’ names, phone numbers, and email or mailing addresses for scheduling the appointment for the observational study, mailing confirmation letters, and making reminder phone calls. This IIF will be maintained by NCSU. These personal identifiers will not be linked to data and will not be shared with FSIS or RTI.
Participation in the observational study and the post-observation interview are voluntary, and participants will be advised that their responses will be treated in a secure manner and will not be linked to their names. The digital video tapes will be stored on a password-protected share drive, accessible only to project staff.
Assurances of data privacy and security are documented in the informed consent form (see Appendix J). The study protocol and instruments were reviewed and approved by NCSU’s Institutional Review Board (see Appendix K).
During the observational study and post-observation interview, participants will not be asked any questions that are personal or sensitive in nature. However, during recruitment, prospective participants will be asked if they or household members have been diagnosed with cancer, diabetes, or other conditions that weaken the immune system. Individuals will not be asked for a specific diagnosis. Immunocompromised individuals are considered at risk for foodborne illness; thus, it is important to collect information on their or their caregivers’ food handling behaviors.
The total estimated burden for each iteration of the observational study is 1,036 hours (see Table A-2), for a total burden of 3,108 hours for 3 years. The study will be advertised via social media, emails, and postings in grocery stores, and prospective participants will complete a screening questionnaire by phone or via a web-link to determine eligibility (see Appendix E). We estimate that 833 individuals will complete the screener and 625 (75%) will be eligible and subsequently contacted by phone to schedule an appointment. Of these, we estimate that 500 (80%) will agree to take part in the study and an appointment will be scheduled. Of these, we estimate that 400 (80%) will show up and complete the observational study and post-observational interview. Each screening is expected to take 8 minutes (0.133 hour), each appointment call/confirmation email/reminder call is expected to take 7 minutes (0.116 hour), and each observation is expected to last 1.5 hours. Before the observation, each participant will read and sign the study’s consent form and watch one or more short videos related to the study, which will take no more than 10 minutes (0.17 hour) to complete. After each observation, participants will be asked to complete a short interview that will take up to 20 minutes (0.33 hour) to complete.
Table A-2. Estimated Annual Reporting Burden for Each Iteration of the Observational Study
|
Portion of Study |
Appendix(s) for Data Collection Instrument or Form |
No. of Respondents |
Annual Frequency per Response |
Total Annual Responses |
Hours |
Total Hours |
|
|---|---|---|---|---|---|---|---|---|
|
Recruitment Information |
B, C, D |
1,600 |
1 |
1,600 |
0.033 |
52.8 |
|
|
Screening questionnaire |
E |
833 |
1 |
833 |
0.133 |
110.789 |
|
|
Appointment phone script, confirmation email, reminder phone script |
F, G, H |
625 |
1 |
625 |
0.116 |
72.5 |
|
Consent form and videos |
J |
400 |
1 |
400 |
0.17 |
68 |
||
Food preparation task/observation |
L |
400 |
1 |
400 |
1.5 |
600 |
||
Post-observation interview |
I |
400 |
1 |
400 |
0.33 |
132 |
||
Total |
|
|
|
|
|
1036.089 |
||
The annualized cost to all respondents for the collection of information is $18,027.95 (1,036.089 x $17.40 per hour) (the 2015 U.S. median hourly wage rate; http://www.bls.gov/oes/current/oes_nat.htm) for each year of the 3-year study.
There are no capital, start-up, operating, or maintenance costs associated with this information collection.
The estimated total cost to the Federal Government for this information collection is $578,061. The costs arise from the time spent by the contractor to develop and conduct the study, analyze the data, and prepare and deliver a final report for all three iterations of the observational study.
This is a new information collection.
The planned schedule for the initial observational study is shown in Table A-3. Once OMB approval is received, it will take up to 30 days to begin recruiting individuals and conduct the data collection for the observational study and post-observational interviews. The contractor will provide FSIS a report that summarizes the study methods and results within 120 days of the last observation. Following the conclusion of the observation study, the contractor will conduct statistical analysis comparing the differences in handling behavior scores between the control and treatment groups for the four food handling behaviors. A comparative analysis will also be conducted on the samples collected from the designated kitchen sites and food samples to determine whether levels of cross-contamination differed between the two groups, as well as to identify the kitchen sites with the highest levels of contamination. There are no plans to publish the data for statistical use. Dissemination of the study results may include internal briefings, presentations, and reports and posting on FSIS’s website.
Table A-3. Project Schedule
Date |
Activity |
Within 30 days following OMB approval |
Begin observational study |
Within 150 days following OMB approval |
Complete data collection for observational study and post-observational interviews |
Within 270 days following OMB approval |
Complete summary report |
The OMB approval and expiration date will be displayed on all materials associated with the study. No exemption is requested.
There are no exceptions to the certification.
Anderson, J. B., Shuster, T. A., Hansen, K. E., Levy, A. S. & Volk, A. (2004). A camera’s view of consumer food-handling behaviors. Journal of the American Dietetic Association, 104(2), 186–191.
Bruhn, C. M. (2014). Chicken preparation in the home: An observational study. Food Protection Trends, 34(5), 318–330.
Clayton, D. A., & Griffith, C. J. (2004). Observation of food safety practices in catering using notational analysis. British Food Journal, 106(3), 211–227.
FDA. (2016). 2016 Food Safety Survey. Available at http://www.fda.gov/downloads/Food/FoodScienceResearch/ConsumerBehaviorResearch/UCM529481.pdf
Mazengia, E., Fisk, C., Liao, G., Huang, H., & Meschke, J. (2015). Direct observational study of the risk of cross-contamination during raw poultry handling: Practices in private homes Food Protection Trends, 35(1), 8–23.
McKenzie, B., & Rapino, M. (2011). Commuting in the United States: 2009. Supplemental Table C. Mean travel time to work by means of transportation and selected characteristics: 2009. Washington, DC: U.S. Census Bureau.
Phang, H. S., & Bruhn, C. M. (2011). Burger preparation: What consumers say and do in the home. Journal of Food Protection, 74(10), 1708–1716.
Redmond, E. C., Griffith, C. J., Slader, J., & Humphrey, T. J. (2004). Microbiological and observational analysis of cross contamination risks during domestic food preparation. British Food Journal, 106(8), 581–97.
Sneed, J., Phebus R., Duncan-Goldsmith, D., Milke, D., Sauer, K., Roberts, K. R., & Johnson, D. (2015). Consumer food handling practices lead to cross-contamination. Food Protection Trends, 35(1), 36–48.
USDA, FSIS. (2016). Food Safety and Inspection Service (FSIS) 2017–2021 Strategic Plan. Available at https://www.fsis.usda.gov/wps/wcm/connect/317d14d6-1759-448e-941a-de3cbff289e5/Strategic-Plan-2017-2021.pdf?MOD=AJPERES
| File Type | application/msword |
| Author | OPPDE/FSIS |
| Last Modified By | SYSTEM |
| File Modified | 2017-11-09 |
| File Created | 2017-11-09 |