The Food Safety and Inspection Service (FSIS) has been delegated the authority to exercise the functions of the Secretary (7 CFR 2.18, 2.53), as specified in the Federal Meat Inspection Act (FMIA) (21 U.S.C. 601, et seq.) and the Poultry Products Inspection Act (PPIA) (21 U.S.C. 451, et seq.). This statute mandates that FSIS protect the public by verifying that meat and poultry products are safe, wholesome, unadulterated, and properly labeled and packaged.
Safe handling instructions (SHI) are required on the labels of raw or partially cooked (i.e., not considered ready to eat) meat and poultry products if the product is destined for household consumers or institutional uses (9 CFR 317.2(l) and 9 CFR 381.125(b)). FSIS first required the SHI label for raw and partially cooked meat and poultry products in 1994 (54 FR 14528). Since that time, the required design of the SHI label has not been changed.
In response to inquiries from consumer groups and other stakeholders about potential changes to SHI requirements related to both label format and content, FSIS gathered input from members of academia, industry, and consumer stakeholders in November 2013. FSIS presented results of the input in the form of suggestions to the National Advisory Committee on Meat and Poultry Inspection (NACMPI) in January 2014. The NACMPI Subcommittee on Food Handling Labels recommended that FSIS pursue changes in the existing SHI label and conduct consumer research to determine the effectiveness of any revisions to the SHI label.
In 2015, FSIS conducted six consumer focus groups (OMB No. 0583-0166; 11/30/2017) to evaluate understanding of the current SHI label and responses to possible revisions. The results from the focus groups suggested that consumers would find certain revisions to the SHI label useful. Participants suggested changes to improve comprehension and adherence to recommended safe handling practices (e.g., add recommendations to use a food thermometer and endpoint temperatures for different cuts of meat and poultry). Based on the results of these focus groups, FSIS determined that additional research using more rigorous, quantitative approaches with a larger sample of consumers was needed to help inform potential revisions to the current SHI label and assess whether a label revision would improve consumer food safety behaviors.
In 2020, FSIS completed a study that comprised a web-based experimental survey and an experimental behavior change study that included meal preparation in a test kitchen environment, eye tracking, and in-depth interviews to design and evaluate potential revised labels for the SHI label (OMB No. 0583-0177; 4/30/2022). For this study, FSIS was somewhat constrained in terms of creating new labels based on recommendations from NACMPI, so the focus was on updating the existing text and icons in the current SHI label and adding information on recommended internal minimum temperatures for different cuts of meat and poultry. This study found that the three labels tested did not perform better than the current SHI label with regard to visual saliency (i.e., noticeability) and behavior change for the safe handling practices displayed on the label (e.g., using a food thermometer or washing hands with soap and water for 20 seconds and then drying) (Cates et al., 2020).
Consumer groups and other stakeholders have continued to advocate for improved labeling for raw and partially cooked meat and poultry products so that they safely prepare these products and thus help to prevent foodborne illness. In response, FSIS is pursuing research to create and assess consumer response to new labeling and plans to start with a “blank slate” (i.e., freely develop the label with no restrictions, ideas, or characteristics instead of simply revising the current SHI label). To do this we will take into consideration recent research via a literature review and input from key stakeholders and experts via listening sessions.
In December 2022 and January 2023, through a contractor, FSIS conducted listening sessions with consumer groups; representatives from the meat and poultry industry; and experts in health communication, food science, and food safety education to collect information on factors to consider when creating new labeling for raw and partially cooked meat and poultry products. Additionally, a literature review was conducted to identify and summarize best practices for label design for attracting attention and motivating behavior change and recommendations for label design based on human factors research. The findings from the listening sessions and literature review will be used for the current study to create new labels for safe handling of raw and partially cooked meat and poultry products. FSIS plans to use an iterative approach with multiple rounds of consumer research to obtain feedback on the new label designs and make refinements. In Phase 1, the new label designs will initially be tested in consumer focus groups to obtain qualitative feedback on the labels, and the findings will be used to refine the label design and messaging. In Phase 2, the revised label designs will be tested in an exploratory web survey, and the quantitative findings from this survey will be used to revise and prepare the final labels for testing in a web-based experimental survey to identify the top three performing labels based on the outcomes of interest such as noticeability, changed food safety–related beliefs, and induced thinking about the risks of contracting foodborne illness. In Phase 3, an observation study will be conducted to obtain empirical evidence on the label design that is most effective at motivating consumers to follow recommended safe handling practices for raw and partially cooked meat and poultry products and will include the use of eye tracking to measure visual saliency for the new labels compared with the control, the current SHI label.
FSIS is requesting approval for a new information collection to conduct Phase 1, consumer focus groups. FSIS will use the focus groups to test new labels for safe handling of raw and partially cooked meat and poultry products to provide qualitative feedback for refining the label design and messaging. FSIS has contracted with RTI International to conduct the focus group study.
A total of 12 focus groups will be conducted: three focus groups in four different locations (Northeast, South Atlantic, West South Central, and West). The focus groups will be conducted with adults (aged 18 years or older) to test new labels for safe handling of raw meat and poultry products and to provide qualitative information for refining the label design and messaging. To provide homogeneity, each focus group will comprise individuals from the subpopulations of interest: English-speaking individuals who are Hispanic (two groups), individuals who live in a rural location (one group), individuals with limited literacy (one group), individuals with limited cooking experience (i.e., beginner cooks) (two groups), parents or guardians of young children (5 years old or younger) (two groups), older adults (at least 65 years old) (two groups), and individuals who are caregivers for both their children and older adults (two groups). A screening questionnaire will be used to screen participants for eligibility (see Appendix A).
A moderator’s guide (see Appendix B) will provide structure for the focus group discussions and ensure that the topics of interest are addressed. Four new label designs, to be shared with the focus group participants, are currently under development. Participants will respond to each label individually, and the order of the labels will be rotated to prevent order effects. Additionally, different design options for a visual cue that could be shown on the front of a raw meat or poultry package to make consumers aware that the product is raw and needs to be handled safely to avoid illness will be tested.
The moderator guide will ask a series of questions to address participants’ response to each of the new label designs:
first impressions (likes, dislikes)
title (whether attention getting, recommended changes)
alert word such as “Caution” or “Attention” (whether attention getting, recommended changes)
format (whether attention getting, recommended changes)
comprehension (understanding of messaging for safe handling instructions and messaging that communicates the risk of not following the recommended instructions, usefulness of information, understanding of icons, recommended changes)
motivation (understanding of call to action, intentions to follow safe handling instructions, recommended changes)
QR code (likelihood of using a QR code, how to use code)
After participants have responded to each label, the moderator will lead participants in an activity to capture participants’ preferences for the best performing label across several different dimensions: (1) likes the most, (2) likes the least, (3) best catches their attention, and (4) encourages them to follow the recommended instructions. The labels will be displayed in the room and participants will be given different colored Post-It Notes to use to vote for their preferred label for each of the different dimensions, one at a time. At the end of the voting, the moderator will ask participants to talk about their votes and why they chose their preferred label. Again, this discussion will be done for one dimension at a time.
Next, participants, will independently complete a rank order exercise to rank their preferred title, alert word, risk message, and instruction length. The last topic of the moderator guide will address participants’ response to the visual cue concepts. Participants will independently complete an exercise to rank order the visual cues in terms of attracting their attention and then discuss their rankings, whether they would refer to such a visual cue, and the perceived usefulness of a visual cue. Participants will independently complete an exercise to rank order the visual cues in terms of attracting their attention and then discuss their rankings, whether they would refer to such a visual cue, and the perceived usefulness of a visual cue. Appendix C provides the materials that will be shared with participants during the focus groups, except for the labels and visual cues which are currently under development.
To provide information to interpret the study findings, we will audio- and video-record the focus groups, and the audio recordings will be transcribed. No electronic copies of the questions will be provided to the participants before the focus group discussions.
The Agency concluded that the proposed data collection will not duplicate any similar study, and the existing knowledge base and literature do not meet the Agency’s informational need.
Data will not be collected from small businesses.
This is a onetime data collection. Without this study, FSIS will not know consumers’ response to the different label designs. This lack of useful information would impede the Agency’s ability to creates a new label for safe handling of raw meat and poultry products that captures consumers’ attention and motivates them to follow recommended food safety practices. Improved consumer food safety practices may help to reduce foodborne illness in the United States.
requiring respondents to report information to the Agency more often than quarterly;
requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;
requiring respondents to submit more than an original and two copies of any document;
requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records, for more than 3 years;
in connection with a statistical survey that is not designed to produce valid and reliable results that can be generalized to the universe of study;
requiring the use of a statistical data classification that has not been reviewed and approved by the Office of Management and Budget (OMB);
that includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or that unnecessarily impedes sharing of data with other agencies for compatible confidential use; or
requiring respondents to submit proprietary trade secret or other confidential information unless the Agency can demonstrate that it has instituted procedures to protect the information’s confidentiality to the extent permitted by law.
This information collection fully complies with 5 CFR 1320.5(d) (2). No special circumstances are associated with this information collection that would be inconsistent with the regulation.
In accordance with the Paperwork Reduction Act, FSIS published a 60-day notice requesting comments regarding this information collection request (88 FR 30713 May 12, 2023). The Agency received no comments in response to Docket No. FSIS- 2023–0012.
We engaged seven people to test the moderator guide and estimate respondent burden by conducting a practice focus group discussion. The names, phone numbers, and length of time to take part in the practice focus group for three of these individuals are provided below:
Ashley Wheeler, 919-541-6171, 90 minutes
Ryan Haughney, 919-990-8429, 90 minutes
Catherine Sanders, 919-515-3007, 90 minutes
This information corroborates our estimated burden of 90 minutes.
We understand that the OMB guidance about incentives for participation in research is based on the principles of the 2006 memo: Guidance on Agency Survey and Statistical Information Collections. We propose providing each focus group participant a $75 gift card and a small gift (food thermometer valued at $5.38 and a magnet valued at $0.23) so that we can effectively recruit hard-to-find populations, ensure a high show rate for the focus groups, and improve data quality. Additionally, participation in focus groups requires substantial commitment and investment of time on the part of the participant in that they must commit to attend the discussion at a certain time on a specific date. Participation also requires participants to travel to a designated location, with the average commuting time in U.S. metropolitan areas estimated at about 27.6 minutes (U.S. Census Bureau, 2019) and may also require that the participant obtain childcare for a fee (for the participants who may have children). Thus, incentives have long been considered a standard practice in conducting qualitative research such as focus groups.
Table A-1 provides information on the cost to participate in the focus group discussion. Although the cost to participate varies depending on whether the participant needs childcare for their child while attending the focus group (from $36.16 to $96.16), we propose to offer all participants the same incentive amount ($75) to avoid introducing selection bias that might occur by offering different incentive amounts to different subpopulations. The proposed $75 incentive amount is in line with the industry standard. These industry-standard stipends help ensure that respondents can be recruited efficiently and ensure their arrival and participation in the groups. These standards also exist to provide fair compensation for costs incurred by participants while attending groups (i.e., travel and childcare expenses). In addition to covering reasonable costs of participation, payment to participants is necessary to ensure that a sufficient number of respondents from the target populations participate in the study. Payment to participants must encourage potential participants to agree to allocate their time to the focus group discussion and maintain that commitment on the day of the research.
Table A-1. Estimated Cost to Participants of Taking Part in the Focus Group Discussion by Whether the Participant Needs to Retain Childcare
Cost Component |
Estimated Number of Units |
Unit Cost |
Total Cost |
Households with Children |
|||
Cost to travel to/from facility |
55.2 milesa,b |
$0.655/milec |
$36.16 |
Cost of childcare during travel time (1-hour round trip) and attending the focus group (15 minutes before appointment to park; 90-minute discussion; and 15 minutes after group to checkout, receive incentive, and return to vehicle) |
3.0 hours |
$20/hourd |
$60.00 |
Total |
|
|
$96.16 |
Households without Children |
|
|
|
Cost to travel to/from facility |
55.2 milesa,b |
$0.655/milec |
$36.16 |
Total |
|
|
$36.16 |
a Source: Travel Time to Work in the United States: 2019 (census.gov)
b The average commute in a U.S. metropolitan area is an estimated 27.6 minutes to a designated location. Assuming participants travel 60 miles per hour, the total number of roundtrip miles is 55.2 miles.
c Source: http://www.gsa.gov/portal/content/100715
d Source: Average Babysitting Rate: What Should You Pay for Child Care? | Sittercity 2023 hourly rate for nanny
Offering no incentive or a smaller incentive could potentially exclude sections of the population who cannot attend the groups, either because of the cost of travel or childcare or the cost of missing work. Excluding sections of the population would limit the qualitative information that would be gained through the focus group discussion and potentially bias the information needed to address the research questions of interest, thus negatively affecting data quality.
Moreover, the $75 incentive payment proposed is consistent with what OMB has approved for other 90-minute focus group studies, for example, OMB No. 0583-0184: Focus Group Research to Inform Consumer Food Safety Education and Outreach Efforts; OMB No. 0583-0173: Food Safety Behaviors and Consumer Education: Focus Group Research; OMB No. 0583-0166: Professional Services to Support Requirement Gathering Sessions for SHI; and OMB No. 0583-0141: Consumer Research, Assessing the Effectiveness and Application of Public Health Messages Affecting Consumer Behavior Regarding Food Safety.
To encourage recruited individuals to not only attend but also to arrive on time for the focus group discussions, we will include all those who sign in 15 minutes before each group is scheduled to start in a drawing for a chance to win an extra $25 (in the form of a gift card). We anticipate that without the incentive and drawing, we would need to screen more people to achieve the desired cooperation rate. The current estimated annualized burden for the participant screening is about 102.14 hours. Without the incentive, we expect the burden to be approximately 255 hours, an increase of approximately 150%. The cost to participants and the federal government would increase accordingly.
The privacy of the focus group participants will be ensured by using an independent contractor (RTI) to collect the information, by enacting procedures to prevent unauthorized access to respondent data, and by preventing the public disclosure of the responses of individual participants.
The only Information in Identifiable Form (IIF) that will be obtained is the participants’ names, phone numbers, and email or mailing addresses for sending confirmation letters and making reminder phone calls. This IIF will be maintained at each of the local market research firms in their own proprietary files. These personal identifiers will not be linked to data and will not be shared with FSIS or RTI. The only IIF that RTI will obtain is the participants’ names via the signed informed consent. These data will not be shared with FSIS.
Participation in the focus groups is voluntary, and participants will be advised that their responses will be treated in a secure manner and will not be linked to their names. During the focus groups, only first names will be used. The focus group discussions will be transcribed for use by the RTI research team in developing a report, but participants’ first names will be replaced with the word “participant.” The digital audio and video files will be stored on a password-protected share drive, accessible only to RTI project staff and destroyed after 5 years.
FSIS staff may view the focus groups in person (by watching from an observation room through a one-way mirror) or via live video stream. FSIS will not have access to the digital audio and video files of the focus group discussions.
Assurances of data privacy and security are documented in the informed consent form (see Appendix D). The study protocol and instruments were reviewed and deemed exempt by RTI’s Institutional Review Board (see Appendix E).
RTI and FSIS will not have access to focus group participants’ personal information. No personally identifying information will be included in the data files delivered to the Agency. The focus group study conducted by the subcontractor is considered a service; therefore, a Privacy Threshold Analysis and Privacy Impact Assessment are not required.
During the focus group discussions, participants will not be asked any questions that are personal or sensitive in nature.
For the 12 focus groups, it is expected that 768 individuals will complete the screening questionnaire. It is estimated that 144 individuals will be eligible and agree to participate. Among the 144 individuals recruited for the focus group study, it is estimated that 96 will actually show up and participate in the focus group study (eight people per group. Each screening questionnaire is expected to take 8 minutes (0.133 hour). Participating in the focus group discussion will take 90 minutes (1.5 hours). Table A-2 details the estimated annual reporting burden. The estimated annual reporting burden for the 12 focus groups is 246.14 hours, which is the sum of the burden estimates for the screening and focus group discussion for respondents and nonrespondents.
The annualized cost to all respondents for collecting the information is $7,039.60 (246.14 x $28.60 per hour [the 2021 U.S. median hourly wage rate of $22 plus 30% fringe]). See http://www.bls.gov/oes/current/oes_nat.htm.
Table A-2. Estimated Annual Reporting Burden for the Focus Group Study (12 groups)
|
|
|
Respondents |
Nonrespondents |
|
|
|||||||||||
Study Component |
Sample Size |
Annual Frequency per Response |
Count |
Freq x Count |
Hours per Response |
|
|
|
|
|
Total Hours |
||||||
Screening question-naire |
768 |
1 |
144 |
144 |
0.133 (8 min.) |
19.15 |
624 |
624 |
0.133 (8 min.) |
82.99 |
102.14 |
||||||
Focus group discussion |
96 |
1 |
96 |
96 |
1.5 (90 min.) |
144.0 |
48 |
48 |
0 |
0 |
144.00 |
||||||
Total |
|
|
|
|
|
163.15 |
|
|
|
82.99 |
246.14 |
||||||
No capital, start-up, operating, or maintenance costs are associated with this information collection.
The estimated annual cost to the federal government is $155,217. The costs arise from the time spent by the contractor to develop and conduct the study, analyze the data, and prepare and deliver a final report.
This is a new information collection.
The planned schedule for this information collection is shown in Table A-3 for the focus groups. Once OMB approval is received, it will take up to 60 days to recruit individuals and conduct the 12 focus groups. RTI will provide FSIS a summary report of the focus group discussions within 60 days of the last focus group. No statistical analyses will be conducted, and there are no plans to publish the data for statistical use. Dissemination of the study results may include internal briefings, presentations, and reports and posting on FSIS’s website.
Table A-3. Project Schedule for Focus Groups
Date |
Activity |
Within 60 days following OMB approval |
Conduct 12 focus groups |
Within 60 days following last focus group |
Completion of focus group summary report |
The OMB approval and expiration date will be displayed on all materials associated with the study. No exemption is requested.
There are no exceptions to the certification.
Cates, S. C., Blitstein, J., Shumaker, E., Ramirez, D., Brophy, J., Hayes, M., Chapman, B., Binder, A., Shelley, L., Goodson, L., Goulter, R., & Jaykus, L-A. (2020, September 23). Modernizing safe handling and ready-to-eat/not ready-to-eat labeling instructions: Behavior change study. U.S. Department of Agriculture, Food Safety and Inspection Service. https://www.fsis.usda.gov/sites/default/files/media_file/2022-02/SHI_Behavior_Change_Study%20Final%20Report_9_23_20.pdf
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Boykin, Catherine |
| File Modified | 0000-00-00 |
| File Created | 2023-10-26 |