SSB Ass Ill Wrkr Plcy Stdy

Supporting Statement Part B

Assessment of Ill Worker Policies Study

OMB No. 0920-NEW

New Information Collection Request

Supporting Statement - B

Submitted: September 2017

1. Program Official:

2. Adam Kramer Sc.D., M.P.H., R.S.

3. Environmental Health Officer

4. Centers for Disease Control and Prevention

5. National Center for Environmental Health

6. Emergency and Environmental Health Services

7. Environmental Health Services Branch

8. 4770 Buford Highway, NE F – 58

9. Atlanta, GA 30341-3724

10. 404-498-1228 (Phone)

11. 770-488-3635 (Fax)

12. E-mail: ank5@cdc.gov

Table of Contents

20. 1. Respondent Universe and Sampling Methods

2. Procedures for the Collection of Information

3. Methods to Maximize Response Rates and Deal with Nonresponse

4. Test of Procedures or Methods to be Undertaken

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

List of Attachments

Attachment 1 – Authorizing Legislation

Attachment 2 – 60-day Federal Register Notice

Attachment 3a – Industry Partners

Attachment 3b – Publications

Attachment 4 – Telephone Manager Recruiting Script

Attachment 5 – Manager Informed Consent and Interview Form

Attachment 6 – Manager Interview Marking Guide

Attachment 7 – Food Worker Informed Consent and Survey Form

Attachment 8 – Restaurant Environment Observation Form

Attachment 9 – Guide To Developing A Restaurant Ill Worker Management Plan

Attachment 10 – NCEH-ATSDR Research Determination Form

B. Collections of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

CDC’s EHS-Net program is comprised of retail food establishments in selected geographical areas in: California; Minnesota; New York City; New York State; Rhode Island; Tennessee; Southern Nevada Health District, NV; and Harris County, TX. While the number of areas included in EHS-Net is small, they are demographically diverse and provide good geographical coverage of the U.S. (northeast, mid-west, south, and west). When the sampling methods outlined here for ensuring a representative sample in the current study are used, the results of the collection covered by this OMB package can be used to generalize to the population of retail food establishments in the given EHS-Net site(s).

The respondent universe is all retail food establishments (hereafter referred to as restaurants) in the EHS-Net catchment area. Restaurant lists will be obtained from the restaurant databases maintained by the EHS-Net sites. CDC will use these restaurant lists to generate the sampling frame used to draw the sample for this study.

Each site will be required to enroll 40 restaurants (Table B.1). The restaurants will be evenly split into intervention and control restaurants (20 intervention and 20 control restaurants per site). Since there are no previously published (population) studies that have examined the prevalence and content of ill worker management practices in restaurants as this time, we are unable to determine whether this sample size will be able to support at least an 80% study power to detect statistical differences between study groups. Thus, data on expected prevalence of practices between different groups of restaurants are not available as inputs for proper calculation of study sample size and power. Modeling of various scenarios supports that this sample size should be sufficient to identify approximately a 15% or greater difference between groups (Figure B.1) with an alpha of 0.05 and beta of 0.8. Experience from prior EHS-Net studies also indicates that a sample size of 320 should be sufficient for the analytic purposes outlined below, since the analytic parameters are not likely to be considered rare (in distribution) events. Data collected from this study will provide the necessary information for sample size and power calculation for future studies.

Table B.1

Strata (EHS-Net Sites)	Entity	Number of Entities
California	Restaurants	40
Minnesota	Restaurants	40
New York	Restaurants	40
New York City	Restaurants	40
Rhode Island	Restaurants	40
Tennessee	Restaurants	40
Southern Nevada Health District	Restaurants	40
Harris County, Texas	Restaurants	40

Figure B.1

The design is a quasi-experimental non-equivalent group pre- post-test design with a stratified random sampling plan in which each site serves as its own mutually exclusive stratum. There are two primary reasons for stratifying by site. The first is that food safety regulations vary by jurisdiction. For example, Tennessee state food safety regulations differ from New York state food safety regulations. These regulations can and do greatly influence restaurants’ food safety practices and policies. EHS-Net site/jurisdiction, therefore, poses as the largest source of variability from a study design perspective. Thus, it is a critically important factor for stratification. The second reason for stratifying by EHS-Net site only is due to practical concerns that limit our ability to stratify on other variables of interest. EHS-Net sites participate in EHS-Net through a cooperative agreement. See Table B.2 for EHS-Net sites’ cooperative agreement numbers. The nature of this agreement is such that one site cannot be expected to do a disproportionate amount of work in comparison to other sites (because each site receives relatively equal funding amounts). If we did not stratify by EHS-Net site but by some other factor such as ownership (independently owned or belonging to a corporate regional chain), it is likely that some sites would have to carry a greater burden than other sites in term of recruiting and collecting data in a larger number of restaurants. However, we will be collecting data on these factors of interest and will account for their heterogeneity through statistical modeling. Finally, the need for each site to share an equal burden in data collection is the reason why a fixed-sample allocation method was used for each site (40 establishments per site), instead of a proportionate-sample allocation.

Table B.2

EHS-Net Sites	CDC-NCEH Cooperative Agreement Numbers
California	EH001299
Minnesota	EH001295
New York	EH001296
New York City	EH001300
Rhode Island	EH001293
Tennessee	EH001294
Southern Nevada Health District	EH001301
Harris County, Texas	EH001297

Restaurants will be randomly selected, with equal probability, within their respective EHS-Net site, independent of other sites. This process will give each restaurant in a particular sampling frame the same probability of being selected for study participation. There are three reasons for employing this sampling strategy: reducing sampling error, maintaining equal representation by site, and ensuring generalizability. First, as stated previously, the total target population of restaurants from all EHS-Net sites combined constitutes a highly heterogeneous group. To control for such heterogeneity in the total sample, restaurants will be stratified by EHS-Net site so they can be grouped into more homogeneous strata and then sampled within stratum independently. This reduction in heterogeneity of the total sample will lead to reduction in sampling error, which can improve representativeness of the selected sample and provide estimates (e.g., means) that tend to have less variability than estimates produced from samples that were drawn using the un-stratified, simple random sampling method. Second, with equal allocation of samples (40 restaurants per site), each EHS-Net site will have equal representation in the parameter estimates of the combined sample. An additional benefit is that even sites with small sampling frames will have sufficient data points to support their site-specific analyses. Third, by ensuring that the sampling of restaurants is done by an entity (CDC) separate from the data collectors (EHS-Net sites) and employing a random selection method, we are able to minimize the potential for selection bias. Parameter estimates or study findings obtained from an unbiased study sample could be generalized to the entire EHS-Net target population.

The average response rate across EHS-Net studies that used methods similar to the proposed study is 45% (L. Brown et al., 2016; Laura G Brown et al., 2014; Radke et al., 2016). We expect a similar response rate for the proposed study.

2. Procedures for the Collection of Information

As indicated earlier, each EHS-Net site will provide CDC with a list of all restaurants in their catchment area. This list will serve as the sampling frame for the site. CDC will use a random number generator in SAS 9.3 to produce a random sample of restaurants from this restaurant list for each site. As we expect some restaurants will refuse to participate and some will be ineligible to participate, we will select more than the needed number of restaurants--100 restaurants for each site. Once they receive their sample list from CDC, personnel in each site will contact restaurants by telephone to recruit their participation in the study. If the manager is willing to participate, the EHS-Net specialist will arrange a mutually convenient time to conduct the data collection.

In instances where an EHS-Net site is unable to recruit 40 restaurants from the first list of 100 restaurants, CDC will randomly select another group of 50 restaurants for the site to use to recruit additional respondents. Recruitment will be considered complete once data are collected in 40 restaurants. EHS-Net sites will recruit via the telephone and will keep a log of each contact with the restaurants to document participation rates and reasons for refusal and/or ineligibility.

CDC will not know which restaurants on the sample lists participated in the study, and thus will not be able to link restaurant names with any study data. The restaurant identifying information will be maintained by the respective EHS-Net site to facilitate the follow-up site visits, but will be destroyed when it is no longer needed. Additionally, on all forms only the specific coded restaurant identifier will be used to minimize a risk of someone inadvertently seeing a completed form and being able to associate it with a specific restaurant.

Data will be collected in the restaurants by the environmental health specialists. For the manager interview portion of the study, the EHS-Net specialist will obtain verbal informed consent and then conduct a face-to-face interview with a manager who has authority over the kitchen and can speak English well enough to complete the interview in English. This interview will include questions on restaurant characteristics, policies, and procedures for managing ill workers, along with manager characteristics and knowledge, attitudes, and practices to managing employee illness. This will take about twenty minutes to complete. For intervention restaurants, an educational intervention will be introduced indicating the need for ill worker management plans and introduction of a toolkit for making their plans.

For the worker survey portion of the study, the data collector will obtain verbal informed consent from and then administer a written survey (Attachment 7). The survey will include questions on worker characteristics and their level of agreement with restaurant food safety practices. The survey will be administered to all workers that handle food and voluntarily agree to participate. The survey can be administered in English or Spanish, and completed anytime during the visit.

For the restaurant environment observation (Attachment 8) portion, the data collector will observe the kitchen and note the presence or absence of practices that would minimize the spread of illness from an ill employee.

Both intervention and control restaurants will have a follow up site visit with a similar data collection around 4-6 months of the initial visit. During the site visit, similar interviews, survey, and observations will occur as occurred during the initial site visit. If the intervention is showing success in the intervention restaurants it will be provided to the control restaurants at this second visit, and a second follow up visit will be conducted in an additional 3-6 months.

The data collectors are experienced and knowledgeable in environmental health and food safety and will have received training from CDC on data collection for this study. The EHS-Net administrator in each EHS-Net site and CDC staff will perform quality assurance procedures to check for data entry errors.

Managers’ concerns about the safety practices of their restaurants may result in selection bias- a lower rate of study participation among restaurants with worse or non-existent safety practices compared to restaurants with better safety practices. We have conducted studies using methods similar to those used in this study in the past, and these studies have found a wide range of food safety practices, including poor ones (Bogard, Fuller, Radke, Selman, & Smith, 2013; Laura Green Brown et al., 2012; Coleman, Delea, Everstine, Reimann, & Ripley, 2013; Green Brown, Khargonekar, & Bushnell, 2013; Green et al., 2006; Kirkland et al., 2009; Lee et al., 2004; Sumner et al., 2011). While the potential for selection bias exists, these studies indicate that these biases may be minimal. Plus, the study protocol incorporates procedures to minimize the potential for and to detect any indication of selection bias. For example, EHS-Net staff will be trained in the recruitment process in order to keep non-response rate as low as possible, which will help minimize selection bias.

The interview and survey data collected for this study may be influenced by social desirability bias- the tendency for people to report greater levels of socially desirable behavior (such as not working while ill) than they actually engage in, or to report their best behavior rather than their typical or worst behavior. Although it is difficult to eliminate this bias altogether, it can be limited by ensuring respondents that the information they report will be anonymous, which we will do (Leary, 2004).

Due to logistical limitations, we will only interview managers that speak English well enough to be interviewed, and provide a written survey to workers in English or Spanish. The food workers will need to have sufficient literacy to read in their preferred language. The use of this criterion may introduce bias, as less literate individuals may have different food safety knowledge, attitudes and practices than more literate workers, but the resources are not available to include non-English speaking staff in the study or translate all of the documents into the potential universe of languages spoken in restaurants.

Any presentation of data from this study will acknowledge these potential biases and include a discussion of how they impact data interpretation.

3. Methods to Maximize Response Rates and Deal with Nonresponse

We will engage in several activities designed to maximize response rates. First, all recruiters will receive training on the recruiting process that will be locally developed by EHS-Net sites. Second, multiple attempts will be made to contact potential respondents. Specifically, recruiters will make 10 attempts over 5 days to get a participation response from establishments they have not been able to contact, and 5 attempts over 5 days to get a participation response from restaurants that have not provided a response (e.g., ‘call back later’). Third, recruiting scripts will emphasize three issues that have been shown to increase response rates—the anonymous nature of the data collection, the importance of the respondents’ participation in the study, and the additional training materials to reduce the incidence of ill employees working.

4. Test of Procedures or Methods to be Undertaken

The data collection materials and methods were based on those used in other previous, successful EHS-Net studies (Bogard et al., 2013; Laura Green Brown et al., 2012; Coleman et al., 2013; Green Brown et al., 2013; Green et al., 2006; Kirkland et al., 2009; Lee et al., 2004; Sumner et al., 2011). All data collection materials were reviewed and evaluated by key EHS-Net specialists whom are experienced with collecting data for EHS-Net studies. They were also reviewed by CDC EHS-Net personnel with extensive experience in developing and conducting EHS-Net studies. Additionally, all data collection materials were evaluated in pilot tests with 9 retail food establishments. Given that we are experienced in collecting data from retail food establishments with these types of instruments and methods (this is will be the tenth multisite study we have conducted in retail food establishments using similar data collection instruments and methods), we are confident that the study is designed well and do not anticipate the need to make changes to the data collection instruments. If we do need to make changes as a result of the pilot, we anticipate that they will be minor. OMB will be notified of any changes to the data collection instruments through the non-substantive change request.

Data Analysis Plan. The primary purpose of this data collection is to examine whether an educational intervention can lead to an increase in written ill worker management plans. We also wished to understand the knowledge, attitudes, and practices of food service workers to working while ill. To address the purpose of this data collections, we will conduct descriptive analyses (frequencies, means, etc). Tables B.4.1-4.3 contain the variables included in these analyses. Table B.4.4 contains variables used to describe the restaurant, along with the managers and food workers. Table B.4.5 is a table shell that illustrates how we may analyze and present the descriptive data collected from this study.

Table B.4.1 Interview variables measuring the change in ill worker policies

Item Content	MI#
Policy existence
Does this restaurant have a policy about what to do if an employee is sick	26
Is the policy written or verbal	27
Are employees trained on this policy	28
How are employees trained on this policy	29
Who does this policy apply to	32,33
Who in the restaurant would be able to make changes to this restaurants ill worker policy	43
Reporting of illness
Do you require employees to let you know when they are sick	34
Does the restaurant maintain a log of when employees call in sick or are sent home sick	35
Approximately how many employees have you had that were out sick over the past month	36
When you hire a new employee do you ask about prior illnesses	39
Restriction/Exclusion of ill employees
What symptoms or illnesses does the policy cover and what actions do you take	30
When to allow return to work
If you send an employee home or they call in sick what criteria do you use to let them return	31
Practices to reduce the likelihood of an employee working while ill
Do you have any processes or procedures in place to keep ill workers from working	41
What processes or procedures have you implemented	42
Practices to minimize the spread of illness from an ill food worker or the environment
Do you have written policies for cleaning of the establishment	45
Does the policy differentiate between the routine cleaning and whether a bodily fluid is spilled	46
Are there specific policies to address cleaning of vomit or diarrhea	47
Does the policy include how to clean up the vomitus or feces	48
How to disinfect the area	49
What type of sanitizer do you use and at what concentration for disinfecting these incidents	50
Do employees use any personal protective equipment while cleaning these incidents	51
What type of equipment is used	52
What do you do to food that may have been potentially exposed	53
What do you do with the plates or other utensils that may have been exposed	54
Does this restaurant allow employees to handle ready to eat foods with their bare hands	23

MI = Manager Interview

Table B.4.2 Interview/Survey variables measuring ill worker knowledge, attitudes, and practices

Item Content	MI#	WS#
Knowledge
Are you able to ask employees about any illness symptoms or diagnoses they may have	25
Attitudes
In your opinion, which of the following processes would be most likely to be adopted by this establishment (Paid or additional sick leave; Maintaining an on-call employee schedule; Paying for employee immunizations; Subsidizing employee health insurance; allowing employees to make up missed shifts)	44
When I am sick I choose to come to work		6
Management stresses food safety rules even when the restaurant is busy		9
I always consider food safety when I am doing my job		10
Practices
Do you actively look for signs or symptoms of illness in your employees	37
What symptoms do you look for	38
What do you do when an employee calls in sick	40
When I am sick my manager encourages me to stay home		7
Even if no one was looking, employees follow all food safety rules		8

MI = Manager Interview WS = Worker Survey

Table B.4.3 Observation variables measuring practices that minimize the spread of illness from a worker

Item Content	OI#
Do any of the following have bare hand contact with ready to eat foods that are not subject cooking afterwards (Employees working on cook line; doing food preparation; servers; other)	1
Is bare hand contact with ready to eat foods permitted in the jurisdiction	2 (MI#23)
Are handwash sinks properly stocked and available	3
Are employees properly washing their hands (Employees working on cook line; doing food prep; Servers; doing warewashing)	4
Does the facility have the equipment/materials to clean up an episode of vomiting/diarrhea that they reference in the manager interview	5

OI = Site Survey MI = Manager Interview

We will also need to describe the restaurants, managers, and workers from which we collect the data. Table B.4.4 contains the variables needed for these analyses.

Table B.4.4 Interview/Survey variables

Item Content	MI#	WS#
Restaurant
Which of the following options best describes the restaurant style (Family style, Fast casual, Fast food, Fine dining, Buffet, Café/bistro, other)	13
What is your approximate sales per customer	14
What is your average number of transactions or tickets per day	15, 16
What is your average employee turnover per month	18
In general, what is the average length of employment for (Managers, Chefs, Cooks)	19a-b
Is the person in charge of the kitchen a Certified Kitchen Manager	22
Does the restaurant have a Certified Kitchen Manager for all hours of operation	24
To the best of your knowledge has this restaurant ever had an incident of vomiting or diarrhea that required cleaning	55
Employee
How many years have you worked in food service	1	1
Have you ever had food safety training	2	2
Have you ever been certified in food safety (such as with the ServSafe ® Food Safety Manager Course)	3	3
How long have you been employed at this food service establishment	4	4
What title would best describe your position	5
What area of the kitchen do you primarily work in		5

MI= Manager Interview WS = Worker Survey

Table B.4.5- Table Shell: Descriptive data on existence of ill worker policies

	n	%
Restaurant has ill worker policy (MI26)
Yes	xx	xx
No	xx	xx
Restaurant has written ill worker policy (MI27)
Yes	xx	xx
No	xx	xx
Ill worker policy meets minimum code requirement (MI30-31,34)
Yes	xx	xx
No	xx	xx
Ill worker policy contains provisions to encourage workers to not work while ill (MI41)
Yes	xx	xx
No	xx	xx
Most common practices to keep ill workers from working (MI42)
Paid sick leave	xx	xx
On-call employee worker schedule	xx	xx
Allowing make-up shifts	xx	xx
Subsidized health insurance	xx	xx
Other	xx	xx

The primary purpose of this study is to assess whether an educational intervention led to A) the creation of ill worker policies; B) the expansion of the existing policies; and C) the inclusion of incentives for workers to not work while ill. To address this purpose, a chi-square analysis will be conducted to see if there was a statistically significant difference between the intervention and control restaurants in their implementation of the outcomes. If a statistically significant difference is detected, we will conduct tests of association with logistic regression models. Analysis will involve bivariate tests for association between each individual explanatory (independent) variable and the outcome (or dependent) variables of interest. Odds ratios will be calculated to assess the strength and direction of the bivariate relationships. For those bivariate associations found to be statistically significant at p<.30, the explanatory variables will be used as candidate “predictors” to examine their multivariate relationships with the outcome variables. Multivariable logistic regression will be used to model for the effects that these explanatory variables have in explaining the variations observed in the outcome variables.

The secondary purpose of this study is to assess the relationships among A) restaurant and staff characteristics and B) restaurant staff ill worker attitudes and practices. To address this purpose of this data collection, we will conduct tests for association and logistic regression models. Analysis will involve bivariate tests for association between each individual explanatory (independent) variable and the outcome (or dependent) variables of interest. Odds ratios will be calculated to assess the strength and direction of the bivariate relationships. For those bivariate associations found to be statistically significant at p<.30, the explanatory variables will be used as candidate “predictors” to examine their multivariate relationships with the outcome variables. Multivariable logistic regression will be used to model for the effects that these explanatory variables have in explaining the variations observed in the outcome variables.

Explanatory variables in these analyses include those measuring restaurant and staff characteristics. Outcome variables include those measuring staff ill worker attitudes and practices. However, analyses will focus on key practices. Table B.4.6 lists the key explanatory variables and key practice outcome variables based on the individual variables listed in the table. Table B.4.7 is a table shell that illustrates how we might analyze and present the data examining the relationships between restaurant and staff characteristics and key practices.

Table B.4.6- Key explanatory and practice outcome variables included in explanatory analyses

Explanatory variables	Outcome variables
Restaurant characteristics Industry segment, (MI13) Volume of business, (MI14-16) Turnover of staff, (MI18-19) Certified kitchen manager present, (MI22,24) Previous incident, (MI55) Employee characteristics Years of experience, (MI1,4, WS1,4) Food safety training, (MI2-3, WS2-3) Position in restaurant, (MI5, WS5)	Existence of ill worker policies, (MI26-34, 37-39) Expanded ill worker policies (includes addition of provisions), (MI41) Presence of incentives to not work while ill, (MI42) Existence of practices to prevent contamination from the environment, (MI45-54, OBS1-5) Employees working while ill is a problem, MI7,10,12, WS7) Attitude towards food safety practices, (WS6, 8-10, OBS1-5)

MI=Manager Interview, WS=Worker Interview, OBS=Observation

Table B.4.7- Table Shell: Key restaurant and manager characteristic explanatory variables associated with the practice outcome variable of whether the restaurant has food allergen plans, bivariate analyses

Explanatory variables	Restaurant has Ill worker management plans (MI: 26-27)
	OR (95% CI)	P
Restaurant characteristics
Volume of business (MI15-16)
> xxx	x.xx (ref)	.xxx
< xxx	x.xx
Turnover of staff (MI18-19)
> xx	x.xx (ref)	.xxx
< xx	x.xx
Certified kitchen Manager (MI22,24)
Some hours	x.xx (ref)	.xxx
All hours	x.xx
Not present	x.xx
Industry Segment (MI13)
Fast food	x.xx (ref)	.xxx
Fast casual	x.xx
Fine dining	x.xx
Family style	x.xx
Buffet	x.xx
Other	x.xx
Previous incident of vomit/diarrhea (MI55)
Yes	x.xx (ref)	.xxx
No	x.xx
Employee characteristics
Employee experience (MI1,4, WS1,4)
> xx years	x.xx (ref)	.xxx
< xx years	x.xx
Employee certified (MI2-3, WS2-3)
Yes	x.xx (ref)	.xxx
No	x.xx
Position in restaurant (MI5, WS5)
Manager	x.xx (ref)	.xxx
Cook	x.xx
Food preparation	x.xx
Other	x.xx

OR=Odds Ratio, P=probability level, MI=Manager Interview, WS=Worker Survey, Obs= Observation

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The following people were primarily responsible for the design, including the statistical aspects, of the data collection and will be primarily responsible for data analysis. Laura Brown is the primary contact for statistical aspects and data collection.

Laura Green Brown, Ph.D.

Behavioral Scientist

Centers for Disease Control and Prevention, National Center for Environmental Health

Lrg0@cdc.gov

770-488-4332

Adam Kramer, Sc.D., M.P.H, R.S.

Environmental Health Officer

Centers for Disease Control and Prevention, National Center for Environmental Health

Ank5@cdc.gov

404-498-1228

Rick Hoover, Ph.D.

Behavioral Scientist

Centers for Disease Control and Prevention, National Center for Environmental Health

Xmo2@cdc.gov

706-765-8857

Personnel in the eight EHS-Net sites will be responsible for data collection (See table below). In some cases, environmental health specialists from non-EHS-Net sites assist with data collection; these personnel are not included in this table.

Site	Number of Personnel
California Department of Health	1 full-time
Minnesota Department of Health	1 full-time 1 part-time
New York Department of Health	1 full-time 1 part-time
New York City Department of Health and Mental Hygiene	1 full-time
Rhode Island Department of Health	1 full-time
Tennessee Department of Health	1 full-time 1 part-time
Southern Nevada Health District	1 full-time
Harris County, Texas	1 full-time

References for Part B

15