Supporting Statement Part B- EHS-Net 9 4 2015 (clean)

Supporting Statement – Section B

Environmental Health Specialists Network (EHS-Net) Program Generic Package

Reinstatement (with change)

OMB No. 0920-0792

Expiration Date: February 28, 2015

Submitted: January 25, 2021

Laura Brown, Ph.D.

EHS-Net

Centers for Disease Control and Prevention

National Center for Environmental Health

Emergency and Environmental Health Services

Environmental Health Services Branch

4770 Buford Highway, NE F – 58

Atlanta, GA 30341-3724

lrg0@cdc.gov

770-488-4332

Table of Contents

B. Collections of Information Employing Statistical Methods…………………………..	3
1. Respondent Universe and Sampling Methods…………………………………………	3
2. Procedures for the Collection of Information………………………………………….	4
3. Methods to Maximize Response Rates and Deal with Nonresponse………………….	6
4. Tests of Procedures or Methods to be Undertaken…………………………………….	6
5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data………………………………………………………………………...	10

3. List of Attachments

Attachment 4- Example of EHS-Net Manager Interview

Attachment 5- Example of EHS-Net Observation

Attachment 7- Example of EHS-Net Data Collection Manager Informed Consent

Attachment 8- Example of EHS-Net Data Collection Worker Recruiting/Informed Consent Script

Attachment 9- Example of EHS-Net Worker Interview

B. Collections of Information Employing Statistical Methods

Data will be collected in a random sample of the retail food establishments in the jurisdictions of the state and local health departments that will be funded through the EHS-Net cooperative agreement. While the number of jurisdictional areas included in EHS-Net is relatively small (8), they are demographically diverse and provide good geographical coverage of the U.S. (north, south, east, west). When appropriate statistical methods are used, the results of collections covered by this generic OMB package can be used to generalize to the population of retail food establishments in the given EHS-Net sites. Furthermore, the geographic and demographic variability across the sites may make it possible for CDC to use data collected from these studies to draw conclusions about relationships that are likely relevant to establishments in other parts of the U.S.

B.1. Respondent Universe and Sampling Methods

Respondent Universe

The respondent universe is all retail food establishments in the EHS-Net sites. Establishment lists will be obtained from the databases maintained by the EHS-Net sites. CDC will use these lists to generate the sampling frame used to draw the establishment sample for each data collection.

Sampling Methods

Sample Size and Power Calculation. A sample size calculation was conducted to estimate the total number of retail food establishments necessary to provide sufficient power to detect group differences in outcomes of interest. The design parameters were set to detect an odds ratio of at least 2.0 or greater (a medium –small effect size) with a .05 Type I error rate, power of .90, and based on the assumption of equivalent group sizes. Calculations established that a total of 372 retail food establishments would need to participate to achieve the desired parameters. With eight EHS-Net sites participating in data collection, each site will need to enroll 47 establishments to meet this requirement.

Sampling Design. Each study will utilize a stratified random sampling design where each EHS-Net site serves as its own mutually exclusive stratum. There are two primary reasons for stratifying by EHS-Net site. The first is that food safety regulations vary by jurisdiction. For example, Tennessee state food safety regulations differ from New York state food safety regulations. These regulations can and do greatly influence retail food establishment food safety practices and policies. Thus, we felt that EHS-Net site/jurisdiction was a critically important factor for stratification. The second reason for stratifying by EHS-Net site is that EHS-Net sites participate in EHS-Net through a cooperative agreement. The nature of this agreement is such that one site cannot be expected to do a disproportionate amount of work in comparison to other sites (because each site receives relatively equal funding amounts). If we did not stratify by EHS-Net site, it is likely that some sites would have to collect data for each study in far more retail food establishments than other sites would.

Establishments will be randomly selected, with equal probability, within their respective EHS-Net site, independent of other sites. This process will give each establishment on the list the same probability of being selected for study participation. We will sample without replacement, meaning that establishments will not have the chance to be selected for the study sample more than once. The use of sampling without replacement is a common and justifiable practice when the target population is considerably larger than the number of samples needed to be drawn from that population, as is the case with these studies.

Response Rate. The average response rate for recent EHS-Net studies that used methods similar to those proposed in this package is approximately 60%. We expect a similar response rate for future studies. Although this response rate is lower than the 80% response rate mentioned in the OMB guidelines, it is a rate that allows us to feel comfortable with our planned analyses. (Frey & Oishi, 1995).

B.2. Procedures for the Collection of Information

Sampling and Recruitment

As indicated earlier, each EHS-Net site will provide CDC with a list of all establishments in the EHS-Net sites. This list will serve as the sampling frame for the site for each data collection. CDC will use a random number generator in SAS 9.2 to produce a random sample of establishments from this list for each site for each study.

For each study, personnel in each site will contact establishments by telephone to recruit them for participation in the study. If the manager is willing to participate, the EHS-Net specialist will arrange a mutually convenient time to conduct the data collection. A log of each incident of contacts with the establishments will be kept in order to document participation rates and reasons for refusal and/or ineligibility. This log will be kept in such a way as to maintain retail food establishment and retail food establishment staff anonymity.

Data Collection

Data will be collected in the retail food establishments by CDC-funded environmental health specialists employed by the eight EHS-Net sites (the state and local health departments participating in the EHS-Net cooperative agreement). These environmental health specialists are skilled and experienced in food safety and in retail food environments.. For the manager interview/pen-and-paper assessment portion of the studies, the EHS-Net specialists will obtain verbal informed consent and then conduct a face-to-face or telephone semi-structured interview, and/or a written assessment with the establishment manager (Attachments 4 and 7). The interviews/assessments will include questions on establishment and manager characteristics and food handling and food safety practices. This process will take about thirty minutes to complete. The process will be similar for the worker interview/assessment portion of the studies (Attachments 8 and 9). However, the EHS-Net worker data collection process will only take about 10 minutes to complete. Additionally, to increase cooperation, we will allow establishment management to provide feedback on which workers can be interviewed (Attachment 7).

For the observation portion of the studies, the EHS-Net specialists will observe the kitchen environment and observe food handling practices (Attachment 5). This observation will take about 30 minutes to complete. Thus, data collection will take about an hour and 10 minutes per establishment (30-minute manager interview, 10-minute worker interview, and 30-minute observation). However, workers will only be engaged with the data collectors during the interview/assessment because the kitchen observation does not actively involve them.

Quality Control Procedures

The data collectors will be experienced and knowledgeable in environmental health and food safety and will have received training from CDC on data collection for each study. Data entry will be double-checked by the EHS-Net administrator in each EHS-Net site.

Potential Biases

Managers’ concerns about the food safety of their establishments may result in a lower rate of study participation among establishments with worse food safety practices compared to establishments with better food safety practices. In the past, we have conducted studies using methods similar to those proposed in this package, and these studies have found a wide range of food safety practices, including poor ones (Delea et al, 2010; Green et al., 2006; Kirkland et al., 2009; Lee et al., 2004; Marcus et al, 2010).

The observation data collected for these studies may be influenced by reactivity on the part of those observed. In other words, those observed may not respond naturally when they know they are being observed. However, observation data on behavior is considered to be more accurate than self-reported data, particularly when measures are taken to limit the observers’ influence on the observed (Leary, 2004). In our studies, those measures will include the following: 1) observers will attempt to remain relatively unobtrusive during the observation, and 2) when possible, the precise details on which aspects of behavior are being recorded will not be provided to those being observed.

The interview data collected for these studies may be influenced by the social desirability bias- the tendency for people to report greater levels of socially desirable behavior (such as safe food preparation practices) than they actually engage in, or to report their best behavior rather than their typical or worst behavior. Although it is difficult to eliminate this bias altogether, it can be limited by ensuring respondents that the information they report will be anonymous, which we will do (Leary, 2004).

The fact that management will help select the workers to be interviewed may also introduce bias, as management may select workers they believe are knowledgeable about food safety. However, anecdotal evidence suggests this selection technique is necessary to increase participation.

In the past, we only interviewed managers/workers that spoke English well enough to be interviewed in English. The use of this criterion may have introduced bias, as non-English speakers may have different food safety knowledge and practices than English speakers, but the resources were not available to include non-English speaking workers in the studies. However, the 2016-2020 cooperative agreement will encourage sites to collect data from non-English speaking managers/workers, and will include additional funding for this purpose. Specifically, sites may employ translators to translate data collection instruments into the most common non-English languages spoken in their sites and/or to conduct the data collection interviews. Information collections submitted under this generic OMB clearance will state whether multi-lingual resources/materials will be used, and if none are used, will discuss why they are not used, and the potential implications of their lack of use.

Any presentation of data from EHS-Net studies will acknowledge potential biases and include a discussion of how they may impact data interpretation.

B.3. Methods to Maximize Response Rates and Deal with Nonresponse

For each study, we will engage in several activities designed to maximize response rates. First, all recruiters will receive training on the recruiting process. Second, multiple attempts will be made to contact potential respondents. Specifically, recruiters will make 10 attempts over 5 days to get a participation response from establishments they have not been able to contact, and 5 attempts over 5 days to get a participation response from establishments that have not provided a response (e.g., ‘call back later’). Third, recruiting scripts will emphasize two issues that have been shown to increase response rates—the anonymous nature of the data collection and the importance of the respondents’ participation in the study.

For the past four EHS-Net studies, we were able to compare restaurants that participated in our studies to restaurants that refused to participate on an important characteristic- ownership. These comparisons indicate that for two of the four studies, independently-owned restaurants were less likely than chain restaurants to participate. Independently-owned restaurants may be under-represented in these studies. These data have prompted three actions. First, when appropriate, we will acknowledge this difference and discuss its potential ramifications in any presentation of data from studies in which this difference was found. Second, to reduce or eliminate this difference in future studies; we will refine our recruiting techniques through pilot testing with less than ten managers of independent restaurants to be more effective with independently-owned restaurants. Third, we will continue to collect and monitor data on ownership from participating and refusing restaurants for each study.

B.4. Test of Procedures or Methods to be Undertaken

Data collection materials will be reviewed and evaluated by EHS-Net specialists with retail food establishment food safety knowledge and experience. They will also be reviewed by CDC EHS-Net personnel with extensive experience in developing and conducting EHS-Net data collections. Additionally, all data collection materials will be evaluated in pretests with 9 establishments. The pretests will be used to improve the data collection materials. For example, if the pretest indicates that interview respondents did not understand a question, the question would be revised for clarity.

Data Analysis Plan. A primary purpose of these data collections will be to describe retail food establishment food handling and food safety policy and practices, and establishment and manager/worker characteristics. To address this purpose, we will conduct descriptive analyses (frequencies, means, etc.). Table B.4.1 contain examples of variables to be included in these analyses. Table B.4.2 contains examples of variables used to describe the establishments in which we will collect data and the managers and workers from which we will collect data. Table B.4.3 is a table shell that illustrates how we might analyze and present the descriptive data collected from these data collections. All these examples are based on the example data collection interview provided in Attachment 3.

Table B.4.1. Example variables measuring food handling/safety policies and practices

Item Content	Manager interview #
Food handling/safety policies
Does this establishment require PICs or kitchen managers to be food safety certified?	8
Does this establishment have a policy or procedure concerning hand washing?	14
Does this restaurant have a policy that specifies the final cook temperatures for raw animal products?	16a
Does this restaurant have a policy that specifies the temperature at which potentially hazardous foods should be cold held?	18
Does this restaurant have a policy that specifies the temperature at which potentially hazardous foods should be hot held?	17a
Food handling/safety practices
How do food workers determine the final cook temperature of raw animal foods such as ground beef, chicken, or eggs?	16d
How do food workers determine the temperature of hot or cold held food?	19

Table B.4.2. Examples variables measuring characteristics

Item Content	Manager interview #
Establishment characteristics	1
Is this an independent restaurant or a chain restaurant?	2
Approximately how many meals are served here on your busiest day?
Which one of the options below best describes the menu for this establishment- American, Asian, Mexican, Italian, or Other?	3
How many food workers, excluding managers, work in this restaurant?	11
Manager/worker characteristics
Approximately how long have you been employed as the Person-In-Charge- the PIC- or kitchen manager here?	4
Have you, as the PIC or kitchen manager, ever been food safety certified?	8
What is the highest level of formal education you have completed?	29
What is your primary language?	31

Table B.4.3- Table Shell: Example descriptive data on food handling/safety policies and practices

	n	%
Food handling/safety policies
Does this establishment require PICs or kitchen managers to be food safety certified?
Yes	xx	xx
No	xx	xx
Does this establishment have a policy or procedure concerning hand washing?
Yes	xx	xx
No	xx	xx
Does this restaurant have a policy that specifies the final cook temperatures for raw animal products?
Yes	xx	xx
No	xx	xx
Does this restaurant have a policy that specifies the temperature at which potentially hazardous foods should be cold held?
Yes	xx	xx
No	xx	xx
Does this restaurant have a policy that specifies the temperature at which potentially hazardous foods should be hot held?
Yes	xx	xx
No	xx	xx
Food handling/safety practices
Method for determining the final cook temperature of raw animal foods such as ground beef, chicken, or eggs
Safe	xx	xx
Unsafe	xx	xx
Method for determining the temperature of hot or cold held food
Safe	xx	xx
Unsafe	xx	xx

A secondary purpose of these data collections is to determine how retail food establishment and manager/worker characteristics are related to food handling/safety practices. To address this purpose, we will conduct tests for association and logistic regression models. Analysis will involve bivariate tests for association between each individual explanatory (independent) variable and the outcome (or dependent) variables of interest. Odds ratios will be calculated to assess the strength and direction of the bivariate relationships. For those bivariate associations found to be statistically significant at p<.10, the explanatory variables will be used as candidate “predictors” to examine their multivariate relationships with the outcome variables. Multivariable logistic regression will be used to model for the effects that these explanatory variables have in explaining the variations observed in the outcome variables.

Explanatory variables in these analyses include those measuring key establishment and manager/worker characteristics. Outcome variables include those measuring key food handling/safety practices. Table B.4.4 lists example key explanatory variables and key practice outcome variables. Table B.4.5 is a table shell that illustrates how we might analyze and present the data examining the relationships between establishment and worker characteristics and key practices. All these examples are based on the example data collection interview provided in Attachment 4.

Table B.4.4- Example key explanatory and practice outcome variables

Explanatory variables	Outcome variables
Does this establishment require PICs or kitchen managers to be food safety certified?	Method for determining the final cook temperature of raw animal foods such as ground beef, chicken, or eggs
Is this an independent restaurant or a chain restaurant?	Method for determining the temperature of hot or cold held food
Which one of the options below best describes the menu for this establishment- American, Asian, Mexican, Italian, or Other?
Approximately how long have you been employed as the Person-In-Charge- the PIC- or kitchen manager here?

Table B.4.5- Table Shell: Example key explanatory variables associated with the practice outcome variable of whether the method for determining the final cook temperature of raw animal foods is safe

Explanatory variables	Method is safe
	OR (95% CI)	P
Establishment requires certification
No	x.xx (ref)	.xxx
Yes	x.xx
Ownership
Independent	x.xx (ref)	.xxx
Chain	x.xx
Menu
American	x.xx (ref)	.xxx
Non-American	x.xx
Manager length of employment
> xx years	x.xx (ref)	.xxx
> xx years	x.xx

OR=Odds Ratio, P=probability level

B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The following CDC personnel will be primarily responsible for the design, including the statistical aspects, of the data collections and will be primarily responsible for data analysis.

Laura Brown, Ph.D.

Behavioral Scientist

Centers for Disease Control and Prevention, National Center for Environmental Health

Lrg0@cdc.gov

Rick Hoover, Ph.D.

Statistician

Contractor, Centers for Disease Control and Prevention, National Center for Environmental Health

Xmo2@cdc.gov

Personnel in an estimated 8 EHS-Net sites will be responsible for data collection. These sites are not yet known.

10