EMS_ICR_Section_B revised 090910_120430

Occupational injuries and illnesses among emergency medical services (EMS) workers: A NEISS-Work telephone interview survey

Request for Office of Management and Budget Review and

Approval for Revision of Federally Sponsored Data Collection

OMB No. 0920-0834

Section B

Project officer: Audrey Reichard, Epidemiologist

National Institute for Occupational Safety and Health

Division of Safety Research

1095 Willowdale Road, MS H1808

Morgantown, WV, 26505

Phone: 304-285-6019

Fax: 304-285-5774

E-mail: AReichard@cdc.gov

June 1, 2012

Table of Contents

B. Collection of Information Employing Statistical Methods

B1. Respondent Universe and Sampling Methods 2

B2. Procedures for the Collection of Information 4

B3. Methods to Maximize Response Rates and Deal with Nonresponse 7

B4. Tests of Procedures or Methods to be Undertaken 8

B5. Individuals Consulted on Statistical Aspects and Individuals

Collecting and/or Analyzing Data 9

References

B. Collections of Information Employing Statistical Methods

B.1 Respondent Universe and Sampling Methods

This is a request for approval of a revision to an ongoing project that captures data on EMS workers treated for an occupational injury or illness in a national probability based sample of US hospital emergency departments. EMS workers who are invited to participate in interviews are identified through NEISS-Work, an ongoing, supplemental effort of the National Electronic Injury Surveillance System (NEISS), used to capture and report product-related injuries. The ongoing telephone interviews enhance the information available through NEISS-Work.

Background on NEISS and NEISS-Work

In 1972, as authorized by statute (the Consumer Product Safety Act Sec. 5. [15 U.S.C. § 2054]), CPSC initiated the collection of consumer product-related injury and illness information through a surveillance system that uses a national probability-based sample of hospital emergency departments—the National Electronic Injury Surveillance System (NEISS). The NEISS data are abstractions of selected information from emergency department medical records as collected by contract hospitals and paid records abstractors. CPSC uses information attained through NEISS to conduct in depth follow-up investigations including by telephone. CPSC collects information through the follow-up investigations with the approval of OMB (OMB Control No: 3041-0029) based on information collection extension requests every three years.

NIOSH conducts research using NEISS data as authorized by the Occupational Safety and Health Act, Section 20, "Research and Related Activities" and Section 22(d), "Authority of Director, National Institute for Occupational Safety and Health" (29 U.S.C. 669, 671 (d)). The work-related data attained by NIOSH from CPSC through what is referred to as NEISS-Work do not contain direct personal identifiers such as name, social security number, or contact information. Similarly this information is not provided to NIOSH in the ongoing study, but is obtained from the participating hospitals and retained by CPSC only for the purposes of conducting follow-up interviews.

In the 1990’s NIOSH conducted several follow-up telephone interview studies in collaboration with CPSC under their approval to collect information. OMB was regularly informed of such. Beginning in approximately 2002, CPSC requested that all federal agencies using NEISS for follow-up investigations seek their own OMB approval for the specific follow-up study (as is being done herein).

NEISS-Work

NEISS-Work data are collected from a national stratified probability sample of approximately 5,400 rural and urban hospitals in the U.S. and its territories that have a minimum of six beds and that operate a 24-hour emergency department. General, specialty care, and military hospitals are included in the sample population; however, prison, psychiatric, rehabilitation and long-term care facilities, and Veterans Administration hospitals are excluded. Selection of the current hospital sample was based on a 1995 census of U.S. hospitals. Hospitals were stratified on the basis of both geographical location and hospital size (as determined by the number of annual ED visits). Data collection using the current hospital sample began in 1997 at 101 CPSC NEISS hospitals for consumer product-related cases and at 67 hospitals (2/3 of the CPSC sample) for work-related cases. The NEISS-Work stratified sample consists of 32 small, 9 medium, 6 large, and 15 very large sized hospitals, and 5 children’s hospitals.

Since selection of the NEISS-Work hospital sample in 1997, four small hospitals of the original 67 hospitals have permanently closed and thus were not replaced in the sample. The remaining 63 hospitals in the sample are distributed among the five sample strata with 28 small, 9 medium, 6 large, 15 very large sized hospitals, and 5 children’s hospitals. Currently, one of the nine medium size hospitals is not reporting and a replacement hospital is being sought. Case weights are adjusted to account for this nonresponse until the hospital is replaced.

Telephone interview survey

The EMS worker cases participating in the telephone interviews are identified from the NEISS-Work data. CPSC attempts to obtain contact information for every identified EMS worker case and offer them the opportunity to complete the telephone interview. However, it is expected that the number of EMS workers completing the telephone interview will be much less than the number identified in NEISS-Work due to barriers to obtaining contact information and difficulty making contact with the workers by phone. The current overall response rate for CPSC telephone interviews, including potential respondents for whom contact information is not available, is approximately 40-45%. We began data collection in July 2010 and have interviewed more than 200 cases. As of January 2012, the overall interview response rate was 41%. However, of those EMS workers who could be contacted, the response rate was 80%.Upon completion of four years of data collection we will use the routine NEISS-Work data to perform an analysis of all identified EMS workers to determine if there are any differences between the telephone interview respondent and non-respondent groups.

NIOSH has three criteria for determining reportability of NEISS-Work data results that are intended to ensure reasonable and reliable data quality and appropriate interpretation and use of these data¹:

1. Number of cases treated within the hospital sample must exceed a specified value;

2. The extrapolated national estimates must exceed a specified value; and

3. The coefficient of variation must be less than or equal to 33%.

In a prior NEISS-Work data analysis of EMS workers, we determined that approximately 25 unweighted EMS worker cases per year were required to meet all of the NEISS-Work criteria for reporting a national estimate. Using these same data analyses, we produced a conservative estimate of 350 EMS workers being treated annually in the NEISS-Work hospital sample. Based on this estimate we have projected the aggregated response counts for responses of 1% to 20% for individual question answers at overall interview response rates of 40%-60% over a four year data collection period (Table B.1). For example, in year one if there was an overall interview completion rate of 40%, we would obtain 140 interviews. If 20% of respondents indicated “Yes” for a single question, it is highly likely that the national estimate would be reportable based on the 28 positive responses in that year. However, if only 10% of respondents answered “Yes” this response would not be reportable without aggregation of multiple years of data.

Thus, Table B.1 illustrates that if 20% of respondents indicate a particular answer, we can produce national estimates with overall interview completion rates as low as 40% (i.e., there would likely be at least 25 positive responses in a year). If we decided to report national estimates for individual question categories that represented only 10% of the responses, aggregation of at least two years of data would be required; for 5% responses aggregation over four years of interviews would be required. Better overall interview completion rates of 50% and 60% improve the reliability of the study but do not materially impact the reportability of individual response categories for specific questions. However, interview completion rates of 50% or more combined with at least three years of data collection would potentially permit reporting of individual question response categories representing 5% of injured EMS workers.

Table B.1. Telephone interview data reportability based on response rate, number of years of data collection, and desired level of reporting detail.*

Percent of total responses within a response option	Number of cases collected**
	40% response rate***				50% response rate				60% response rate
	(n = 140/yr)				(n = 175/yr)				(n = 210/yr)
	Year 1	Year 2	Year 3	Year 4	Year 1	Year 2	Year 3	Year 4	Year 1	Year 2	Year 3	Year 4
20%	28	56	84	112	35	70	105	140	42	84	126	168
10%	14	28	42	56	18	35	53	70	21	42	63	84
5%	7	14	21	28	9	18	26	35	11	21	32	42
1%	1	3	4	6	2	4	5	7	2	4	6	8

* Gray shading indicates non-reportable data, defined as having a case count less than 25.

**These numbers were calculated based on collection an annual average of 350 EMS cases per year. The numbers presented should not be interpreted to represent the actual number of cases that would be collected in the future, but rather simply as projected estimates given that the number of non-fatal injuries to EMS workers remains stable. Thus, while these numbers provide reasonable estimates based on the current data, they are only estimates and are subject to change.

***The response rate is calculated as the number of completed interviews divided by the number of treated EMS workers within the NEISS-Work hospital sample on an annual basis.

B.2 Procedures for the Collection of Information

B.2.1 Stratification and Sample Selection

The population for NEISS-Work data is based on two-thirds of the CPSC NEISS sample as described below. For the telephone interview portion of the study, all EMS workers captured in the NEISS-Work data are considered potential participants. Every EMS worker identified in NEISS-Work is offered the opportunity to participate in a telephone interview given that the correct contact information is available for that worker and initial contact is made with the worker.

The NEISS sample design is based on a stratified simple random sample of hospitals with an emergency department (ED) in the U.S. and its territories. A hospital is defined as a general or specialty care facility with a minimum of six beds and a 24-hour ED. The requirement for a hospital to have at least six beds conforms to the American Hospital Association (AHA) registration requirements (AHA, 2004).

T he sample is stratified by hospital size based on the number of emergency department visits annually. Two organizations have historically maintained data on U.S. hospitals and ED usage. Data from the American Hospital Association and the SMG Marketing Group (now doing business as Verispan) have been used at various times to create the NEISS sample frame (Marker and Lo, 1996). Since 1988, the SMG hospital lists and ED usage data have been used for all sample redesigns and annual hospital adjustments. The SMG data were used to construct the current NEISS hospital sample with four size-related strata and a children’s hospital stratum.

Figure B.2.1. Map of NEISS Hospitals in the CPSC Sample, 2003 (CPSC, 2006).

In addition to stratification by hospital size, the NEISS sample is stratified geographically. Within each size stratum, a systematic hospital sample was drawn from a geographically-ordered SMG hospital list. The U.S. distribution of CPSC hospitals in the NEISS sample is shown in Figure B.2.1.

Since the initiation of the NEISS program in 1972, the CPSC hospital sample has been redesigned three times with implementation in 1978, 1990, and 1997. In addition to redesign changes, the number of hospitals in the sample has changed over time as CPSC has tried to enhance the data collection or reduce the system cost depending upon the vagaries of budgetary constraints. NIOSH has undergone similar expansions and contractions in its NEISS-Work data collection efforts. Currently, NIOSH collects data on all work-related injuries and illnesses treated in the ED at two thirds of the CPSC hospitals. The NEISS-Work data collection has been uniform and systematic since January 1, 1998, the last effective date of a break in series.

For the purposes of NEISS-Work methodology descriptions, the number of hospitals in the sample is defined as the number of hospitals in the sample at the time the current sample was initially selected. Currently, NEISS-Work uses the 1997 redesign hospital sample of 67 hospitals. However, at various points in time since 1997 the number of possible reporting hospitals has been reduced owing to hospital closures or hospitals not reporting or dropping out of the NEISS program. When a hospital closes the number of in-scope hospitals decreases because closures are not replaced in the sample. If a hospital simply withdraws from participating in NEISS, a new hospital is recruited and the original hospital is replaced, although there may be an extended lapse in reporting. The withdrawal of a hospital from NEISS or hospital non-response for a period of time does not result in a reduction of the number of in-scope hospitals (although it does influence the case weights for the period). The number of in-scope hospitals and reporting hospitals may change in any month of the year. Nevertheless, the number of hospitals in the sample remains at the number initially chosen in the current redesign.

The 1997 CPSC sample redesign is based on a 1995 SMG sample frame. The full sample had 102 hospitals (1.9% of qualifying hospital EDs), but by the time the sample was implemented one hospital had closed resulting in 101 in-scope hospitals. CPSC used a Keyfitz procedure for resampling a stratified simple random sample that maximized the probability of retaining hospitals from the former sample (i.e., participating hospitals in 1996). As a result, 75 hospitals were retained and 26 new hospitals were recruited. As a part of this redesign, the children’s hospital stratum became a probability based sample and no longer a simple convenience sample.

Although not used for the NEISS sample frame, the American Hospital Association annual surveys illustrate the decrease in emergency departments in community hospitals (nonfederal, short-term general and other special hospitals) while ED visits have increased in number and rate (Figure B.2.2) (AHA, 2006).

Whereas the NEISS sample includes Federal and non-federal hospitals with 5,388 EDs in 1995, the AHA sample of community hospitals with 4,923 ED’s in 1995 is generally representative of U.S. hospital trends as a whole.

Source: The Lewin Group analysis of American Hospital Association Annual Survey data, 1991 – 2004, for community hospitals and US Census Bureau: State and County QuickFacts, 2004 population estimate data derived from Population Estimates, 2000 Census of Population and Housing

Figure B.2.2. (a) Number of ED visits and number of EDs in community hospitals; (b) rate of ED visits per 1,000 persons; 1991-2004 (AHA, 2006).

In October 1997, NIOSH implemented the 1997 CPSC sample design. However, budgetary constraints prohibited using the full 102 hospital sample. To continue with a sample of approximately the same size (i.e., ~65 hospitals) NIOSH obtained a new sample of 67 hospitals that was approximately two-thirds of the CPSC sample for each stratum: 32 small, 9 medium, 6 large, 15 very large, and 5 children’s hospitals. For the new NIOSH sample, 52 hospitals were retained from the prior sample and 15 new hospitals were added. Although adding a large number of new hospitals to the sample created some difficulties, work-related case reporting appeared stabilized by January 1998.

Each NEISS-Work case is assigned a statistical weight based on the inverse probability of selection. National estimates (i.e., the number of injuries and illnesses) are obtained by summing weights for all cases or particular cases of interest. The basic case weight is the inverse probability of selection for the hospitals in each stratum. The inverse probability of selection is the number of hospitals in the stratum universe divided by the number of hospitals in the NEISS sample for the stratum.

CPSC makes two types of routine weight adjustments to the basic case weights. First, weights are adjusted for non-participation if a hospital does not report fully during any given month or to account for hospital mergers, hospital closings or withdrawal from the NEISS-Work sample. Secondly, CPSC makes an annual ratio adjustment to the case weights by comparing the most recent U.S. hospital sample frame (i.e., for the prior year) with the 1995 sampling frame (used in 1997 for the latest NEISS sample). This adjustment is designed to account for changes in ED usage and the number of hospitals with EDs over time to provide the best opportunity for trend analysis and to minimize the expense of frequent sample redesigns. Thus, final case weights for each hospital stratum by month and year are calculated from the basic weight with adjustments for non-reporting and changes in the sampling frame over time.

Weights assigned to cases within the telephone interview data are based on the normally adjusted NEISS-Work weights with additional adjustments for non-response to the telephone interview using a procedure known as raking. The weights assigned to cases with completed interviews are adjusted to be representative of all cases within the stratum, including adjustments for significant biases that are detected among interview respondents (e.g., sex or age) versus all cases treated during the year.

B.2.2 Collection of Telephone Interview Data

Telephone interviewers are contracted through CPSC to complete the follow-back interviews. These interviewers are experienced interviewers and received additional training specific to the EMS worker questionnaire to be used for this study.

Prior to being contacted by telephone, potential participants receive a letter describing the study and their protections as a participant should they choose to participate (Appendix Attachment F). This letter also provides them with the opportunity to opt out of participating in the study by calling a toll-free number. While the time for the telephone interview is not initially scheduled with the participant, participants do have the option at the time of contact to state that it is not a good time and schedule a better time to complete the interview. Also, if the potential participant initially declines to participate, text has been included in the telephone interview script to encourage them to reconsider.

B.2.3 Data Quality Control

Quality control of the data does not involve any additional contact with participants. Rather, data within the telephone interview dataset is reviewed for logical consistency and continuity. Data from NEISS-Work and the telephone interview dataset will also be broadly compared to check for consistency and accuracy. Finally, an assessment of the non-participants versus the participants in the telephone interview portion of the study will be made to determine potential non-response bias.

B.3 Methods to Maximize Response Rates and Deal with Nonresponse

We acknowledge that our response rate to-date of 41% is low. However, it must be noted that this rate of overall response includes EMS cases identified in NEISS-Work for which hospitals will not release contact information or correct contact information is unavailable. These insurmountable barriers drive the response rate down prior to us contacting potential participants. Of all EMS workers who could be contacted, the response rate to-date is 80%.

Given the low response rate, we are taking several steps to help access potential participants and facilitate their willingness to participate. These steps include:

1. A letter describing the study is sent to potential participants in advance of the initial phone call. This letter alerts and prepares potential participants for the phone call requesting their participation.

2. The Office of Emergency Medical Services (OEMS) within NHTSA has informed their stakeholders of this study via documents and meetings. This information dissemination effort by the OEMS may help to garner support among the EMS community, influencing potential respondents to participate. Both NIOSH and NHTSA continue to publicize this collaborative project.

3. Telephone interviewers are required to make at least ten attempts to reach a potential respondent. The contact attempts are made at varying, but reasonable, hours of the day and on varying days of the week. When no personal contact is made after a number of attempts, the interview is set aside and contact attempts are made at a later date as time permits to maximize the response rate while minimizing recall bias issues. Interviewers are trained to be considerate of respondents and their families, leaving a minimal number of messages or speaking with the respondent or another individual of the residence to arrange a convenient interview time. Messages include a toll-free response number so that the respondent may call at their convenience. When no personal contact is made, message system is available, or indicator of an incorrect number, the interviewer typically spreads their call attempts over a longer time period and commonly makes more than 10 contact attempts over the initial contact attempt period and the subsequent missed interview follow-ups.

4. This project uses trained telephone interviewers who are experienced at conducting interviews. This facilitates ease of survey participation for the respondent, increasing the likelihood that they will complete the survey in its entirety.

5. If the participant refuses the initial offer to participate in the study in a non-firm way, the interviewer emphasizes the importance of their participation and inquires as to whether they would be willing to participate at another time of their choosing. The training and experience of the telephone interviewers are a key factor to understanding the reactions of potential participants and appropriately encouraging their participation in cases of refusal.

6. The questionnaire has been designed to be as easy and non-burdensome as possible. This includes ordering the questions in a logical sequence and asking only those questions that are needed for analysis purposes.

Despite the low response rate, one of the benefits of this study is that we capture basic demographic and injury information on all potential participants. Ultimately, we will compare the information we have on respondents and non-respondents using the NEISS-Work dataset to provide insight on any potential response bias. At a minimum, the case weights are adjusted for non-response within each stratum so that the completed interviews within each stratum truly represent that stratum. If other factors are determined to influence answers, raking is performed so that the analysis weights for each variable of interest are equal to the corresponding national estimate.

B.4 Test of Procedures or Methods to be Undertaken

The questionnaire used in this study was designed based on information gathered from published literature and input from EMS professionals. It was pilot tested on nine EMS workers identified in the NEISS-Work database. These pilot tests were performed by CPSC contracted telephone interviewers. In addition to these pilot tests, the questionnaire was reviewed by researchers both internal and external to NIOSH with expertise in EMS work and/or survey administration. Revisions were made to the questionnaire as a result of the pilot test results and the reviewer comments.

In addition to soliciting peer reviews from other researchers, NIOSH also held a face-to-face meeting with our NHTSA partners and reviewed the questionnaire in its entirety. This review was focused on reducing the length of the questionnaire to approximately 20 minutes. Consequently, we scrutinized each question individually to confirm that it would provide useful information for analyses in terms of content and in accordance with likely response numbers that would determine reportability.

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Individuals who designed the data collection

Tom Schroeder, MS

Statistician, Director

Division of Hazard and Injury Data Systems

U.S. Consumer Product Safety Commission

Phone: 301-504-0539 x1179

E-mail: TSchroeder@cpsc.gov

Larry Jackson, PhD

Chief, Injury Surveillance Team

Division of Safety Research, NIOSH

Phone: 304-285-5980

E-mail: LLJackson@cdc.gov

Individuals who will collect the data

CPSC staff and contracted interviewers under the direction of:

Tom Schroeder, MS

Statistician, Director

Division of Hazard and Injury Data Systems

U.S. Consumer Product Safety Commission

Phone: 301-504-0539 x1179

E-mail: TSchroeder@cpsc.gov

Individuals who will analyze the data

Audrey Reichard, MPH, OTR

Epidemiologist, Injury Surveillance Team

Division of Safety Research, NIOSH

Phone: 304-285-6019

E-mail: AReichard@cdc.gov

Larry Jackson, PhD

Chief, Injury Surveillance Team

Division of Safety Research, NIOSH

Phone: 304-285-5980

E-mail: LLJackson@cdc.gov

Tom Schroeder, MS

Statistician, Director

Division of Hazard and Injury Data Systems

U.S. Consumer Product Safety Commission

Phone: 301-504-0539 x1179

E-mail: TSchroeder@cpsc.gov

REFERENCES

AHA, 2004. Hospital Statistics, 2004 ed: Health Forum, Chicago, IL; 212 pgs [cf. 2006 guide, accessed November 17, 2006 at: http://www.aha.org/aha/content/2006/pdf/AHAGuide2007

Definitions.pdf].

AHA, 2006. TrendWatch Chartbook 2006, Trends Affecting Hospitals and Health Systems:

AHA, Washington, DC, April 2006; accessed November 17, 2006 at: http://www.ahapolicyforum.org/ahapolicyforum/trendwatch/content/CB6Chapter3.PPT.

Gershon, RRM, Vlahov, D, Kelen, G, Conrad, B, Murphy, L. 1995. Review of accidents/injuries among emergency medical services workers in Baltimore, Maryland. Prehospital Disaster Med 10:14-18.

Hogya, PT, Ellis, L. 1990. Evaluation of the injury profile of personnel in a busy urban EMS system. Am J Emerg Med 8(4):308-311.

Maguire, BJ, Hunting, KL, Guidotti, TL, Smith, GS. 2005. Occupational injuries among emergency medical services personnel. Prehosp Emerg Care 9: 405-411.

Marker D, Lo A, 1996. Update of the NEISS Sampling Frame and Sample: Westat Inc., Final Report, October 11, 1996; Rockville MD.

Studnek, JR, Ferketich, A, Crawford, JM. 2007. On the illness and injury resulting in lost work time among a national cohort of emergency medical services professionals. Am J Ind Med 50: 921-931.

USDOL. U.S. Department of Labor, Bureau of Labor Statistics. (no date) Industry injury and illness data. Available from: http://www.bls.gov/iif/oshsum.htm#98Summary%20News%20Release. Accessed Oct 5, 2007

1 Because of confidentiality restrictions, NIOSH does not publicly release the minimum sample case or national estimate requirements. Variance requirements are released.