Attachment E External Peer Review Comments

APPENDIX E

Standard Electronic External Review Form

for Division of Safety Research Intramural Projects

A. IDENTIFICATION

Name of Project Officer : [Audrey Reichard, Larry Jackson]

Title of Proposed Project: [Occupational injuries and illnesses among emergency medical services (EMS) workers: A NEISS-Work telephone interview survey]

Name of Reviewer: [Robyn Gershon]

Telephone Number of Reviewer:

Fax Number of Reviewer:

E-mail address of Reviewer: [Gershon, Robyn (rg405@columbia.edu)]

******************************************************************************

B. CRITIQUE

1. Significance:

Does this study address an important problem in occupational safety? If the aims of the project are achieved, how will scientific knowledge be advanced? What will be the effect or impact of this study on the DSR mission to reduce worker injuries?

Emergency medical services (EMS) workers are a large and essential workforce in the United States. Although the exact number of EMS workers in the United States is difficult to determine, estimates range from 200,000 to over 775,000 state licensed emergency medical technicians (EMTs), not counting volunteers. The work force provides vital pre-hospital care to approximately 16 million patients each year. Yet, despite the fact that this is a large and essential work force, only a limited amount of data are available regarding their occupational health and safety. In particular, data on the epidemiology of injuries in this work group are especially sparse. Data available suggest high rates of injuries and exposure in this work-force. This study is designed to address important knowledge gaps by supplementing surveillance data collected through the national Electronic Injury Surveillance System (NEISS-Work) with telephone interview surveys of injured workers. The researchers propose to conduct this study on a stratified national sample, thus they will be positioned to provide an epidemiological assessment of non-fatal injuries and illness in EMS workers in the U.S.

2. Approach:

Are the scientific framework, design (including the composition of the study population), methods, and analyses adequately developed, well integrated, and appropriate to the aims of the project? Does the project officer acknowledge potential problem areas including feasibility, and consider alternative tactics?

This study is designed as a collaboration between NIOSH scientists and the Office of Emergency Medical Services (OEMS) in the National Highway Traffic Safety Administration (NHTSA). Data will be collected from two inter-related data sources, the routinely collected NEISS- Work data (from the occupational supplement to the NEISS) and NEISS-Work telephone interview surveys, which will be conducted by the Consumer Product Safety Commission (CPSC). CPSC staff will collect contact information from all cases identifies as EMS workers from hospital records. CPSC will

then conduct telephone interviews with injured EMS workers. Neither of the collaborating partners will have access to indentifying information on the injured EMS. The collaborating partners will then combine the data on EMS work-related injuries from NEISS with the CPSC interview data to develop a sample of injured EMS for whom extensive injury-related data are available. This sample will represent a geographically stratified sample of U.S. hospitals from urban, suburban and rural communities. Using a cross-sectional design, the researchers will analyze the final data set to estimate the prevalence of injuries in EMS workers and to identify risk factors for these injuries.

This approach is reasonable as it will allow for the supplementation of the standard information obtained by NEISS with the more detailed worker interview data. The combined data set will provide important information that will increase our understanding of the risk and risk factors for injuries with regards to EMS workers. Importantly, this rich data set will provide national estimates. The two collaborating partners have successfully worked together on a feasibility pilot study of this project. The partners complement each other and together they have the requisite capability to conduct this study. This is a risk assessment study, and as such, no cost benefit or prevention effectiveness components are included. However, this is necessary first step in the trajectory of injury prevention for this at-risk workgroup.

The methodological plan is well described, with careful attention to the data collection techniques that will be used. Special care has been paid to the inclusion criteria and case definition. The study will take place over a study period of two years, as the data will be collected prospectively over this time period. The researchers estimate that approximately 175 interviews will be conducted each year, given an estimated 50% response rate. Additional time may be required (up to as many as four years) to help ensure adequate power. The methods as described are appropriate in attaining the study objectives.

3. Innovation:

Where needed, does the project employ novel concepts, approaches or methods? Are the aims original and innovative? Does the project challenge existing paradigms or develop new methodologies or technologies?

The study is novel in two key ways. First, it will represent a powerful collaboration between two important agencies that are natural partners in injury prevention. Second, the follow-up phone interview is an innovative approach to obtaining important injury details that will be lacking from the standard incident report. Thus, the researchers will capitalize on the direct access (through CPSC interventions) to injured workers by linkage through the hospitals (through NEISS data). This is creative problem solving.

4. Project Officer (Investigator):

Is the project officer appropriately trained and well suited to carry out this work? Is the work proposed appropriate to the experience level of the project officer and other researchers (if any)? Please do not include descriptive biographical information unless important to the evaluation of merit. For new or less experienced NIOSH staff, note if the level of supervision appears adequate.

Both Ms. Reichard and Dr. Jackson from NIOSH are very experienced epidemiologists. Both are familiar with large data sets and with surveillance of injury data. Ms. Reichard, the PI, has the overall reasonability for the day-to-day management of the project and she has an exemplary record of publication of her study findings. She has the training and expertise to fulfill the role of PI. Ms. Reichard will be supervised by Dr. Jackson, an experienced injury surveillance team leader. He will presumably also assist in the complex data analysis. The NHTSA will be represented by Mr. Wijetunge, a highway safety expert with extensive EMS field experience. He will help coordinate the NHTSA data and will serve as an important link to the EMS community.

5. Environment:

Does the scientific environment in which the work will be done contribute to the probability of success? Do the proposed experiments take advantage of unique features of the scientific environment or employ useful collaborative arrangements? Please do not include a description of available facilities or equipment unless important to the evaluation of merit.

The environment for this project is excellent.

6. Overall Evaluation:

In one paragraph, briefly summarize the most important points of the Critique, addressing the strengths and weaknesses of the application in terms of the five review criteria. Recommend a score reflecting the overall impact of the project on the field of occupational safety and health, weighting the review criteria as you feel appropriate for each application. An application does not need to be strong in all categories to be judged likely to have a major impact and, thus, deserve a high merit rating. For example, an investigator may propose to carry out important work that by its nature is not innovative, but is essential to move a field forward.

The most important aspect of this study is that it will allow for national estimates of injuries in EMS to be calculated as well as for a more thorough analysis of risk factors for these injuries. This is essential in moving forward on national recommendations for prevention and risk reduction. Thus, the team is planning to address important gaps in the field of occupational health. The data will be important to State EMS programs as well, as it may lead to important new standards or guidelines. Therefore, this proposed study represents a very important first step in risk management. This is the major strength of the proposal, although another important strength is the collaborative nature of the project. The collaborators are two very well matched and complimentary entities. They have worked together in the past, and hopefully this study will lead to even more synergies. Another strength is the experienced and well established team of NIOSH investigators; their skills are nicely complimentary. The potential limitations as noted by the researchers include a concern regarding necessary funding. Another potential limitation may be a lack of participant response, however it is possible that the response will be strong, as EMS are generally quite cooperative. Another potential limitation is that the surveillance systems will be capturing only a specific subset of injuries (i.e., those requiring incident reporting and hospitalization, at least emergency care) and therefore will not capture near-misses, minor injuries and all injuries resulting in fatalities. Also, the system is not well designed to capture infections or other work-related illness; some of these may go unrecognized and/or treated in the community setting. Also, I am not sure if NEISS-Work captures EMS data from the voluntary sector Nevertheless, this system as proposed is an important first step and will positively impact the field of occupational safety and health.

******************************************************************************

C. OTHER CONSIDERATIONS

7. Gender, Minority, and Children Inclusion (As Relevant)

The researchers note that only EMS 18 years and older are eligible for participation. This is reasonable for a number of reasons, especially the fact that most agencies require EMS to be 18 years old and older. Gender and minorities will be represented as per the community representations and therefore is reasonable.

8. Human Subjects Note that NIOSH projects involving human subjects must obtain review and approval from the NIOSH Human Subjects Review Board.

The risk to subjects is reasonable. Steps will be taken to assure that they will be provided with an introductory letter prior the phone call. The option to decline is straightforward. Identification of subjects is limited and the collaborating partners will not have direct contact with subjects or their identity.

9. Researcher Hazards

N/A

10. Other

A strong letter of support from NHTSA documents a very high level of cooperation and collaboration.

Standard Electronic External Review Form

for Division of Safety Research Intramural Projects

A. IDENTIFICATION

Name of Project Officer : [Audrey Reichard, Larry Jackson]

Title of Proposed Project: [Occupational injuries and illnesses among emergency medical services (EMS) workers: A NEISS-Work telephone interview survey]

Name of Reviewer: [Gordon Smith]

Telephone Number of Reviewer:

Fax Number of Reviewer:

E-mail address of Reviewer: [Smith, Gordon (gssmith@som.umaryland.edu)]

******************************************************************************

B. CRITIQUE

1. Significance:

Does this study address an important problem in occupational safety? If the aims of the project are achieved, how will scientific knowledge be advanced? What will be the effect or impact of this study on the DSR mission to reduce worker injuries?

This is an excellent study that makes creative use of the comprehensive work – NEISS System to answer an important question regarding details of injuries to EMS workers. This group has been difficult to study because of the difficulties in identifying them in traditional occupational injury and disease surveillance systems. It is an example of the value of establishing a healthcare system based occupational surveillance system that has the capacity to do in depth follow back interviews of cases of interest identified by the NEISS.

The introduction provides a very good justification for the study and why the study is important. The objectives are also clearly stated and very achievable. However I would like to have seen more on the value of comparing rates with other workers.

Project officer response: Added text on the value of comparing EMS worker injury rates with the injury rates of other workers to the “Intended/Potential Use of Study Findings” section on page 2 of the protocol.

The study addresses an important public health problem namely injuries to EMS workers. The identification of injuries and circumstances surrounding them will provide valuable new information for directing prevention efforts. A clear dissemination plan is provided and the study should result in valuable publications.

2. Approach:

Are the scientific framework, design (including the composition of the study population), methods, and analyses adequately developed, well integrated, and appropriate to the aims of the project? Does the project officer acknowledge potential problem areas including feasibility, and consider alternative tactics?

The methods to be used in the study are clearly described and well tested in studying other conditions. The investigators provide convincing pilot data for the aims of project. The investigators also acknowledge the difficulties in achieving successful follow-up interviews. The ability to get these interviews is crucial to the success of the project. As mentioned in the proposal it is likely that EMS workers will be more likely than some other groups to participate in the study. I concur with this statement.

The study is planned for 5 years (2 years of data collection) which seems long, but the delays in OMB clearance made any reduction in time difficult. The authors acknowledge that more time may be necessary to collect enough cases. I agree and given the investment in the study it is essential it be given continuing support to allow it to achieve the goals.

It is possible that NEISS may underreport work injuries to EMS workers more than other workers. Not all their injuries may get reported into the system, especially for more minor injuries, as they may be examined either by fellow EMTs or informally in the ED. EMTs have better access to the ED than the general population. As part of the final sensitivity analysis it will be useful to compare how injuries by EMTs compare to those for other worker injuries. It may be that they appear to be more severe as the minor injuries may be less likely to get a formal medical chart and make it into the CPSC reporting system.

Identification of EMT’s is always difficult. Selection of cases based on recording occupation and industry may still under report EMT’s. It may be useful to include a specific checkbox on the work screen to indicate was this an EMT or other appropriate words.

Project officer response: While we appreciate the suggestion to include a checkbox on the work screen to indicate that a case is an EMS worker, this is not an option without adding additional time delays and, more importantly, costs to this study. We do not feel the benefit gained from a checkbox is worth the added delay and costs. In fact, we have concerns that such a box would actually reduce the number of identified cases by screening out firefighters providing EMS. Our current proposed method of case identification relies not only on occupation and industry, but also on remarks in the narrative comment field that indicate the patient was serving as an EMS worker at the time of injury. We feel that using that a computerized search of those fields as well as an additional review of possible cases as the NEISS-Work quarterly data are submitted to NIOSH will adequately capture nearly all of the EMS worker cases in the NEISS-Work dataset.

3. Innovation:

Where needed, does the project employ novel concepts, approaches or methods? Are the aims original and innovative? Does the project challenge existing paradigms or develop new methodologies or technologies?

The project is innovative in that it seeks to make use of an existing research system to answer a public health problem that has up till now been difficult to study well. It does not develop a new methodology but rather is a new application of a tried and true method of investigating work injuries.

4. Project Officer (Investigator):

Is the project officer appropriately trained and well suited to carry out this work? Is the work proposed appropriate to the experience level of the project officer and other researchers (if any)? Please do not include descriptive biographical information unless important to the evaluation of merit. For new or less experienced NIOSH staff, note if the level of supervision appears adequate.

The investigators are well qualified and have the skills to conduct the research project. Larry Jackson has extensive experience in this area and will provide appropriate vision and leadership for the study.

5. Environment:

Does the scientific environment in which the work will be done contribute to the probability of success? Do the proposed experiments take advantage of unique features of the scientific environment or employ useful collaborative arrangements? Please do not include a description of available facilities or equipment unless important to the evaluation of merit.

The research environment in which the study will be conducted is very well suited to the proposed study design. It makes excellent use of the research tool that NIOSH has invested its resources in. The Work- NEISS is a unique data system that collects valuable data on work injuries. The ability to conduct follow-up phone interviews of selected cases is a unique data system that collects valuable data on work injuries. The ability to conduct follow-up interviews of selected cases is a unique and cost effective tool for conducting in depth investigations such as those proposed.

6. Overall Evaluation:

In one paragraph, briefly summarize the most important points of the Critique, addressing the strengths and weaknesses of the application in terms of the five review criteria. Recommend a score reflecting the overall impact of the project on the field of occupational safety and health, weighting the review criteria as you feel appropriate for each application. An application does not need to be strong in all categories to be judged likely to have a major impact and, thus, deserve a high merit rating. For example, an investigator may propose to carry out important work that by its nature is not innovative, but is essential to move a field forward.

The real strength of the study is that it uses perhaps the only to method to identify and study a representative sample of EMS workers. These are a difficult group of workers to identify in standard occupational health databases. As written the proposal relies on EMS workers being identified through their occupation being recorded. I would recommend that a specific check box be included in the data entry system to specifically enquire regarding EMS workers “Yes/No”. This would prompt the data entry person to look harder for evidence that the injured person was an EMS worker as they can be given various job titles. The other concern will be whether they can get enough cases to ensure reliable results. Once the system is set up it should be continued until they reach the required sample size.

In summary, the study is innovative and will produce valuable data on EMS workers. It is a good use of the system NIOSH has set up and I enthusiastically support the proposal—score 9 out of 10.

******************************************************************************

C. OTHER CONSIDERATIONS

7. Gender, Minority, and Children Inclusion (As Relevant)

I have no concerns regarding the inclusion of gender and minority groups, and children as the proposal addresses those concerns well. No children under 18 years of age will be studied. The inclusion of females and minorities will be determined by their distribution in the population being studied.

8. Human Subjects Note that NIOSH projects involving human subjects must obtain review and approval from the NIOSH Human Subjects Review Board.

Human subjects concerns are well addressed. The investigators are well aware of the confidentiality issue involved and have a proven record of addressing these in their other studies.

9. Researcher Hazards

There are no research hazards expected.

10. Other

Appendices

There is a clear letter of support from NHTSA expressing their commitment to the study.

Appendix C Pre-interview letter. I was surprised to see that the letter starts by only mentioning CDC not NIOSH. It seems the CDC branding may not be as appropriate here as starting off with NIOSH as part of CDC.

Project officer response: Mention of NIOSH was intentionally left out of the introductory letter and the telephone script because in past NEISS-Work follow-back studies there were problems with people misinterpreting NIOSH as OSHA and declining to participate. We feel it is best to avoid this potential confusion that may impact response rates.

Standard Electronic External Review Form

for Division of Safety Research Intramural Projects

A. IDENTIFICATION

Name of Project Officer : [Audrey Reichard, Larry Jackson]

Title of Proposed Project: [Occupational injuries and illnesses among emergency medical services (EMS) workers: A NEISS-Work telephone interview survey]

Name of Reviewer: [Rebecca Heick]

Telephone Number of Reviewer: (563)355-1010/(309)738-2316

Fax Number of Reviewer:

E-mail address of Reviewer: [Rebecca Heick [epidemiologist2006@msn.com]

******************************************************************************

B. CRITIQUE

1. Significance:

Does this study address an important problem in occupational safety? If the aims of the project are achieved, how will scientific knowledge be advanced? What will be the effect or impact of this study on the DSR mission to reduce worker injuries?

This project addresses a significant knowledge gap with regard to occupational injuries in EMS. The specific information collected in the telephone survey will add depth to the knowledge base, allowing for development of more EMS-specific prevention strategies. If the aims of this project are achieved, the ability to develop evidence-based injury prevention strategies will be greatly enhanced. EMS is a unique field with many challenges that do not exist in other occupations; this work has the potential to improve the health and safety of this important, but understudied, group.

2. Approach:

Are the scientific framework, design (including the composition of the study population), methods, and analyses adequately developed, well integrated, and appropriate to the aims of the project? Does the project officer acknowledge potential problem areas including feasibility, and consider alternative tactics?

In reviewing the research protocol, a few issues of concern arose. My first concern is that the study population may be smaller than the researchers anticipate (requiring additional years of data collection and/or lacking in generalizability). They acknowledge they are studying only those injuries treated in the ED (which are likely to be more severe and less frequent) and state that they believe ED treatment for injury seems logical in that the EMS provider is likely transporting the patient to that location anyway (i.e. convenience). Based upon prior research (and personal experience as an EMS provider), I believe this reasoning may be faulty and the numbers likely lower than they plan as EMS providers may not seek treatment in the ED, but rather wait until the shift is concluded and then seek treatment elsewhere. In addition, requirements of workers’ compensation carriers may prohibit the injured provider from reporting to the ED unless the injury is considered serious or the result of a motor vehicle crash, etc. where the involved mechanism of injury is significant. I believe this scenario of underrepresentation may be especially true of back injury (which is identified as a frequent problem among EMS providers) and so it may be underrepresented in the study data.

Project officer response: We acknowledge that the study population may be smaller than expected (see Limitations section on page 15). However, our estimates are based on past numbers of EMS workers that appeared in the NEISS-Work database and are tempered by the current response rate to NEISS follow-back telephone interviews. We are expecting that after two years we will have a broad level of reportable data. NHTSA and NIOSH are both committed to this project and as long as funding and staff continue to be available, the projected intent is to collect data for up to four years if needed to allow for reportability of responses that represent at least 5% of all respondents. The possibility of collecting data for an additional two years is noted in the footnotes of the study time line section (page 4) as well as in the Estimated Number of Participants section on page 7.

We also acknowledge that EMS providers may seek treatment at other venues. This may be especially true in cases of sprains and strains that may not demand emergency treatment. We have added a paragraph on page 14 to specifically address this limitation in interpretation of the numbers resulting from the study population. However, it is important to note that our projected numbers for EMS worker participation in this study are not based on the idea that EMS workers may seek treatment at ED venues more commonly. Clarification of how we arrived at the estimated numbers of EMS participants in this study has been added to page 8.

My second concern is that the researchers may not be able to meet the aims of the project, especially those focused on the magnitude and types of injury that occur (what percentage of all injuries will likely be ED treated?) and the characteristics of injured providers. This concern stems from the fact that they focus only on ED-treated injuries and that (again) response rates may be low and influenced by whether an EMS provider is paid or volunteer (paid more likely to suffer certain types of injury that may require ED treatment?) or serving on a particular service type. On some level the discussion about “injuries of greatest concern” (Obj. 2) boils down to this question: Is one ED treated injury more likely to end a career (or impact quality of life) than one or more non-ED treated injuries (and therefore be of greater concern)? I’m not sure this study as it currently stands can adequately address this aim.

Project officer response: The objectives section has been more explicitly defined by including the descriptive “ED treated” within objectives 1, 2, and 4. However, we do feel we will be able to offer some insight into the larger issue of all injuries occurring to EMS workers as we have cited that an estimated 34% of all work-related injuries are treated in the ED (page 14).

The researchers have adequately addressed the potential limitations due to inclusion of only ED treated injuries and low response rates. I would like to see additional consideration given to the issue of whether or not the aims can be adequately addressed by the study (i.e. second concern above). These are issues that are commonly encountered in studies involving EMS providers and are difficult to address short of conducting a large prospective cohort study of a random stratified sample of EMS providers (likely not feasible due to cost constraints).

3. Innovation:

Where needed, does the project employ novel concepts, approaches or methods? Are the aims original and innovative? Does the project challenge existing paradigms or develop new methodologies or technologies?

I do not see significant innovation, challenge to existing paradigms, or development of new methodologies in the design of the study; however, at this juncture, I believe these factors are secondary to our lack of sound basic data regarding the occupational injury experiences of EMS providers.

4. Project Officer (Investigator):

Is the project officer appropriately trained and well suited to carry out this work? Is the work proposed appropriate to the experience level of the project officer and other researchers (if any)? Please do not include descriptive biographical information unless important to the evaluation of merit. For new or less experienced NIOSH staff, note if the level of supervision appears adequate.

I believe the project officer and the other researchers are well prepared and that this work is well within the capabilities of this team. There is a great deal of experience with NEISS-Work and surveys. I also believe it is a great strength to have an EMS practitioner on the team.

5. Environment:

Does the scientific environment in which the work will be done contribute to the probability of success? Do the proposed experiments take advantage of unique features of the scientific environment or employ useful collaborative arrangements? Please do not include a description of available facilities or equipment unless important to the evaluation of merit.

I believe that with the established collaboration between NHTSA and NIOSH this project is likely to produce data that is both sound and useful. With the support of both organizations behind this project, it is likely to find success in adding to the limited knowledge that currently exists.

6. Overall Evaluation:

In one paragraph, briefly summarize the most important points of the Critique, addressing the strengths and weaknesses of the application in terms of the five review criteria. Recommend a score reflecting the overall impact of the project on the field of occupational safety and health, weighting the review criteria as you feel appropriate for each application. An application does not need to be strong in all categories to be judged likely to have a major impact and, thus, deserve a high merit rating. For example, an investigator may propose to carry out important work that by its nature is not innovative, but is essential to move a field forward.

Overall, I would give this application 84 out of 100 points. This work carries great significance because of the lack of literature in this area and has significant potential to further our development of evidence-based prevention strategies (35/35). I do have a few concerns about the methodology and approach for this study (28/40). The sample size is likely to be low with only two years of data collection planned. This low sample size (and the fact that only ED treated injuries are included) may lead to erroneous conclusions, especially if certain types of injuries or certain types of providers are more likely to seek ED treatment (which I strongly believe to be the case). I do see great strength in the details that will be collected in the phone interview. This type of information is often lacking in other studies and will be very beneficial. While I do not see significant innovation in the study, it is still very valuable (1/5). I believe that all members of the research team are well versed in the methods needed to carry out the study successfully and that the expertise of a practicing EMS provider is an important piece (10/10). The environment of collaboration between NIOSH and NHTSA will help to ensure that the study is completed in a sound scientific manner and that knowledge will be gained from this work (10/10). In my opinion, this study (though carrying certain limitations) has great potential to add to our knowledge regarding occupational injuries among EMS providers. By remaining cognizant of the limitations of this methodology, I believe the work can be conducted and presented in such a manner as to benefit all stakeholders and drive future research.

******************************************************************************

C. OTHER CONSIDERATIONS

7. Gender, Minority, and Children Inclusion (As Relevant)

N/A

8. Human Subjects Note that NIOSH projects involving human subjects must obtain review and approval from the NIOSH Human Subjects Review Board.

The human subjects’ protections that are in place are adequate to protect participants.

9. Researcher Hazards

N/A

10. Other

In reviewing the appendices, I would like to make a few additional comments. In Appendix B, you do not mention “training” in any of the boxes on the left, though it is included in the text of the proposal and in the “Goal” box at the bottom of the page. I find this inconsistency confusing.

Project officer response: Training has been added to additional areas on the flowchart in Appendix B.

In Appendix D under “Injury Details” (question 7), consider how a person who is “on-call” (possibly 24/7 if volunteer) would respond to this question. I see where you are headed (possible contributions of fatigue, etc), but find that this question could be hard to answer for some respondents.

Project officer response: Clarification was added to this question that “on call” hours not involving actual duty should not be included.

In the “Motor Vehicle Questions,” you might consider adding information about whether it was a patient transport or not as this may influence responses to other questions (location, seatbelt use, etc).

Project officer response: While it would simplify analysis to add this question, the section containing motor vehicle questions is already lengthy. We feel that if needed, this information can be collected from the information provided in the narrative injury description with a fairly high degree of accuracy.

For question 32, please consider adding CPR as an option (rather than leaving it as a “fill in” answer) as exposures during CPR do occur with some frequency.

Project officer response: This option was added to question 32.

Question 44: Why are you only interested in whether or not the patient was under the influence? This question could be rewritten to include other choices and would garner additional information.

Project officer response: We initially had other questions asking whether the patient was under the influence of drugs or diagnosed with a psychiatric disorder. However, after much discussion with people who have worked as EMS providers, it was decided that the responses to this would be highly subjective and potentially very inaccurate. We left the alcohol related question as there are obvious signs to alcohol intoxication that EMS workers are likely to accurately assess to a high level of validity.

Questions 46 and 55: Consider including a response for slick surfaces or wet/muddy surfaces – this can be an important contributor to these injuries.

Project officer response: This has been added. See questions 47 and 51.

In question 67, you list the AED only. Please consider rewording this to read AED or cardiac monitor (or something similar) as a cardiac monitor and AED are not the same thing.

Project officer response: This change has been made.

1