Supporting Statement B

A SURVEY OF FIRE AND LIFE SAFETY EDUCATION ACTIVITIES

AMONG PENNSYLVANIA FIRE DEPARTMENTS

Information Collection Request

PART B

Submitted by:

Shane Diekman, PhD, MPH

Behavioral Scientist

Home and Recreation Injury Prevention Team

Division of Unintentional Injury Prevention

National Center for Injury Prevention and Control
Centers for Disease Control and Prevention

4770 Buford Highway, NE, Mailstop K-63

Atlanta, GA 30341

Work: 770-488-4901, Fax: 770-488-1317

Email: sdiekman@cdc.gov

March, 2008

TABLE OF CONTENTS

Pennsylvania Case Study Evaluation

B. Statistical Methods 3

B.1. Respondent Universe and Sampling Methods 3

B.2. Procedures for the Collection of Information 3

B.3. Methods to Maximize Response Rates and Deal with Nonresponse 3

B.4. Tests of Procedures or Methods to be Undertaken 4

B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 5

List of Tables 7

B. Statistical Methods

B.1. Respondent Universe and Sampling Methods

The respondent universe includes all chiefs of Pennsylvania fire departments. The 2007 National Directory of Fire Chiefs and EMS Administrators, a proprietary database obtained for this study, contains detailed information on the 2,199 Pennsylvania fire departments. For the proposed study, a total of 454 fire departments will be not be eligible for sampling because they are missing data in the National Directory (n=354), they serve populations less than 500 individuals (n=26), they are a non-public fire department (n=5), or they participated in the 2006 national FLSE survey (n=69). [Note: To reduce respondent burden, data will be used from fire departments participating in the 2006 national FLSE survey, but those fire departments will not be sampled in the present study]. Of the remaining 1,745 fire departments, all career (n=35) and career/volunteer combination (n=117) fire departments will be selected for study inclusion as well as a random selection of volunteer only fire departments (n=502); the sampling strategy for the volunteer only fire departments will result in the selection of 50% of the volunteer fire departments serving populations greater than 10,000 and 25% of the fire departments serving smaller populations. The final selected study sample will include 654 fire departments.

B.2. Procedures for the Collection of Information

A pen-and-paper survey instrument (Appendix C) will be mailed to the leadership of all 654 fire departments in Pennsylvania identified for study selection through the 2007 National Directory of Fire Chiefs and EMS Administrators. A mailed questionnaire format will allow participants the opportunity to complete and return the survey at their convenience. A cover letter, mailed with the survey instrument, will include an invitation to participate in the survey from the leadership at FEMA and the Pennsylvania Fire Commissioner. This letter will also explain the purpose of the survey, the voluntary nature of the research, the procedures used for confidentiality, and provide contact information for questions or concerns. Participants will have the option of returning their completed surveys by mail in a self-addressed stamped envelope provided, by fax, or participants can request an electronic copy of the survey that can be returned by e-mail. Based on 2006 FLSE survey experience, it is expected that a small proportion (less than 10%) of respondents will electronically submit their responses. Reminder postcards will be sent to fire departments that have not responded to the survey within four weeks and follow-up phone calls will be made to fire departments that have not responded by eight weeks.

B.3. Methods to Maximize Response Rates and Deal with Nonresponse

To increase the accuracy and reliability of the data collected for this survey, several methods will be used to maximize the response rate. First, a letter from national and local leaders in the fire service will be included in the initial mailing of the survey. This letter will draw more attention to the survey, increase the credibility of these efforts among respondents, and stress the importance of the survey participation. Second, aggressive follow-up procedures will be implemented beginning one month after the initial survey mailing. Fire departments that have not completed the survey and have not declined participation will be sent a postcard reminder also from national and local leaders in the fire service. This postcard will again emphasize the importance of the survey and include contact information for questions or additional copies of the survey. Two months after the initial mailing, phone calls will be made to non-responding fire departments. These phone calls will occur at different times of day to maximize likelihood of reaching fire chiefs when they are available at the office. Up to three phone calls will be made to each department on varying days of the week and times of day. Additional copies of the survey will be sent upon request to departments through mail, e-mail, or fax. At three months following the initial survey mailing, these follow-up phone call procedures will be repeated for departments that have still not completed the survey or declined participation.

Approximately four months after the initial survey mailing, data collection and all follow-up procedures will cease. At that time, we expect to have an estimated 260 completed surveys and a 40% response rate. This response rate estimate is based on the 2006 national administration of the FLSE survey, the scope of the proposed project, and expected personnel resources for follow-up. In 2006, the national FLSE survey was distributed to 7,390 fire departments and 1,523 responded, which equates to a 20.6% response rate. However, due to limited resources, follow-up phone calls were only placed to a percentage of non-responders. Given the smaller scope of the proposed project and the availability of a full-time project coordinator, we will be able to make follow-up phone calls to all non-responding departments.

Once all surveys have been collected, the accuracy and reliability of the data will be checked through analyses of non-response and missing data across the study period. An analysis of non-response will highlight differences between survey responders and non-responders by comparing the two groups on known variables such as fire department type (career, volunteer, or combination) and size of the population the department serves. An analysis of missing data will determine if any significant differences exist in selected survey items across the data collection period.

B.4. Tests of Procedures or Methods to be Undertaken

Johns Hopkins School of Public Health will employ procedures throughout the project lifecycle to ensure a high quality product. Quality control began during the national instrument development phase. Experts in the fire service field, national fire service organizations, and fire chiefs provided feedback regarding wording comprehension, type of questions, and other factors that determine if the instrument is capturing the information it is intended to capture. The content and design of the proposed survey instrument has been modified for its administration in Pennsylvania in collaboration with the Home Safety Council, CDC, and Johns Hopkins School of Public Health. Lessons learned from the 2006 national FLSE survey administration and feedback received from fire chiefs, either written on the survey itself or through presentations at conferences, have been incorporated into an updated version. In addition, several formatting changes, such as providing clearer directions and definitions, offering examples, and changing the response options, will make the survey easier for respondents to complete.

B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Leading national fire service and fire and life safety organizations were consulted to inform the development of the 2006 national FLSE survey.

Table B-1: Member Organizations on the Survey Advisory Committee

Organization	Contact Person	Title	Telephone
State Fire Marshal’s Office, Virginia Department of Housing and Community Development	Charles E. Alitzer	Virginia State Fire Marshal	(804) 371 - 7170
National Association of State Fire Marshals	James Burns	President	(877) 996 - 2736
Carson Associates, Inc.	Peg Carson	Public Education Specialist	(540) 347 - 7488
National Fire Protection Association	John Hall	Assistant Vice President, Fire Analysis and Research Division	(617) 770 - 3000
National Fire Protection Association	Judy Comoletti	Assistant Vice President of Public Education	(617) 984 - 7287
International Fire Service Training Association	Dennis Compton	Fire Chief	(405) 744 - 5723
National Fallen Firefighters Foundation	Teresa Crisman	Public Affairs	(240) 882 - 6772
International Association of Fire Chiefs	Joelle Fishkin	FLSS staff liason	(703) 273 - 0911
National Volunteer Fire Council	Sara Lee	Deputy Director	(202) 887 - 5700
Dallas Fire & Rescue Department	Joe Pierce	Deputy Chief	(214) 670 - 4607
National Volunteer Fire Council	Heather Schafer	Executive Director	(202) 887 - 5291
JR Communications	Julie Reynolds	Marketing Communications Specialist	(781) 344 - 2055
Home Safety Council	Meri-K Appy	President	(202) 330 - 4900
Home Safety Council	Angela Mickalide	Director of Education and Outreach	(202) 330 - 4900

Johns Hopkins School of Public Health, which administered and analyzed the 2006 national FLSE survey, will oversee data collection and perform both the descriptive and analytic steps of the analysis plan.

List of Tables

Table B-1: Member Organizations on the Survey Advisory Committee