NIOSH Customer Satisfaction Survey - Publication 2003

May 24 2003

NIOSH Publications and Dissemination:

Customer Satisfaction Survey

Abstract

In 2003, NIOSH, in collaboration with its stakeholders, developed and administered customer satisfaction survey for mailing. The purpose was to determine stakeholders’ experience with NIOSH publications and information services. Six-hundred eighty eight of the 1,200 anonymous and randomly selected participants responded, yielding a 57% response rate. Each of the respondents was affiliated with one of four professional associations: the American Association of Occupational Health Nurses (AAOHN), the American College of Occupational and Environmental Medicine (ACOEM), the American Industrial Hygiene Association (AIHA), and the American Society of Safety Engineers (ASSE). In filling out the questionnaire, the majority of respondents identified themselves as being well educated and experienced professionals, who worked in large organizations with Internet access and performed a variety of occupational safety and health (OSH) activities. The respondents ranked OSHA and NIOSH as primary sources of OSH information. Between 70% and 80% of them “use or refer to” NIOSH informational products and had taken a course where NIOSH materials were used. Forty-nine percent of them acknowledged that NIOSH informational materials were cited or referred to in their organization’s safety and health policies and procedures. Ninety-seven percent strongly agreed or agreed that “NIOSH is an important resource for the occupational safety and health community.” The respondents also strongly agreed or agreed with statements that characterized NIOSH publications as useful, practical, clear, and impartial. The NIOSH Web site is popular with this audience. They requested additional informational materials for educating and informing others and were also interested in practical applications and guidelines. Although the findings reflect favorably for NIOSH, between 20% and 25% of the respondents had not used, or were not familiar with, NIOSH publications or information materials. These survey results suggest that NIOSH (1) could expand its outreach to the those 20%-25% who did not use NIOSH materials; (2) could examine the proportion of technical to non-technical publications NIOSH produces; (3) could re-work dissemination plan strategies; and (4) could expand future efforts to include non-OSH professionals. As a cautionary note, the 43% non-respondents may limit the ability to make inferences about the full population.

Customer Satisfaction Survey: Executive Summary

NIOSH, over 30 years, has produced nearly 5,000 documents bearing the Institute name.¹ The majority of these informational products are available from the NIOSH Web site, the NIOSH technical inquiry service, publication distribution office, or from NIOSH exhibits. As public concern for occupational safety and health issues has increased, the demand for NIOSH’s informational materials also has increased.

A NIOSH goal is to assess the impact of its informational materials on the safety and health of workers. Government agencies are using “customer satisfaction surveys” as a tool to assess performance standards that influence impact. In 2003, NIOSH collaborated with four of its major stakeholder associations in developing and administering a customer satisfaction survey. The organizations were the American Association of Occupational Health Nurses (AAOHN), the American College of Occupational and Environmental Medicine (ACOEM), the American Industrial Hygiene Association (AIHA), and the American Society of Safety Engineers (ASSE). To ensure data quality, the survey instrument was reviewed by an independent survey expert prior to distribution, all data transcription from survey to e-file were double entered by independent teams, and the data were analyzed by an outside statistical survey team.²

The NIOSH Customer Satisfaction Survey (NCSS), consisting of 26 questions, 6 sections, and 11 printed pages, was reviewed and approved for distribution by the Office of Management and Budget (OMB No. 0920-0544) for a one-year period beginning in March 2002 and ending on March 31, 2003. NIOSH mailed the survey in mid January 2003 to a random sample of 1200 members, 300 from each of the four associations. Participation was voluntary and all responses were anonymous. Six-hundred eighty eight (688) surveys were completed and returned to NIOSH by March 31, 2003. The combined response rate from the four associations was 57 percent (688/1200).³

Key findings included the following:

Availability/Usage of NIOSH Publications⁴

 Seventy-nine percent of the survey participants when asked have you ever “used or were referred to a NIOSH publication;” responded yes (539); 21% (141) marked no, and 8 were blank. ⁵ (Q. #11)

 Eight-four percent of the survey participants identified OSHA and 74% identified NIOSH as primary sources of occupational safety and health information (OSH) when asked: “In the past 12 months, have you read or referred to OSH information published by any of the following organizations or sources”?(Q. #10 (16&18)⁶

 Seventy-six percent of the survey participants responded yes when asked if NIOSH publications had been used as informational materials in a course or educational program that they had attended (10%- no, 14%-don’t know). (Q. #13)

• Forty-nine percent of the survey participants indicated that NIOSH publications had been “referred to or cited by name” in their “organizations’ policy and procedures,” whereas 30% indicated they had not been cited, 17% did not know and 4% did not apply. (Q. #19)

• Eighty percent of the survey participants indicated that their organization’s safety and health practices had been influenced by NIOSH publications either a lot (36%) or a little (44%), whereas 3% indicated no and 17% did not know. (Q. #20)

• Seventy-six percent of the survey participants, on at least two occasions in the past year, recommended to a colleague one more of NIOSH’s information or communication products.⁷ (Q. #14d)

NIOSH as an Information Source

• Ninety-nine percent of the survey participants who used or referred to NIOSH materials either strongly agreed (71%) or agreed (28%) with the statement that “NIOSH is a credible source for obtaining OSH information,” 1% had no opinion. (Q. #15(a))

• Ninety-seven percent of the survey participants who used or referred to NIOSH materials either strongly agreed (65%) or agreed (32%) with the statement that “NIOSH is an important resource for the OSH community,” 3% had no opinion. (Q. #15(h))

• Nine-two percent of the survey participants who used or referred to NIOSH materials, either strongly agreed or agreed with survey statements that characterized NIOSH’s communication products as containing up-to-date information (current), impartial (80%), at the appropriate technical level (92%), clearly written (91%), and provided useful recommendations (92%). (Q. #16(a-e))

• Sixty-eight percent of the survey participants reported using NIOSH printed publications either frequently (24%) or occasionally (44%) in the past 12 months. (Q. #14(a)) A similar number of respondents also reported accessing the NIOSH Web site either frequently (32%) or occasionally (36%) in the past 12 months.⁸ (Q. #14(b))

• Seventy-eight percent of the survey participants requested or received occupational safety and health information from NIOSH through its Web site, 43% through NIOSH’s technical information inquiry service (800#), and 16% through NIOSH’s exhibit program. (Q. #12)

• Eighty-seven percent of the survey participants who used or referred to NIOSH materials either strongly agreed (38%) or agreed (~49%) with the statement that “the NIOSH Web site provides high-quality, useable information” (1% disagreed, and 12% had no opinion). ((Q. #15(c)) Furthermore, 84% of the respondents either strongly agreed or agreed with the statement that “The NIOSH Web site provides effective access to NIOSH information and publications (2% disagreed and 14% had no opinion). (Q. #15(d))

Use and Types of NIOSH Publications

• Survey participants reported having referred to NIOSH publications at least three times or more during the past 12 months for the following purposes or tasks (Q. #18)

• Conducting occupational safety and health training

• Establishing occupational safety and health programs

• Performing hazard exposure assessment

• Formulating new/revised safety and health policy/practices

• Survey participants assigned a value to a list of suggestions for improving NIOSH’s information dissemination, and placed the highest value (sum of values1&2)⁹ on the following items (Q. #21(h,c,l,f) :

  • Package/distribute NIOSH publications around common themes

  • Announce publications on the NIOSH Web site

  • Create CD-ROM collections of publications

  • Publicize and increase circulation of Health Hazard Evaluation Reports

• Survey participants identified the following types of publications that “NIOSH should consider emphasizing in the future.” Multiple responses were allowed. The topics are listed below in order based on the percentages of responses: (Q. #25):

• Practical applications and guides (check lists, “how to,” self audits

• Recommended safety and health standards, criteria documents

• Booklets for educating and informing workers/owners and managers Case studies (Health Hazard Evaluations/Fatality Investigations)

• Scientific/technical documents

• Analytical methods manual (35%)

Conclusions:

Overall Favorable view of NIOSH services/products

• The respondents to this survey are actively involved in professions directly related to occupational health and safety, and are among the best educated of the NIOSH audience. As a result, these factors created a group of respondents who are practitioners and have the potential to provide an accurate and critical assessment of NIOSH material.

• The overall positive findings expressed by this group about NIOSH products and services lend real weight to the favorable conclusion.

• Specifically, the respondents agreed with statements characterizing NIOSH publications as credible, impartial, and useful.

Recommendations

• Develop strategies for increasing positive response levels on questions of utility, relevance, quality and accessibility.

• Expand web-based announcements of NIOSH publications.

• Produce more theme type publications, CD collections of materials and HEE collections.

• Survey other important audiences for NIOSH materials including employers, workers, labor and trade associations, and other nonprofessional bodies.

• Assess NIOSH dissemination plans. Results from this survey suggest that NIOSH continue update its mailing list to ensure that NIOSH publication products are reaching those individuals or organizations that have a continuing need for copies of all publications. Re-thinking of the direct mail route of dissemination could perhaps save the Institute money, and improve customer satisfaction.

ACKNOWLEDGEMENTS

Any project of this scope involves the dedication and efforts of a large number of individuals. The conception and overall responsibility for the project belongs to Paul Schulte, Director, Education and Information Division (EID). Michael Colligan, renowned social psychologist, served as the original project officer. Dr. Colligan brought together the stakeholders and collaborators from each of the four participating professional organizations (AIHA, ASSE, ACOEM, and AAOHN). He established a series of focus groups that lead to the first draft of the survey. Alexander Cohen, an expert in occupational safety and health training and research, conducted interviews with members from the professional associations and provided technical direction in the development and design of questions for the survey. Vern Anderson was responsible for the completion of the survey development, direction of the survey, analysis and development of the report. Andrea Okun assisted in the oversight and assurance of quality for the project. In 2003, a post doctoral student and health communication researcher, Nancy Muturi, fielded tested and further refined the wording of the questionnaire items. She also constructed the supplemental “List of NIOSH Publications,” and developed a random sampling process for generating mailing lists from each of the four sets of databases. The databases of names and addresses were provided by the current presidents of each of the four participating professional associations. The final version of the survey was reviewed, prior to printing, by Alfred Tuchfarber, a survey expert from the Institute for Policy Research, University of Cincinnati, Ohio. The printing was supervised by Penny Arthur, NIOSH staff. Survey mailings were performed by the NIOSH Publication distribution-contract staff, headed by Sherri Diana. Cathryn Martin, special office assistant, served as logistic and quality control coordinator in managing, accounting for, and securing the surveys. Alexa DuWors organized and compiled the qualitative data. A professional data entry firm (CONVERGYS) was hired to transcribe the information from the printed questionnaire into a dataset compatible for analyses. Eric Rademacher, a survey expert and statisticians from the University of Cincinnati, Institute for Policy Research, provided tables of percentages for each of the 26 survey questions as well as pertinent review comments. NIOSH programmer and statistician, Rolland Rogers, Jr., performed an independent analysis that confirmed that the original data set was formatted appropriately and that all variables were properly labeled and accounted for. Cathryn Martin assisted in constructing the charts, tables and formatting for this report. Writer-editor Amy Lamborg reviewed this draft.

.

Table of Contents

NIOSH Publications and Dissemination: 1

Customer Satisfaction Survey 1

Abstract 1

Customer Satisfaction Survey: Executive Summary 2

Availability/Usage of NIOSH Publications 3

NIOSH as an Information Source 4

Use and Types of NIOSH Publications 5

Conclusions: 6

Favorable view of NIOSH services/products 6

Ideas for improvement 6

Ideas for more research Error! Bookmark not defined.

ACKNOWLEDGEMENTS 7

Introduction to the Study 10

Methods of Research 12

Focus Groups 12

Survey Content 12

Sampling Methods 13

Population 13

Sample Size 13

Response Rate 13

Distribution Size 14

Survey Mailing 14

Data Analysis Plan 15

Data Quality Procedures 15

Survey Procedure 16

Survey Research Findings 17

Mailed and E-mailed Survey Returns 17

Affiliation Response Rate 17

Section I: Background of Respondent 17

Section II: Respondent’s sources for OSH information 19

Printed, Electronic, and Others 19

Electronic Sources 19

Sources of OSH information 20

Accessing NIOSH Information 21

Use of electronic and conventional access 22

Use of NIOSH information in educational/training courses 23

Determining how frequently NIOSH publications and web site are used. 24

Section III: NIOSH as a Source for OSH information. 26

Modes of information delivery 26

Section IV: Opinions about NIOSH Publications 29

Ratings of NIOSH’s publications 29

NIOSHWeb site and the 800# 31

Section V: Use of NIOSH Publications 31

How NIOSH materials are used 31

Impact of NIOSH on policies and procedures 32

Section VI: Suggestions for NIOSH Products and Services 34

Improving NIOSH information dissemination 34

Discussion-Conclusion 39

Future Actions 41

Appendix: I 43

Response Summary Questionnaire 43

Reference Section 53

Introduction to the Study

Customer satisfaction surveys are one tool to meet the objectives of the Government Performance and Results Act (GPRA), CDC: President’s Management Agenda: Home Page and Executive Order 12862, Setting Customer Service Standards. All government agencies are mandated to create performance standards, including measures of customer service and to survey customers regarding their level of satisfaction with the services and benefits provided.

As mandated in the Occupational Safety and Health Act of 1970 (PL 91-596), the mission of the National Institute for Occupational Safety and Health (NIOSH) is to conduct research and investigations on work-related disease and injury and to disseminate information for preventing identified workplace hazards (Sections 20 (a) (1) and (d)). This dual responsibility recognizes the need to translate research into workplace application to impact worker safety and well-being. Specifically, NIOSH, through its recommendations and communication efforts, seeks to promote greater awareness of occupational hazards and their control, influence public policy and regulatory action, shape national research priorities, change organizational practices and individual behavior, and ultimately, improve American working life. While the means for packaging and delivering NIOSH-generated information varies, one of the primary communication vehicles is its series of numbered print publications.

Each year NIOSH develops and disseminates between 35 and 50 primary “numbered” publications, along with one hundred or more annual “series” publications, such as the NIOSH FACE Reports, and the Health Hazard Evaluation Reports. Today, the NIOSH publication inventory contains nearly 4,500 scientific and educational documents/ communication products relevant to occupational safety and health issues.¹⁰ These publications can be accessed through NIOSHTIC-2, a web-enabled searchable database providing public access to more than 33,000 NIOSH-funded publications produced since 1970. (http://www.cdc.gov/NIOSH/NIOSHTIC2).

The NIOSH Customer Satisfaction Survey (NCSS) was designed to determine: (1) to what extent NIOSH publications are perceived as credible, useful sources of information about occupational safety and health issues (a product evaluation); (2) to what extent is NIOSH successful in distributing its occupational safety and health information products to the appropriate user audience (a marketing evaluation); and (3) to what extent (and in what ways) have NIOSH publications influenced workplace safety and health program practices (an impact evaluation). The project aim was to determine what improvements could be made in the nature of NIOSH publications and/or their manner of delivery that could enhance their use and benefits.

The purpose of this report is (a) to describe the process for developing and administering the NCSS, (b) to report the findings from the survey, and (c) to consider the implications for future actions. The results will be used by those involved in preparing and disseminating various types of NIOSH documents to aid in future writing and distribution efforts.

In addition, the survey results will be shared with the partner associations, namely, ACOEM, AAOHN, AIHA, and ASSE. The outcomes of the survey hold important benefits to the associations’ individual members. Specifically, each association will learn more about their members’ work activities, organizations for which they work, years of experience, education, sources of OSH information and their training needs. In turn, the associations can form alliances with NIOSH to participate in NIOSH’s strategic planning to assure NIOSH is meeting their needs for protecting workers.

Methods of Research

Focus Groups

The survey content and format was developed with the assistance of stakeholders who participated in one of three local focus groups. The members from the focus group were drawn from members of the four collaborating associations, namely the AIHA, ASSE, AAOHN and ACOEM. Members of these organizations were selected because it was necessary to recruit persons who had some knowledge and expertise in occupational health and safety and who were likely to be familiar with NIOSH.

The first focus group examined different types of survey instruments that have been used by various organizations to assess customer satisfaction. Ideas for specific questions were identified and various formats were considered. Members of the subsequent focus groups were given draft questionnaires to determine their opinions, concerns, beliefs, awareness, and experiences related to the survey questions. Their opinions were used to modify and improve the survey used in this study. The final version NCSS and protocol was reviewed by three nationally recognized survey experts as well as the CDC Human Subjects Review Board.

Survey Content

The NCSS consisted of six sections, as outlined in Table 1. Each section was designed to illicit specific information that would assist NIOSH in focusing its publications on the needs of the users. The report of the findings from this survey also is organized by section. The survey mailing also included a seven page List of NIOSH Publications. This list of approximately 260 publications (1997-2002) was provide as a memory aid to the survey.

Content		Questions
Section I	Background -Tell Us About Yourself	Q1--Q8
Section II	Your Sources for OSH Information	Q9--Q14*
Section II	Your Opinions About NIOSH’s Information	Q15--Q17
Section IV	Your Opinions About NIOSH Publications	Q16
Section V	Your Use of NIOSH Publications	Q18--Q20
Section VI	Marketing NIOSH Products & Services	Q21--Q26
Final Comments: Open Structure
*Question 11 served as a branching question that if answered “no,” terminated the survey.

Table 1 Survey contents and associated questions

participants. The list, which was organized by categories and topic areas, industry, media and type of hazard also served as a useful reference guide for the survey participants to assist them in ordering additional publications through the NIOSH 800# publication ordering system. The survey participants were also directed to the NIOSH Web site where electronic versions of the titles were available.

Sampling Methods

Population: The target audience consisted of occupational safety and health professionals who belonged to one or more of the four primary professional organizations: AIHA (12,400 members), ACOEM (6,500 members), AAOHN (13,000 members) and ASSE (33,000 members). Of the largest group (ASSE), only a segment identified as Certified Safety Professionals (CSP’s, N=7000) were included in the target population because of the type of work they did. As a result of the members’ professional affiliations and status, the final target population of about 40,000 was considered for this survey to represent a relatively homogenous population.¹¹

Sample Size: Because the target population was considered homogenous for statistical purposes (internal variances are small), the sample size needed for the analysis is also smaller. Dillman (2000) provides a formula and Table 5.1 for estimating sample sizes for populations considered “more uniform in their response variations” than a 50/50 split. According to Dillman when the target population exceeds five thousand (up to one million or more), the required sample size for this type of population may range from 660-700.¹² Specifically, a sample size of 700 is more than sufficient for 95% confidence level with a sampling error of + 3%. This precision level is appropriate here for making needed inferences and estimating proportions.

Response Rate: An appropriate response rate is determined from the size of the acceptable sample. Since this was NIOSH’s first randomized survey mailing, no precedent existed.¹³ The literature suggested that mailed surveys might yield response rates as low as 20% to as high as 80 % (Dillman, 2000; Hayes, 1998). In situations, however, where the sampled respondents are connected or affiliated with a common organization or an association, a response rate of 60% or better can be expected (Dillman, 2000; Cochran 1977). A response rate of 60%.was anticipated because the survey population was affiliated with a professional organization that endorsed the NIOSH survey. Indeed, the associations provided support and publicity by providing letters to their membership encouraging participation in the NIOSH survey. Finally, NIOSH followed nearly all recommendations from survey experts (Hayes, 1998) for improving response rates, which included use of follow-up letters and reminder cards at appropriate intervals, as well as the offer of a token appreciation item (i.e., t-shirt, valued at $2.50).

Distribution Size: Once the response rate was projected and the required sample size was established, the distribution size was determined.¹⁴ Distribution size is computed by dividing the required sample size of 700 by the estimated response rate (60%). Rounding up to the nearest 100, yields a proposed distribution size of 1200 for the survey mailing. With four associations, the 1200 distribution plan allowed an equal number of persons (300) to be drawn at random from each of the four associations. By choosing an equal number of surveys (300) to be mailed to each group, each association had an equal opportunity to respond and thereby make their views known.¹⁵ In this way a large association was not able to exert more influence than a smaller association on the survey results.¹⁶

The final sample was screened to ensure that persons having multiple association memberships were selected once, and assigned to the group most aligned with their professional training based on the title listed on their membership address. NIOSH employees who are members of the four associations were excluded from the membership listings prior to drawing the sample. An electronic version of the questionnaire also was developed, as suggested by OMB in their approval notice, giving respondents the option to complete the survey on the NIOSH Web site and email the survey anonymously back to NIOSH. Table 2, Rows (a-b) provides a view of the sample, segmented by professional association.

Survey Mailing: The survey mailing (January 21, 2003) was preceded by a personalized letter sent two weeks earlier from NIOSH. The pre-survey letter was designed to alert the member that he/she had been randomly selected from a directory provided by their association to participate in the NIOSH survey and that the survey package would arrive in approximately two weeks. The pre-survey letter also included a letter from the president of the respondent’s association pledging the support for the survey. In addition, the survey director’s phone number and email address was included for those members who wanted further information or who did not want to participate in the survey. Of the 1200 mailed surveys, 11 were returned with unknown addresses. The eleven were replaced with random selections from each association from which the unknown address was identified. Reminder post cards were sent to non-respondents two weeks after the original survey mailing. Four weeks after the initial survey, a second follow-up (reminder) mailing with a second copy of the survey was sent to those individuals who had not returned the questionnaire. The initial mailing generated a 38% response rate. The follow-up reminder mailing yielded another 13%. By March 31, the remaining 6% of the surveys were returned.

Data Analysis Plan

This report is organized to present the findings of the survey as percentages. For a specific question or sub-question, the data consist of the sum of the response frequencies weighted by the number of respondents answering that question to form percentages. The percentages are shown in the tables, charts, and text of this report. Where respondents had only a single choice, the sum of the response frequency equals 100% (small rounding variations can occur).¹⁷

To enhance the informational value from individual questions, the data also were broken down or organized by the association each respondent was affiliated with. Since each survey was printed with a unique control number that tracked affiliation membership, both survey distribution and association membership were controlled. The affiliation designator provided a convenient way to organize the data according to professional training and job responsibilities/activities. The same computational rules were followed for the questions where separate organizational affiliation data were shown.¹⁸ The tables and charts in this report provide the frequency of the response as a percent for the question. For the combined respondent sample, the label designator “Total All” is used in the charts. The chart labels for each of the four associations are as follow: AIHA, ASSE, ACOEM, and AAOHN.

A number of questions instructed the respondents to mark all answers from a supplied list that was applicable. Questions that allowed multiple responses produced a value that served as a measure of rank order.¹⁹ The percentages are computed by taking the sum of the selections (votes) accumulated for each item choice and dividing that number by the number of voters (i.e., respondents). An example is Q. #1 from the Survey found in Appendix II.

Data Quality Procedures

Section 515 of Public Law 106-554, known as the Data Quality Act, required the Office of Management and Budget to promulgate guidance to agencies ensuring the quality, objectivity, utility, and integrity of information (including statistical information) disseminated by Federal agencies. OMB’s government-wide guidelines, published as interim final on September 28, 2002 (66 F.R. 49718) and finalized on February 22, 2002 (67 F.R. 8452), can be found at the following website http://www.whitehouse.gov/omb/ All aspects of this study were conducted as to comply with the standards of the Data Quality Act.

Survey Procedure

The NIOSH Customer Satisfaction Survey (NCSS), consisting of 26 questions, was reviewed and approved for distribution by the Office of Management and Budget (OMB) for a one-year period beginning in March 2002 and ending on the 31^st March.²⁰ The printed survey was 11 pages in length including the cover and instructions. A number of the questions had multiple parts, required multiple responses, and provided opportunities for written comments. Depending on the respondent’s familiarity with NIOSH services and information, the survey could be completed in 10 to 25 minutes.

NIOSH mailed the survey in mid-January 2003 to a random sample of 300 members from each of the four organizations, providing a combined sample size of 1200. Participation was voluntary and all responses were anonymous. All participants were instructed to complete the first 11 questions of the survey. The initial 11questions were designed to gauge the type of work, experience, education, and organization affiliation of the respondents, i.e., the demographics of the respondents. Questions 9 and 10 sought information on the respondent’s preferred sources of occupational safety and health information. Question 11 asked specifically about NIOSH as a “source of information” for the respondent. If the respondent indicated that “he or she had never used or referred to a NIOSH publication,” or visited the NIOSH Web site, 800 #, or a NIOSH exhibit, the respondent was instructed to skip the remaining questions, turn to the back page of the survey, and follow the instructions to return the survey as completed in an enclosed self addressed, pre-paid government envelope.

Survey Research Findings

Mailed and E-mailed Survey Returns

At the end of the 10-week period that was allowed for the return of the forms, (03/31/2003), 688 completed forms were received. The 688 surveys represented a 57 % overall response rate of potential respondents. Although NIOSH projected a 60% response rate with a mailing of 1,200 surveys, the 688 surveys completed and returned was within the original targeted range of 660-700 surveys needed for statistical estimation.

Affiliation Response Rate

The data shown in Table 2 provide a summary of the population size, distribution, participation and affiliation rates for each association and the totals. The affiliation response rate was computed from the quotient of the number of surveys returned by the members of each association (e.g. 194 AIHA) and the total returned (688). AIHA had the highest participation rate with 64% and accounted for 28% of the sample respondents, whereas ACOEM participation rate was 49% and accounted for 21.5% of the total sample. Although not shown in Table 2, 603 or 88% returned the printed version by mail. The remaining 85 (12%) chose to complete the electronic survey “on line” and return it using the e-mail option.

Survey Facts	AIHA		ASSE		ACOEM		AAOHN	Totals
(a) Population Size	12,400	7,00021		6,500		13000		38,900
(b) Surveys mailed (randomly)	300		300		300	300		1200
(c) No. of surveys returned	194		180		148	166		688
(d) Participation rate (c/b)	64%		60%		49%	55%		57%
(e) Affiliation response rate (c/688)	28.2%		26.2%		21.5%	24.1%		100%

Table 2 Survey sample information by association

Section I: Background of Respondent

From a list of 14 safety and health professions, the respondents were asked to chose one or more professions that most closely matched their own(Q. #1). The results are shown in Table 3. The most frequently chosen professions were industrial hygienists (31%), safety professional/ manager (31%), nurse (25%), and physician (18%). The most common “other” category used the term environmental expert or engineer as a professional title. As expected the four most frequently identified professions were consistent with the professions represented by the four collaborating associations, i.e., physicians (ACOEM), nurses (AAOHN), industrial hygienist, (AIHA) and industrial engineers ASSE).

Table 3. (Q. #1) Self reported professional identities.

Percent	Occupation	Percent	Occupation
31%	Industrial hygienist	3%	Researcher
31%	Safety professional/manager	3%	Toxicologist
25%	Nurse	2%	Ergonomist/ work physiologist
18%	Physician	1%	Human resource specialist
12%	Environmental Engineer	-%	Economist
9%	Educator/trainer	–%	Physical therapist
7%	Risk/loss control manager	–%	Industrial psychologist

Table 4 shows that the majority of respondents were well educated, experienced, worked in large organizations, and had received some training in the past year. Seventy-one percent had at least one day of OSH training and the other 29% had a week or more of OSH training in the past year.

Table 4 (Qs. #2, 3, 4, and 6) Self-reported education, training hours received, years of experience, and size of organization.

Education (level)	Training (hrs/yr)	Experience (yrs)	Organization (size)
4% College	5% (none)	11% (0-5)	5% Self
8% Associate	15% (1-8)	17% (6-10)	11% <50
34% Bachelor	17% (9-16)	27% (11-15)	3% 50 – 100
33% Master	34% (17-40)	17% (16-20)	14% 101 -500
9% Doctoral	25% (41-120)	28% (>20)	67% > 500
12% Post Doc	4% > 3 weeks

Table 5 provides information on the types of job activities most commonly performed by the practitioners in the respondent sample. Some of the key activities include establishing safety and health programs, providing worker training, interpreting OSHA guidance, performing hazard assessments, and delivering medical support. Although not shown in the tables, the respondents were asked “what type of organization they had worked for” (Q. #5). The most frequently identified work setting was “company /business” at 43%, followed by a “hospital/clinic” setting at 19%. Private consultants were the third most frequently identified at 13%, and industry/trade associations at 10%.²² One question dealt with Internet access at work (Q. #7). As of the 1^st quarter of 2003, 97% of the respondents had access to the Internet at work.

Table 5 (Q. #8) Self reported primary professional responsibilities or activities.

	Professional Responsibilities		Professional Responsibilities
52%	Establish/conduct OSH programs	41%	Interpret/apply/enforce OSHA regs.
41%	Conduct worker OSH training	36%	Deliver medical services
20%	Conduct professional-level OSH training	25%	Keep records/report injury and disease cases
35%	Perform hazard assessments	6%	Develop/conduct research projects
25%	Perform accident /loss control analyses	37%	Conduct workplace safety audits and inspections
18%	Design/implement hazard control measures	15%	Perform air sampling/monitoring
31%	Formulate new/revised OSH policy/practices	2%	Design/lay out facilities to meet production/safety specifications

Section II: Respondent’s sources for OSH information

Printed, Electronic, and Others (Q. #9): Survey respondents were asked how often, as measured in terms of days, weeks, and months, etc. did they access printed, electronic, and/or other sources of OSH information. Printed sources included traditional books, journals, and technical reports. Electronic sources included the Web, CD-ROMs, and software, and the “other sources,” included colleagues, experts, and information from attending conferences/ personal meetings. This question was aimed at determining preferences and usage for accessing OSH information. The data are displayed in Table 6.

Table 6 (Q. #9) How often and what modes respondents use to find OSH information.

Printed Sources		Electronic Sources		Other Sources
34%	Daily	41%	Daily	22%	Daily
39%	Weekly	38%	Weekly	30%	Weekly
21%	Monthly	14%	Monthly	26%	Monthly
3%	Quarterly	3%	Quarterly	12%	Quarterly
0%	Bi-annually	1%	Bi-annually	4%	Bi-annually
1%	Yearly	1%	Yearly	4%	Yearly
1%	Never	2%	Never	2%	Never

Despite the computer-driven information revolution, printed materials continue to be an important source for providing OSH information.. Both printed and electronic information are accessed daily and weekly, each about 30% of the time or more. Meetings and colleagues also provide information not only on a daily basis, but also monthly and quarterly.

Sources of OSH information (Q. #10): Respondents were provided with a list of 23 organizations that serve as sources of OSH information. Question 10 asked whether they had “in the past 12 months read or referred to OSH information provided or published by any of the listed organizations or sources.” Since multiple responses were allowed, the data in Table 7 is best interpreted as a rank order score. The results show that federal agencies are important sources for finding OSH information. Specifically, OSHA was ranked first with an 84% score, NIOSH was second with 74%, and EPA at 44%. Professional associations also ranked high as sources of information.

Table 7 (Q. #10) Summary data showing where respondents look for OSH information.

	Association Sources			Association Sources
30%	AAOHN American Association of Occupational Health Nurses	3%		HFS Human Factors Society
45%	ACGIH American Conference of Governmental Industrial Hygienists	4%		HPS Health Physics Society
32%	ACOEM American College of Occupational and Environ Medicine	39%		NFPA National Fire Protection Agency
44%	AIHA American Industrial Hygiene Association	8%		NIEHS National Institute of Environmental Health Sciences
46%	ANSI American National Standards Institute	74%		NIOSH National Institute for Occupational Safety and Health
9%	APHA American Public Health Association	30%	NSC National Safety Council
40%	ASSE American Society of Safety Engineers	84%		OSHA Occupational Safety and Health Administration
22%	ATSDR Agency for Toxic Substances and Disease Registry	9%		NRC Nuclear Regulatory Commission
3%	CCPS Center for Chemical Process Safety	15%		Trade Organizations
11%	DOE Department of Energy	8%		Labor Organizations
44%	EPA Environmental Protection Agency	33%		Commercial newsletters, magazines, publications, etc. (Specify):

Chart 1 shows the responses by association for the two primary sources of OSH information. Respondents who were members of AIHA identified NIOSH and OSHA at nearly the same frequency (~90%), whereas respondents who belonged to ASSE, ACOEM, and AAOHN selected OSHA more frequently, followed closely by NIOSH. (Q. #10 (16&18)

Accessing NIOSH Information Question 11 was designed to separate practitioners who had used or referred to NIOSH publications from those who had never used NIOSH information products. The results indicated that 21% of the respondents had not used NIOSH information. Since those respondents would not be familiar with NIOSH’s information products, they were directed to stop and return the survey in the preaddressed envelope. Of the 680 responders, 141 (21%) responded no.” The remaining 539, (79%) indicated that “they had used or referred to NIOSH information,” i.e., printed publication, its web site, and 800-number, and therefore, were instructed to continue with the survey.²³

The data in Chart 2 show that the percentage of those that had used or referred to a NIOSH publication varied among the four stakeholder groups. Respondents who were members of AIHA and ASSE were much more likely to have used a NIOSH publication, 96% and 83% respectively, while members of AAOHN and ACOEM indicated lower usage, 67% and 66%, respectively.

Use of electronic and conventional access (Q. #12)²⁴

Survey participants who had “requested/received” NIOSH information identified the NIOSH Web site (78%) as their primary source for obtaining these materials; followed by the NIOSH 800-number (43%), NIOSH fax-line (19%) and NIOSH exhibit program (16%). For each information service, Chart 3 provides the relative percentages (or ranking) for all respondents and by association.

Use of NIOSH information in educational/training courses (Q. #13) This question was designed to assess the use of, or extent to which NIOSH publications are being used as informational materials in training and educational programs. Based on the response to this question, more than three out of four respondents (76%) reported attending a course or educational program where NIOSH materials had been used, 14% did not know and 10% answered no. Chart 4 separates the respondents according to their professional affiliation. The findings remain consistent across organizations.

Determining how frequently NIOSH publications and its web site are used.

Question 14 used a frequency scale with four levels. Each level included a numerical value to ensure consistency. The levels were defined as follows: frequently (>10), occasionally (3-10), rarely (<3) and never (0). The responses were further separated by association. The findings in Chart 5 and 6 show the percentages for all respondents and then by professional association. A majority of the respondents reported frequent and/or occasional use of both NIOSH’s printed publications and NIOSH’s Web site. Combining the percentages for “frequently” and “occasionally,” 68% of the respondents used both printed publications and the NIOSH web site at least three times a year.

Chart 7 (Q. #14d) provides a unique measure of the value of NIOSH publications. The respondents were asked how frequently had they “recommended a NIOSH publication … to a colleague.” Four choices of frequency were provide, including never. The premise is that documents or informational materials that are personally valued are more frequently shared with colleagues than a less useful publication. Approximately, 10% of the respondents recommended a publication or informational product more than ten times in the past year. About 76% of the respondents recommended a NIOSH product more than twice in the past year.²⁵ Chart 7 also shows that regardless of professional affiliation, the values on the frequency scale are relatively consistent.

Section III: NIOSH as a source for OSH information.

NIOSH as a source of information: (Q. #15) Question 15 consisted of eight statements about NIOSH as a source of occupational safety and health information. To assess these items, a five-point Likert rating scale was provided. The scale provided four levels of agreement (strongly agree, agree, disagree, strongly disagree) and a no-opinion option. The data in Table 8 shows that 99% of the survey participants who had used or referred to NIOSH materials either strongly agreed (71%) or agreed (28%) that “NIOSH is a credible source for obtaining OSH information” (1% had no opinion). Items 15b through 15 h registered lower percentages of agreement as well as low percentages of disagreement. The majority of respondents chose the no opinion option, which suggested that a large percentage of the respondents in this survey had not used or likely was not aware of the informational services identified in questions 15b through 15h.

Table 8 (Q # 15): Summary of respondents’ perceptions of NIOSH’s information delivery (SA- Strongly Agree; A-Agree; DA-Disagree; SD-Strongly Disagree, NO- No Opinion)

STATEMENTS	SA	A	DA	SD	NO
NIOSH is a credible source.	71%	28%	–%	–%	1%
NIOSH Listserv keeps me informed about NIOSH products/services26	4%	17%	2%	2%	75%
NIOSH Web site contains quality, usable information.	38%	49%	1%	–%	12%
The NIOSH Web site provides effective access.	32%	52%	2%	–%	14%
The NIOSH 800-number provides usable information.	8%	27%	1%	–%	64%
The NIOSH 800-number provides effective access	9%	28%	2%	1%	60%
The NIOSH exhibits at conferences keeps me informed	15%	42%	3%	1%	39%
NIOSH is an important resource for the OSH community.	65%	32%	–%	–%	3%

The data in Table 8 also showed general satisfaction with the NIOSH Web site. Eighty-seven percent of the survey participants who used or referred to NIOSH materials either strongly agree (38%) or agree (49%) with the statement that “the NIOSH Web site provides high-quality, useable information” (1% disagree” and 12% had no opinion) (Q. #15c). Similarly, 84% of the respondents either “strongly agree” (32%) or “agree” (52%) with the statement that “the NIOSH Web site provides effective access to NIOSH information and publications” (2% disagree and 14% had no opinion).

Chart 8 (Q.#15a) shows how the respondents, grouped by affiliation, rated the statement concerning NIOSH’s credibility. If you combine the percentages for agree and strongly agree, the level of support for the creditability statement was nearly 100%. For a research and service agency, the issue of creditability is one of the key factors needed in carrying out its mission.

-

Chart 9 (15h) shows how the respondents, grouped by affiliation, rated the statement concerning NIOSH being “an important resource for the occupational safety and health community.” Again, when the percentages for agree and strongly agree are combined, the level of support for the concept of NIOSH as a resource also was rated at nearly 100%. The values for AIHA were some what higher in the “strongly agree” category than corresponding values observed with the other three organizations (~60%). A possible reason is that over the years NIOSH has published hundreds of documents that are particularly relevant to the work of industrial hygienists.

Section IV: Opinions about NIOSH Publications

Ratings of NIOSH’s publications: (Q. #16a-k) The questionnaire contained a series of eleven statements about NIOSH publications. A five-point Likert rating scale was used to assess the respondents’ perceptions of NIOSH publications. The scale had four levels of agreement, namely: strongly agree, agree, disagree, strongly disagree, and a no-opinion option. Table 9 provides the level of agreement for each statement for all respondents, while Chart 10 provides the results in a bar graph. The majority of the respondents either strongly agreed or agreed with the survey statements that characterized NIOSH’s communication products as containing up-to-date information (92%), providing useful recommendations and guidance (92%) being clearly written (91%), at the appropriate technical level (92%), user –friendly (87%), providing practical recommendations (83%), “at the appropriate length for topic” (82%), and impartial (80%). These data support one of NIOSH’s primary responsibilities, which is to provide high quality technical publications to the OSH community.

Table 9 (Q. # 16): Summary of respondents’ perceptions of NIOSH publications.

(SA- Strongly Agree; A-Agree; DA-Disagree; SD-Strongly Disagree, No. No Opinion)

STATEMENTS		SA	A			DA	SD	NO
NIOSH publications contain current, up-to-date information.		30%	62%			3%	–%	5%
NIOSH publications are impartial.	21%			59%	6%		–%	14%
NIOSH publications are at the appropriate technical level.	24%			68%	2%		–%	6%
NIOSH publications are clearly written.	21%			70%	4%		–%	5%
NIOSH publications provide useful recommendations and guidance.	29%			63%	3%		–%	5%
NIOSH publications provide practical recommendations	19%			64%	7%		–%	10%

NIOSH Web site and the 800#

Two statements were used to obtain information related to the ease of accessing NIOSH publications through the NIOSH Web site and NIOSH’s 800-number (Q.16 h & k).²⁷ Seventy-two percent of the respondents either strongly agreed or agreed that “finding NIOSH publications on the NIOSH Web site is easy,” while 6% disagreed and 22% had no opinion. Significantly fewer respondents had an opinion about the ease of using the NIOSH 800-number publication ordering system; 65% of the respondents had no opinion. Of the remaining respondents, 31% agreed or strongly agreed that the 800-number ordering system was easy to use and 4% either disagreed or strongly disagreed.

Section V: Use of NIOSH Publications

How NIOSH materials are used: NIOSH continues to develop a variety of informational materials, often in multiple formats, electronic, printed, and CD-ROMs. These materials may range from a simple two-page Fact Sheet to an extensive, electronic chemical database, such as the NIOSH Pocket Guide to Chemical Hazards. Question 18 was designed to determine “if NIOSH materials were being used by the respondents” as a reference tool in performing various job duties, and, if so, to what extent (the frequency of usage). Table 10 provides a condensed view of the data, while providing some insight on the issue of usage and relevance of documents to OHS practitioners. This table shows that NIOSH materials are used frequently in job activities such as training, developing OSH programs, hazard exposure assessments, and formulating OHS policies.

Table 10 (Q. #18) Frequency of use of NIOSH publications in performing OHS jobs, based on past 12 months. (Greater than 10 times a year (>10), between 3 and 10 times, less than 3, 0-never, NA. Not Applicable).

OHS Functions	>10	3-10	<3	0	NA
Establish OHS programs	12%	35%	28%	8%	17%
Conduct OHS training	15%	36%	25%	8%	16%
Perform hazard exposure tests	15%	31%	25%	9%	20%
Implement hazard controls	9%	30%	24%	10%	27%
Formulate OHS policy/practices	11%	35%	27%	10%	17%
Conduct research projects	7%	14%	14%	15%	50%
Conduct workplace inspections	8%	24%	28%	17%	23%
Perform air sampling/monitoring	14%	20%	16%1	12%	38%
Prepare reports for management	8%	21%	29%	16%	26%

Chart 11 (Q. #18) provides a view of the data using a simple horizontal bar graph. A couple of findings stand out. First, most of these practitioners are less interested in scholarly documents, particularly those related to research activities. With respect to the other four job activities, there was a mix of usage which may account for the range of frequency seen below. NIOSH publications are frequently use or referred to when practitioners are involved in hazard assessment and training.

Impact of NIOSH on policies and procedures. One of the key measures of effectiveness is to examine the extent to which an organization (a workplace) has used, referred to, or adopted guidance and recommendations produced by a non-rule making agency, such as NIOSH.

Chart 12 shows the responses by association for Q. # 19. There was remarkable consistency across all four organizations with respect to this question. For all the respondents, slightly less than 50% indicated that NIOSH publications had been referred to or cited by name in their organizations’ policy and procedures, whereas 30% indicated that they had not been cited, 17% did not know, and 4% indicated the question did not apply.

Question 20 (Q. #20) asked “to what extent your organization’s safety and health practices (had) been influenced by NIOSH publications,” 36% responded a lot, 44% a little, 17% did not know; 3% marked no. (Chart 13 shows the responses by association). Again, if you combine the top two choices, “a little” and “a lot,” the results are encouraging, ranging from about 70% to 85%. This question provides yet another marker for performance and can be used as a baseline or used to set a goal.

Section VI: Suggestions for NIOSH Products and Services Improving NIOSH information dissemination:

The survey form provided a list of 13 statements, suggesting ways for NIOSH to improve its products and information delivery (Q. #21). The statements were selected from the comments made by the members of the original focus groups who helped develop the content for this survey. Each survey statement included a five point Likert scale. The scale ranged from one (1), a high value through five (5) a low value. The respondents were asked to indicate a value for each suggested improvement. Table 11 shows values for Q. #21

Table 11 (Q. #21) provides some useful ideas for the dissemination of information. If the percentages for values 1 and 2 are combined, statement “h” is ranked first with the suggestion of “Package/distribute NIOSH publications around common themes (83%).”²⁸ This is followed by the Statement “c,” which is to “Announce publications on the NIOSH Web site (81%).” Creating CD-ROM collections of publications was third with 69%. Publicizing and increasing the circulation of Health Hazard Evaluations Reports was fourth at 65%. Chart 14 provides a graphic view of the respondents’ preferences for improving NIOSH’s dissemination.

Table 11 (Q #21) Value judgments of respondents regarding specific suggestions

STATEMENTS	1 High		2			3	4	5 Low
a. Produce more NIOSH catalogues that describe publications and reports.	25%		25%			26%	12%	12%
b. Announce publications on NIOSH Listserv.	24%		26%			26%	10%	14%
c. Announce publications on NIOSH Web site.	49%		32%			14%	3%	2%
d. Promote availability of NIOSHTIC-2 database for search/access of its reports.	25%		31%			24%	10%	10%
e. Establish reader interest profiles and send publications that match reader profiles.	17%		27%			26%	14%	16%
f. Publicize and increase the circulation of NIOSH Health Hazard Evaluation Reports.	24%		41%			23%	8%	4%
g. Produce more Investigative reports	18%		36%			31%	10%	5%
h. Package/distribute publications around common themes.		41%		42%	12%		3%	2%
i. Increase use of direct mail ticklers.		10%		20%	28%		20%	22%
j. Develop technical documents understandable to workers.		21%		30%	29%		13%	7%
k. Support chat room on Web for OSH		8%		17%	27%		26%	22%
l. Create CD-ROM collections of publications		36%		33%	17%		7%	7%
m. Make publications in different languages.		11%		14%	21%		16%	38%

,

Table 12 (Q. #22) provides information on the preferences of the respondents for being notified regarding new NIOSH publications. Clearly, the NIOSH website received the highest preference (88%), followed by the NIOSH publications catalog (63%).²⁹. This suggests that the respondents in this survey understand and use the NIOSH Web site to seek out information on new publications. Their preference for receiving NIOSH publications by “direct mail” was considerably lower (46%) and matched the score for the NIOSH Listserv. (46%).³⁰

Table 12 (Q. #22) Preferred way of being notified about sources of information on new publications.

STATEMENTS	1 High	2	3	4	5 Low
NIOSH Listserv	20%	26%	24%	9%	21%
Statements	High 1	2	3	4	Low 5
NIOSH Web site	67%	21%	7%	3%	2%
NIOSH 800-number	12%	23%	26%	19%	20%
NIOSH Conferences/exhibits	21%	31%	26%	13%	9%
NIOSH Direct mailing	18%	28%	22%	14%	18%
NIOSH Publications catalog	30%	33%	20%	10%	7%

Although the NIOSH 800-number was not selected as a high preference source for learning about new NIOSH information (35%), the 800-number remains popular as a source to reach NIOSH for not only questions about new publications, but also to receive publications and request information on issues related to OSH.³¹

Table 13 (Q. #23) provides information on various sources for obtaining information. Again, the NIOSH Web site received the highest ratings for accessing NIOSH information, whether by e-mail or downloaded from the NIOSH Web site. Since 97% of these professionals had access to the Internet at work (Q. #7), the Internet provides access at any point in time and from any location. The other choices would perhaps be more limiting as to time and place.

Table 13 (Q. #23) Preferences for learning about new OSH information.

STATEMENTS	1 High	2	3	4	5 Low
Request via Web site e-mail	63%	24%	8%	2%	3%
Download from NIOSH Web site	74%	16%	7%	1%	2%
Call NIOSH 800-number	14%	28%	26%	14%	18%
Telephone NIOSH authors/staff	9%	17%	28%	22%	24%
Request via NIOSH FAX line	7%	20%	30%	20%	23%
Request at NIOSH conferences or exhibits	13%	26%	29%	18%	14%

Table 14 (Q. #24) was an open-ended question in which the respondents were asked to “list three new safety and health topics that NIOSH should address in future publications.” The responses were organized/catalogued by subject matter. The results are shown below for the six most popular topics in order of frequency.

New Topic	New Topic
1. Mold Containment	5.Counter Terrorism
2. Indoor Air Quality	6. Safety-related Topics
3. Ergonomics-related	7. Health Effects topics
4. Respirator Selection and Use	8. Smallpox Protection

Table 15 (Q. #25) provides a list of seven types of publications (information products) that the original focus group participants suggested be included in the survey. The rank score reflects the choices of the survey respondents when asked to select as many of the types of publications (information products) that “NIOSH should consider emphasizing in the future.” Publications dealing with practical applications and guidelines generated the highest ranked score. There also continues to be a demand for publications dealing with recommended safety and health standards. Materials for educating workers and owner/managers were also popular. In general with OSH practitioners, such as the survey respondents, there appears to be less interested in technical or scientific types of documents.

Table 15 (Q. #25) Types of publications selected by survey respondents.

Publication Types	Rank %
Practical applications/guides (checklists, self audits, “how to” publications	81%
Recommended safety and health standards	72%
Booklets or documents for educating workers	67%
Booklets or documents for educating owners/managers	55%
Case studies (Health Hazard Evaluations & FACE)	51%
Scientific/technical documents	41%
Manual of Analytical Methods	35%

Discussion-Conclusion

The survey respondents, in general, were well educated and experienced in their respective fields of expertise. The four most common occupations were industrial hygiene, safety professional/manager, occupational nurse, and occupational physician. The majority of them had attended an OSH training course in the past year where NIOSH materials were used or provided as part of the training. Most of the respondents worked in large companies or businesses and performed a range of jobs that included establishing and conducting OSH programs, interpreting and applying OSHA regulations, conducting training and workplace safety audits and inspections. The sample had near universal access to the Internet at work. As a result, the majority of respondents strongly preferred to access the NIOSH Web site to learn about new publications rather than receive notification of new NIOSH publications through direct the mail.

The survey respondents typically looked to government sources for information on occupational safety and health. NIOSH, along with OSHA were the two main sources consulted on a regular basis. Other highly ranked sources included ACGIH, and ANSI. One surprise was the popularity of National Fire Protection Agency as a source for OSH information. Various commercial newsletters, magazines, and publications were also identified as sources for OSH information. The respondent’s also identified their own professional associations as important sources of information.

These findings serve as a useful guide for developing information and communication products with stakeholder associations such as AIHA, ASSE, ACOEM, and AAOHN. It is recommended that NIOSH be more proactive in developing partnerships with these associations. Some steps may include increasing the sharing of information through the use of Web site cross-promotions and disseminating information through various associations and commercial publications. There also may be opportunities prior to the national associations meetings to collaborate with the associations and sponsor exhibits promoting common areas of interest.

The findings from the Customer Survey indicated that overall NIOSH is meeting the information needs of the majority of OSH professionals. Nearly 80% of the respondents had used NIOSH materials in some form, and 80% of those believed that NIOSH informational materials had influenced their organization’s safety and health practices. Forty-nine percent indicated that NIOSH publications had been “referred to or cited by name” in their “organizations’ policy and procedures.”

About a third of the respondents reported that NIOSH’s publications had a lot of influence on their organization. The remaining two-thirds were split between a little and don’t know. Nearly fifty percent of the sample was certain that NIOSH materials had been used or cited by name in their organization’s policy and procedures. This was consistent across professions.

The respondents also gave NIOSH nearly a 100% grade for “credibility.” Moreover, when the respondents were asked to express their degree of agreement with the statement that “NIOSH is an important resource for the occupational safety and health community,” 97% strongly agreed or agreed with the statement. Additionally, NIOSH received strong support from the respondents for its publications being up-to-date, impartial, clearly written, at the appropriate technical level, as well as providing useful and practical information.

There are, however, opportunities for improvement. Specifically, 21% of the respondents had not used or ever referred to NIOSH materials. When the data were analyzed by association membership, this effect was most noted among survey respondents who belonged to the two medical associations (ACOEM and AAOHN).

These finding may reflect either a lack of awareness of NIOSH’s information or a lack of awareness of NIOSH as the source of occupational safety and health information. Either way, this suggests a need for improved communication, dissemination and/or attribution. This finding presents opportunities for more direct knowledge sharing and interaction with the four stakeholder associations.

The survey provided a list of nine main tasks. The respondents were then asked how frequently, during the last 12 months, had they used or referred to NIOSH publications in performing those tasks. Only a small percentage of the respondents, usually less than 15%, refer to, or use NIOSH materials on a frequent (>10) basis in performing any of the nine OSH functions. Second, a comparable percentage during the same 12 month time period indicated that they had never used NIOSH publications in performing the identified job functions. In fairness, a number of the listed functions/tasks were more suited to the types of activities that industrial hygienists (IH) normally perform. Since only 25% of the sample constituted IHs, it is apparent that those unique IH functions, such as air sampling and work place inspections would not show high percentages. Since the majority of respondents in this survey were practitioners in contrast to researchers, it is not surprising that they were less interested in research oriented documents. However, NIOSH materials are used/referred to frequently in performing job activities such as training, developing OSH programs, hazard exposure assessments, and formulating OHS policies. Finally, the data suggest that NIOSH may want to focus on developing informational materials that assist practitioners in performing their tasks. The fact that NIOSH materials for a number of OHS activities were “rarely or never consulted” should be further investigated to determine the reason(s) behind this finding.

The NIOSH Customer Survey also identified customer preferences for future information products. For example, respondents were supportive of information that provided practical and clear guidance on matters related to policy, such as the “NIOSH Criteria Documents.” The survey also revealed that there is a demand for information and tools designed and packaged for more direct application, “checklist, how-to-do,” and self audits. In response, NIOSH has initiated a program geared to the translation of NIOSH research into practical, more applied documents, which lend themselves to improved implementation in the workplace. This new program is called “research to practice” or r2p. Effective work practices may cover the range of activities from perception to impact.

The r2p initiative recognizes the vast amount of technical information that NIOSH has produced over the years and the fact that this information has not been accessible, or summarized in a fashion that it can be used at the workplace or in training sessions. NIOSH is taking a very proactive stance to ensure that research efforts, where feasible, can be translated into materials or booklets and documents for workers, owners, and managers. Similarly, case study information is seen as a valuable resource. More than 2,500 HETA investigations and a similar number of fatality reports and control technology reports have been collected and published around common themes or hazards. Many of these case reports have relevance to situations where similar worksite hazards and prevention opportunities exist, and are associated with work practices that promote healthy work.

Future Actions

The survey results also suggest that additional marketing of eNews and r2p to the associations represented by these respondents would be worthwhile. NIOSH also will expand its efforts to meet the information needs of its other customers. Most notably, given the interest expressed for more communication and outreach (conferences, symposia, and training). NIOSH could hold regional stakeholder meetings over the next few years to solicit “research -translation needs” (r2p) from small business, the health care sectors, and others, as needed. The meetings could be geared to different industries to examine the patterns of hazards in the key industries and to develop a more cohesive approach to prevention. Information gathered at these meeting could be used to direct the development of research designed for practitioners faced with the responsibilities of ensuring a safe and healthful workplace.

The survey findings also indicate that NIOSH needs to reconsider its dissemination plans for its primary publications. Over the years, various mailing lists have been compiled based on user requests. New publications are routinely sent to all members on the mailing lists. More recently NIOSH had been purchasing specific mailing lists for special purpose publications, such as those aimed at firefighters. Competition for the attention of busy OSH practitioners is even more difficult now with the Internet. NIOSH needs to find ways to market the value of its documents to the practitioners. These individuals need information that will assist them to do their job easier and better. This message needs to be clear and out in front. Credible information that enhances the effectiveness of OSH practitioners will be well received, but targeted groups must be made aware of NIOSH’s products. This is more likely to occur if the targeted or user groups are brought into the document development process at the beginning stages.

NIOSH will also continue efforts to improve communication with stakeholder associations. The NIOSH Web site has added features requested by customers, notably topic pages highlighting recent advances in surveillance, evaluation, and control of occupational hazards. Future changes will be made including the capability to search the NIOSH Web site for case studies of effective controls and interventions.

Appendix: I

Response Summary Questionnaire

SECTION I

BACKGROUND—TELL US ABOUT YOURSELF

The following questions ask about your professional training and background.

Please fill in the circle next to your response completely: , NOT or

1. How would you define your professional training/background? (Fill in ALL circles that apply.)

18% - Physician	3% - Researcher
31% - Safety professional/manager	–% - Physical therapist
3% - Toxicologist	31% - Industrial hygienist
7% - Risk/loss control manager	–% - Industrial psychologist
2% - Ergonomist/work physiologist	1% - Human resource specialist
25% - Nurse	9% - Educator/trainer
–% - Economist	12% - Other (Specify):__________________

2. What is the highest educational level that you have attained? (Fill in ONE circle.)

4% - Some college

8% - Associate degree

34% - Bachelor’s degree

33% - Master’s degree

9% - Doctoral degree

12% - Post-doctoral studies or training

3. In the past 12 months, how many hours/days/weeks did you spend attending training courses dealing with occupational safety and health issues? (Fill in ONE circle.)

5% - None

15% - 1–8 hours

17% - 9–16 hours

34% - 17–40 hours

25% - 41–120 hours

4% - More than 3 weeks

4. How much professional experience do you have in occupational safety and health matters?
   (Fill in ONE circle.)

11% - 0–5 years
17% - 6–10 years
27% - 11-15 years

17% - 16 – 20 years

28% - than 20 years

5. In your current job, what type of organization do you work for?
   (Fill in ALL circles that apply.)

43% - Company/business

7% - University/college

–% - Labor organization

10% - Industry or trade association

19% - Hospital/clinic

13% - Private consultant

1% - Self-employed health service worker

–% - Professional society staff

9% - Federal government agency

4% - State government agency

2% - Local government agency

7% - Other (Specify): ________________________________________________________

6. In your current job, how would you classify the size of the organization in which you work?
   (Fill in ONE circle.)

5% - Self only

11% - Very small (less than 50 employees)

3% - Small (50 to 100 employees)

14% - Medium (101 to 500 employees)

67% - Large (more than 500 employees)

–% - Don’t know

6. In your current job, do you have access to the Internet?

97% - Yes

3% - No

6. In your current job, which of the following activities best represent your primary responsibilities? (Fill in up to FIVE circles.)

52% - Establish/conduct occupational safety and health programs
41% - Conduct worker occupational safety and health training
20% - Conduct professional-level occupational safety and health training
35% - Perform hazard exposure assessments
25% - Perform accident investigations/loss control analyses
18% - Design/implement hazard control measures
31% - Formulate new/revised occupational safety and health policy/practices
41% - Interpret/apply/enforce OSHA regulations
36% - Deliver medical services
25% - Keep records/report injury and disease cases
6% - Develop/conduct research projects
37% - Conduct workplace safety audits and inspections
15% - Perform air sampling/monitoring
2% - Design/lay out facilities to meet production/safety specifications
9% - Other (Specify): _______________________________________________________

SECTION II

YOUR SOURCES FOR OCCUPATIONAL SAFETY AND HEALTH INFORMATION

6. In your current job, how often do you use the following sources of occupational safety and health information? (Fill in ONE circle for EACH source.)

Printed Sources (i.e., books, journals, tech reports) 34% - Daily 39% - Weekly 21% - Monthly 3% - Quarterly - % - Bi-annually 1% - Yearly 1% - Never

Electronic Sources (i.e., Web search, CD-ROMs, software) 41% - Daily 38% - Weekly 14% - Monthly 3% - Quarterly 1% - Bi-annually 1% - Yearly 2% - Never

Other Sources (i.e., colleagues, experts, meetings, conferences) 22% - Daily 30% - Weekly 26% - Monthly 12% - Quarterly 4% - Bi-annually 4% - Yearly 2% - Never

6. In the PAST 12 MONTHS, have you read or referred to occupational safety and health information provided or published by any of the following organizations or sources? (Fill in ALL circles that apply.)

30% - AAOHN American Association of Occupational Health Nurses
45% - ACGIH American Conference of Governmental Industrial Hygienists
32% - ACOEM American College of Occupational and Environmental Medicine
44% - AIHA American Industrial Hygiene Association
46% - ANSI American National Standards Institute
9% - APHA American Public Health Association
40% - ASSE American Society of Safety Engineers
22% - ATSDR Agency for Toxic Substances and Disease Registry
3% - CCPS Center for Chemical Process Safety
11% - DOE Department of Energy
44% - EPA Environmental Protection Agency
3% - HFS Human Factors Society
4% - HPS Health Physics Society
39% - NFPA National Fire Protection Agency
8% - NIEHS National Institute of Environmental Health Sciences
74% - NIOSH National Institute for Occupational Safety and Health
30% - NSC National Safety Council
84% - OSHA Occupational Safety and Health Administration
9% - NRC Nuclear Regulatory Commission
15% - Trade Organizations
8% - Labor Organizations
33% - Commercial newsletters, magazines, publications, etc. (Specify):

11% - Other (Specify):

6. Have you ever USED or REFERRED to a NIOSH publication (i.e., printed NIOSH material, NIOSH CD-ROM, NIOSH Web site material, or information from the 800-number)?

79% - Yes (Please Continue to Question 12)

21% - No (Please STOP here. Go to the back page of the questionnaire and follow the instructions for sending the questionnaire back to NIOSH.)

6. Have you ever requested/received occupational safety and health information from NIOSH by any of the following sources? (Fill in ALL circles that apply.)

43% - NIOSH 800-number 19% - NIOSH FAX Line 78% - NIOSH Web site 16% - NIOSH exhibit program 13% - NIOSH staff expert 6% - NIOSH Listserv 15% - NIOSH sponsored conferences 5% - Other_____________________________________________________________

6. Have you ever taken a course or attended an educational program in which NIOSH publications were used as informational materials? (Fill in ONE circle.)

76% - Yes

10% - No

14% - Don’t know

–% - Does not apply

14. Please indicate the frequency that you performed each of the following tasks within the PAST 12 MONTHS. (Fill in the circle of your answer for EACH item below.)

STATEMENTS	Frequently >10 Times	Occasionally 3–10 Times	Rarely <3 Times	Never 0 Times
Used NIOSH printed publications to find occupational safety and health information.	24%	44%	27%	5%
Used the NIOSH Web site to find occupational safety and health information.	32%	36%	25%	7%
Used the NIOSH 800-number to find occupational safety and health information.	1%	9%	30%	60%
Recommended a NIOSH publication, CD-ROM, or material from the NIOSH Web site to a colleague.	10%	29%	37%	24%

SECTION III

YOUR OPINION ABOUT NIOSH AS AN INFORMATION SOURCE

15. Please indicate how strongly you agree or disagree with each of the following statements.
    (Fill in the circle of your answer for EACH item below.)

STATEMENTS	Strongly Agree	Agree	Disagree	Strongly Disagree	No Opinion
NIOSH is a credible source for obtaining occupational safety and health information.	71%	28%	–%	–%	1%
The NIOSH Listserv keeps me informed about its products and services.	4%	17%	2%	2%	75%
The NIOSH Web site contains high-quality, usable information.	38%	49%	1%	–%	12%
The NIOSH Web site provides effective access to NIOSH information and publications.	32%	52%	2%	–%	14%
The NIOSH 800-number provides high quality, usable information.	8%	27%	1%	–%	64%
The NIOSH 800-number provides effective access to NIOSH information and publications.	9%	28%	2%	1%	60%
The NIOSH exhibits at conferences keeps me informed about new NIOSH publications and services.	15%	42%	3%	1%	39%
NIOSH is an important resource for the occupational safety and health community.	65%	32%	–%	–%	3%

SECTION IV

YOUR OPINIONS ABOUT NIOSH PUBLICATIONS

15. Please indicate how strongly you agree or disagree with each of the following statements about NIOSH publications. (Fill in the circle of your answer for EACH item below.)

    STATEMENTS	Strongly Agree	Agree	Disagree	Strongly Disagree	No Opinion
    NIOSH publications contain current, up-to-date information.	30%	62%	3%	–%	5%
    NIOSH publications are impartial.	21%	59%	6%	–%	14%
    NIOSH publications are at the appropriate technical level.	24%	68%	2%	–%	6%
    NIOSH publications are clearly written.	21%	70%	4%	–%	5%
    NIOSH publications provide useful recommendations and guidance.	29%	63%	3%	–%	5%
    NIOSH publications provide recommendations that are practical.	19%	64%	7%	–%	10%
    The length of NIOSH publications is appropriate for coverage of the topic.	17%	65%	5%	–%	13%
    The NIOSH 800-number publication ordering system is easy to use.	8%	23%	3%	1%	65%
    NIOSH publications are user-friendly.	17%	70%	4%	1%	8%
    NIOSH publications are delivered in a timely manner.	15%	47%	6%	1%	31%
    Finding NIOSH publications on the current NIOSH Web site is easy.	17%	55%	6%	–%	22%

16. Based on your past use of NIOSH publications, which publications did you find most useful (List up to three publications). Please provide a short explanation of why you found them useful. (For the following questions, please refer to the enclosed List of NIOSH Publications.)

a.

b.

c.

SECTION V

YOUR USE OF NIOSH PUBLICATIONS

15. Please indicate how often you referred to NIOSH publications (printed documents, CD-ROMS, or information on the NIOSH Web site) during the PAST 12 MONTHS when performing each of the following tasks. (Fill in the circle of your answer for EACH item below.)

TASKS I refer to NIOSH publications when	Frequently >10 Times	Occasionally 3–10 Times	Rarely <3 Times	Never 0 Times	Not Applicable
Establishing occupational safety and health programs.	12%	35%	28%	8%	17%
Conducting occupational safety and health training.	15%	36%	25%	8%	16%
Performing hazard exposure assessments.	15%	31%	25%	9%	20%
Designing/implementing hazard control measures.	9%	30%	24%	10%	27%
Formulating new/revised safety and health policy/practices.	11%	35%	27%	10%	17%
Developing/conducting research projects.	7%	14%	14%	15%	50%
Conducting workplace safety audits and inspections.	8%	24%	28%	17%	23%
Performing air sampling or monitoring.	14%	20%	16%	12%	38%
Preparing reports for management.	8%	21%	29%	16%	26%

The following questions pertain to your organization’s use of NIOSH publications.

19. Has a NIOSH publication been referred to or cited by name in your organization’s policy and procedures?

49% - Yes

30% - No

17% - Don’t know

4% - Does not apply—no written occupational safety and health policy and procedures

19. To what extent have your organization’s safety and health practices been influenced by NIOSH publications?

36% - A lot

44% - A little

3% - Not at all

17% - Don’t know

SECTION VI

MARKETING OF NIOSH PRODUCTS AND SERVICES

19. Listed below are suggested ways for improving NIOSH dissemination of information. Please indicate the value to YOU on a scale of 1 to 5 for each suggested improvement where “1” indicates a high value to you and “5” indicates a low value. (Fill in the circle of your answer for EACH item below.)

STATEMENTS	1 High Value	2	3	4	5 Low Value
Produce more NIOSH catalogues that describe publications and reports.	25%	25%	26%	12%	12%
Announce publications on NIOSH Listserv.	24%	26%	26%	10%	14%
Announce publications on NIOSH Web site.	49%	32%	14%	3%	2%
Promote availability of NIOSHTIC-2 database for search/access of its reports.	25%	31%	24%	10%	10%
Establish reader interest profiles and send publications that match reader profiles.	17%	27%	26%	14%	16%
Publicize and increase the circulation of NIOSH Health Hazard Evaluation Reports.	24%	41%	23%	8%	4%
Produce more Technology Reports and other investigative reports.	18%	36%	31%	10%	5%
Package and distribute NIOSH publications around common themes (e.g., noise and hearing loss, ergonomics, agriculture).	41%	42%	12%	3%	2%
Increase use of direct mail ticklers.	10%	20%	28%	20%	22%
Develop versions of technical documents more understandable to workers.	21%	30%	29%	13%	7%
Support a chat room on the NIOSH Web site to discuss current issues.	8%	17%	27%	26%	22%
Create CD-ROM collections of publications.	36%	33%	17%	7%	7%
Make publications available in different languages. (Specify languages):	11%	14%	21%	16%	38%
Other (Specify):	–%	–%	–%	–%	–%

22. Listed below are various methods of notifying the public about new NIOSH publications. Please indicate YOUR preference on a scale of 1 to 5 for each notification method where “1” indicates a high preference for that method and “5” indicates a low preference. (Fill in the circle of your answer for EACH item below.)

STATEMENTS	1 High Preference	2	3	4	5 Low Preference
NIOSH Listserv	20%	26%	24%	9%	21%
NIOSH Web site	67%	21%	7%	3%	2%
NIOSH 800-number	12%	23%	26%	19%	20%
NIOSH conferences/exhibits	21%	31%	26%	13%	9%
NIOSH Direct mailing	18%	28%	22%	14%	18%
NIOSH Publication catalog	30%	33%	20%	10%	7%

22. Listed below are various methods of obtaining NIOSH publications. Please indicate YOUR preference on a scale of 1 to 5 for each method of obtaining NIOSH publications where “1” indicates a high preference for that method and “5” indicates a low preference. (Fill in the circle of your answer for EACH item below.)

STATEMENTS	1 High Preference	2	3	4	5 Low Preference
Request via Web site e-mail	63%	24%	8%	2%	3%
Download from NIOSH Web site	74%	16%	7%	1%	2%
Call NIOSH 800-number	14%	28%	26%	14%	18%
Telephone NIOSH authors/staff	9%	17%	28%	22%	24%
Request via NIOSH FAX line	7%	20%	30%	20%	23%
Request at NIOSH conferences or exhibits	13%	26%	29%	18%	14%
Other (Specify below):_____	–%	–%	–%	–%	–%

22. Please list THREE new safety and health topics that NIOSH should address in future publications.

a.
b.
c.

22. Which of the following types of publications should NIOSH consider emphasizing in the future? (Fill in ALL circles that apply.)

72% - Recommended safety and health standards (criteria documents)

67% - Booklets or documents for educating and informing workers

55% - Booklets or documents for educating and informing owners and managers

51% - Case studies (Health Hazard Evaluations and fatality investigations)

41% - Scientific/technical documents

35% - Manual of Analytical Methods

81% - Practical applications and guides (checklists, self-audits, "how-to" publications)

22. Would you consider it reasonable for NIOSH to charge a nominal shipping and handling fee (e.g., $5) for publication orders of 5 or more copies?

82% - Yes

9% - No

9% - No opinion

WE CARE ABOUT WHAT YOU THINK. Please use this space to provide us any additional comments you have about this survey or about the services and products provided by NIOSH.

THANK YOU FOR COMPLETING THE QUESTIONNAIRE!

Your help is greatly appreciated. Please take the following steps:

• Place your questionnaire in the pre-paid envelope provided and seal it.

• Fill out the enclosed Survey Response Card to identify the five NIOSH publications you wish to receive from the booklet List of NIOSH Publications.

• Mail the questionnaire and Response Card separately to assure your anonymity.

Reference Section

Alreck, P.L., Settle, R.B., The Survey Research Handbook, 2^nd ed.,1994, McGraw Hill, NY.NY

Cochran, W.G., Sampling Techniques , 3^rd ed., 1997, John Wiley & Sons, NY.,N.Y.

Dillmann, D.A., Mail and Internet Surveys, 2^nd ed., 2000 John Wiley & Sons, NY.NY.

Hayes, B.E., Measuring Customer Satisfaction, 1997, ASQ Quality Press, Milwaukee, WI.

Rea, L.M., Parker, R.A., Designing and Conducting Survey Research, 2^nd.1997, Jossey-Bass, San Francisco, CA.

Fink, A,. How to Analyze Survey Data, 1995, Sage Publications, London, England.

1 NIOSH has funded an additional 28,000 reports and journal articles, nearly 1,000 a year, all of which are indexed at http://www2a.cdc.gov/nioshtic-2/Nioshtic2.htm

2 Institute for Policy Research, University of Cincinnati, One Edwards Center, Cincinnati, Ohio

3 Appendix I includes a copy of the NIOSH Questionnaire and the survey responses.

4 For this Survey, “NIOSH publications” were defined as any combination of NIOSH printed materials, NIOSH CD-ROMs, NIOSH Web site materials, or information from the NIOSH 800-inquiry number.

5 Subsequent data citing “Survey Participants” refer to those 539 respondents who had answered “yes” to Q. 11.

6 Since multiple responses were allowed, the percentages shown are relative rankings, not absolute measures.

7 Q #14d asked about the “frequency” of recommending a NIOSH publications where the choice of “rarely” was defined as (<3). Hence, 76% is the sum of the percents associated with the three frequency labels: frequently 10%), occasionally (29%), and rarely (37%).

8 Where “frequently” is > 10 times per year, “occasionally” is 3-10 per year, and “rarely” is <3 per year.

9 With a Likert scale, the common practice is to combine the top two values to describe the results.

10 Although the current inventory consists of about 4,500 titles, NIOSH has produced more than 5,000 titles that bear the Institute name.

11 Homogeneity describes populations that are not 50/50 splits for the study characteristic.

12 With homogenous populations, sample sizes may be lower because internal variances are lower (Dillman, 2000).

13 In 1976, NIOSH contracted with John Short & Associates to develop a NIOSH information dissemination strategy. They developed an eleven-item questionnaire, which was sent to all NIOSH mailing list addresses. The findings are contained in a report entitled “NIOSH Information Dissemination Strategy.”

14 Distribution size is the number of survey packets to be mailed.

15 With a sample size of 300, fewer than 5% of the members from association would receive a survey packet.

16 Whereas a stratified sample, based on size of the associations would have optimized projections to the total 40,000 members of the combined four associations. Hence a high response rate from a large organization could skew the findings towards the views of the larger organization.

17 Percentages were computed from the sum of positive responses to an item divided by the total # of responses to the item, i.e., # of respondents who chose to answer the question. For example, Q #7 asked about access to the Internet in their job, 659 of the 677 respondents who chose to answer that question, answered “yes,” yielding a “97.3%” and 18 answered “no” for a “2.66%.” Ten respondents did not answer the question. Hence, the participation rate for Q #7 was 98.4%.

18 The findings are displayed in a series of charts and tables in the results section of this report.

19 The sum of the percents for the questionnaire items will typically exceed 100% because each person can cast one vote for each choice that the respondent chooses for that question.

20 OMB No. 0920-0544. To comply with the OMB access initiative, a web-based version of the questionnaire was also available to all survey participants. Appendix II provides a copy of the survey with Summary Responses.

21 The sample from ASSE consisted of the 7.000 Certified Safety Professionals

22 The percentages exceed 100% because respondents were allowed to identify all work settings where they were employed.

23 If the respondents were not familiar with NIOSH information products, they would not have been able to respond to the subsequent questions, which asked about the quality and delivery of NIOSH informational materials.

24 The responses from the remaining questions are from the 79% who responded ‘Yes’ to Q#11.)

25 To compute that value of 76% one would sum the percentages from rarely, frequently, and occasionally, under the all category. This will yield a value of 76%. (Rarely was described as “less than 3 times.”)

26 The NIOSH Listserv preceded the NIOSH eNews and was a first attempt at outreach, which may account for the 75% with “no opinion. The eNews has more than 18,000 subscribers and was launched a month after this survey was completed.

27 Refer to Appendix 1, Q. #16 of the survey for the data on “h & k.”

28 This practice of combining the values of the top two choices from a five point Likert scale is referred to in the marketing survey literature as the “top-two-box” finding.

29 This is based on the “top two box” score.

30 The NIOSH Listserv had been in operation for less than a year and preceded the launch of the current more contemporary NIOSH e-News.

31 The 800-number receives about 80,000 calls a year and disseminates over 900,00 information products per year.

Confidential Page 25