Supptg. stmt. part B

B. Collection of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

1.1 Background

The objective of this study is to develop a baseline of response percentages associated with health and safety information for business-oriented trade associations, professional-oriented trade associations, and labor unions. Percents are required separately for each of eight NIOSH NORA industrial sectors. Percents of interest are also to be obtained for three classes of organizations as a group: (1) business-oriented trade associations, (2) professional-oriented trade associations, and (3) labor unions. In addition, percentages are to be determined for each the three classes of organizations within the eight sectors of business. Examples of desired results are the percentage of organizations within a class or sector that use NIOSH occupational safety and health publications, cite these publications, and agree that these publications are useful and credible.

1.2 Respondent Universe

Currently in the Associations Unlimited Database there are 8342 organizations in the categories of interest, including 3234 business-oriented trade associations, 4819 professional-oriented trade associations, and 289 labor unions. The number of organizations within each class/sector varies greatly, ranging from as few as 10 to as many as 2773. Data will be collected one business or industrial sector at a time beginning with the construction sector, and the data collection will be carried out over a period of one year. Because the three classes of organizations have distinct orientations and types of members, organizations within the same sector are likely to differ in their practices of providing health and safety information to members depending upon class.

1.3 Sampling Methods

Because the business-oriented trade associations, professional-oriented trade associations, and labor unions in the 8 industrial sectors form distinct groups that may differ in results, the entire population will be divided into sub-populations, and simple random sampling will be carried out in each of these strata. Stratification will be done by sector (8) and by the type of organizations within the sector (3) for a total of 24 strata. The numbers of organizations in each of the 24 strata will vary greatly, and estimates within each stratum require sufficient precision to make them useful. Therefore, the approach of disproportionate stratified sampling will be carried out to ensure that there are enough cases included in smaller strata for meaningful results. By this approach the percent of members sampled can differ from one stratum to another. To calculate results for business-oriented trade associations as a group, professional-oriented trade associations as a group, and labor unions as a group, data must be weighted to take into account this sampling design.

1.4 Assumptions for Sample Size Calculations

Sample size calculations were carried out with the commonly used significance level of  = 0.05, for which there is a 5% chance of rejecting the null hypothesis when it is true [false positive]. The precision of an estimate (i.e., how close the estimate is likely to be to the actual population value) is also important when selecting sample size. The sample percent is used as the best estimate of the population percent, and a confidence interval placed around this estimate shows its precision. A wider confidence interval indicates more uncertainty associated with the estimate, making it less useful. Precision is inversely proportional to error (a statistical term, not an indication that a mistake has occurred). For a one-sided confidence interval error is measured as the distance from the estimate to the lower limit of a confidence interval and, for a two-sided confidence interval, as the distance from the upper limit to the lower limit.

Power and precision are two alternative ways of inputting information into sample size calculations for percents. It is desirable to have a power of at least 80%, which means that there is an 80% chance of rejecting null hypothesis when it is false [true positive]. For some of the smallest strata, obtaining 80% power may not be possible with sampling. A power approach begins with null and alternative hypothesized percents and determines the power of various sample sizes for these particular values. The hypothesized percent is an a priori guess about the actual percent in the population, not the sample. A precision approach, as described above, starts with a hypothesized percent and provides the width of two-sided confidence intervals (or half the width for one-sided confidence intervals) for various sample sizes.

The goal of this study is estimation, and the precision approach will be used. Because the focus of the study is to determine a lower bound on the proportion positive in each stratum, a “greater than” comparison is appropriate. Therefore, sample size calculations were based on one-sided confidence intervals rather than the usual two-sided.

Sample sizes depend in part upon the anticipated percents in the population that is being sampled. These hypothesized percents should be based on previous information. Because this study is an initial investigation, no information is available on anticipated percents. For a given sample size, confidence intervals tend to be wider the closer the population percent is to 50% rather than either larger or smaller than 50%. Therefore, the most conservative value of 50% will be used in calculating sample sizes.

1.5 Error Considerations

Table 1 presents the error obtainable from samples of various sizes with a significance level of 0.05, which corresponds to 95% confidence intervals using the normal approximation. Error is presented for a population value of 50% and also for population values of either 90% or 10% for comparison purposes. It was decided that 95 participants should be obtained from each sampled stratum, providing an error of 8.3%. For a sample size of 95 with a hypothesized population value of 50%, on average the lower bound on a confidence interval is 41.7%.

Table 1 provides lower bounds of one-sided confidence intervals for sample sizes ranging from 10 to 200 in steps of five.

Table 1: Error (%) of Estimates Obtainable by Samples of a Range of Sizes

Sample Size	Hypothesis: 90% or 10%	Hypothesis: 50%
200	4.0	5.8
190	4.2	5.9
180	4.3	6.1
170	4.4	6.3
160	4.6	6.4
150	4.8	6.7
140	5.0	6.9
130	5.2	7.1
120	5.4	7.4
110	5.7	7.7
100	6.0	8.1
95	6.2	8.3
90	6.4	8.4

1.6 Adjusting Sample Sizes for Response Rate

Because it is likely that not all contacted unions and business- and professional-oriented trade associations will agree to participate in the survey, the sample size of 95 should be adjusted upward to allow for non-response. Based on past ORISE experience conducting CATI surveys, an approximate 80% response rate is expected. The adjustment is done using the following equation:

Number to contact = (Calculated sample size) / (Estimated response rate)

= (95) / (0.8) = 119

To achieve better than 8.3% error, 120 organizations will be contacted to participate in the study for all strata having 120 or more members. For strata having fewer than 120 members, sampling will not take place. Instead the entire population will be invited to participate in the survey.

Table 2 shows the number of associations and unions to be contacted from each stratum and the number of anticipated survey participants. Extensive efforts will be made to obtain 100% responses for those strata containing fewer than 25 members.

Table 2: Sample and Population Sizes by Stratum

	Number of organizations to be contacted			Number of organizations surveyed based on 80% response rate
Sector	Business Trade Assoc.	Professional Trade Associations	Labor Unions	Business Trade Assoc.	Professional Trade Associations	Labor Unions
Agriculture, Forestry, & Fishing	120	56*	14*	96	45	12
Construction	107*	95*	18*	86	76	15
Healthcare & Social Assistance	23*	120	10*	19	96	8
Manufacturing	120	120	42*	96	96	34
Mining	67*	12*	11*	54	10	9
Services except Health & Social Assistance	120	120	120	96	96	96
Transportation, Utilities, & Warehousing	120	120	23*	96	96	19
Trade: Wholesale and Retail	120	120	14*	96	96	12
Total	797	763	252	639	611	205
	1812			1455

*Estimated population size

Table 3 shows the precision that can be obtained from each anticipated stratum size by presenting the error corresponding to each sample size, assuming that the sample percent is 50%. If sample percents are higher or lower than 50%, the confidence intervals will be tighter for a given sample size. Because an attempt will be made to contact the entire group in strata of size smaller than 120, the obtained percents could be considered as population values rather than estimates. Alternatively, the associations and labor unions in a sector at the time when the data are collected can be viewed as a sample of such groups over time. New organizations arise and existing ones merge or terminate. Therefore, concept of the precision can be meaningful in this context as well.

Table 3: Error (%) of Estimates for Anticipated Strata Sizes

	Error (%) of estimates for sample sizes in Table 2
Sector	Business Trade Association	Professional Trade Association	Labor Union
Agriculture, Forestry, & Fishing	8.3	12.0	21.7
Construction	8.8	9.3	19.5
Healthcare & Social Assistance	19	8.3	25.0
Manufacturing	8.3	8.3	13.6
Mining	10.8	23.0	24.0
Services except Health & Social Assistance	8.3	8.3	8.3
Transportation, Utilities & Warehousing	8.3	8.3	17.7
Trade - Wholesale & Retail	8.3	8.3	21.7

1.7 References

Kalton, Graham, Introduction to Survey Sampling, Sage University Paper # 35, Quantitative Applications in the Social Sciences Series, Sage Publications, Newbury Park, 1983.

Lee, E.S., Forthofer, R.N., Lorimor, R.J., Analyzing Complex Survey Data, Sage University Paper # 71, Quantitative Applications in the Social Sciences Series, Sage Publications, Newbury Park, 1989.

SamplePower, Release 1.20, SPSS, September 24, 1997.

2. Procedures for the Collection of Information

The survey will be conducted using a computer-assisted telephone interview conducted by trained interviewers. The interviewer will call the business-oriented trade association, professional-oriented trade association, or labor union and ask to speak with the individual handling workplace safety and health issues. Once the appropriate person has been reached the interviewer will ask if the person would participate in the survey. If the person agrees the interviewer will explain the survey and time required. If a different time is required the interviewer will ask to reschedule the interview and call back at the appropriate time.

As the interview proceeds the interviewer will enter information provided by the participant into the computer. When the interview is completed the interviewer will ask if there are any other comments the participant would like to mention.

The items in the survey are the product of an initial assessment of business-oriented trade associations, professional-oriented trade associations, and labor unions, development by NIOSH and ORISE personnel, reviews internal and external to NIOSH, pilot tests of eight trade associations, and a questionnaire design expert. The pilot tests also allowed for the determination of the 15 minute estimate of participation time.

3. Methods to Maximize Response Rates and Deal with Nonresponses

Letters (Appendix F) will be sent to the business-oriented trade associations, professional-oriented trade associations, and labor unions to introduce the survey and maximize participation rate. A CATI system will be used for the data collection reducing participant response time, effort, and burden, thereby increasing the potential for respondents to complete the full interview. In addition, for those respondents who do not choose to complete the full interview, they will be asked to complete a 2-question nonresponse form developed to capture a minimum amount of data about dissemination practices in regards to occupational safety and health. This nonresponse form will allow NIOSH to assess whether the associations and unions that complete the full survey are different from those that complete the nonresponse form for these 2 questions. It is our hope that the significance of the survey in terms of future opportunities to partner with NIOSH and receive NIOSH documents will outweigh the 15 minutes of burden to participate in the survey.

4. Tests of Procedures or Methods to be Undertaken

The survey instrument was reviewed by Dr. Grace LeMasters, Professor of Epidemiology and Biostatistics, in the Department of Epidemiology and Biostatistics at the University of Cincinnati, College of Medicine and Jennifer Tierney Lyden, Survey Implementation Specialist, Constella Group, LLC. Dr. Virginia Sublet, ORISE, conducted pre-tests with five organizations and pilot tests with eight trade associations and labor unions to determine the clarity, time, and opinion of participants about the survey.

5. Individuals Consulted on Statistical Aspects and /or Individuals Collecting and/or Analyzing Data Consultants

The final sampling plan was developed in consultation with the following two individuals with expertise in survey sampling:

Janice Watkins, Ph.D.

Chief Statistician

Oak Ridge Institute for Science and Education

Oak Ridge Associated Universities
P.O. Box 117
Oak Ridge, TN 37831-0117

Phone: (865) 576-3146

Martin Petersen, Ph.D.

Senior Statistician

Division of Surveillance, Hazard Evaluations, and Field Studies

National Institute for Occupational Safety and Health

Cincinnati, Ohio 45226

(513) 841-4234

Dr. Janice Watkins, Senior Statistician, ORISE (865 576-3395) designed the sampling frame in consultation with Martin Petersen, Ph.D. Dr. Watkins will analyze the survey data.

Virginia Sublet, Ph.D.

Oak Ridge Institute for Science and Education

Oak Ridge Associated Universities
P.O. Box 117
Oak Ridge, TN 37831-0117

Dr Virginia Sublet, Senior Toxicologist, ORISE (407 909-4744) will oversee the selection of sample respondents, development of the survey and database for the survey results, training of interviewers, collection of survey data, and preparation of individuals sector reports and summary reports.

Appendices…………………………………………………………………………………..

Appendix A: Copy of Authorizing Legislation………………………..………………….....

Appendix B: Copy of Data Collection Instrument………………………..…………………

Appendix C: Copy of 60 Day Federal Register Notice ....................................…………......

Appendix D: Copy of 60 Day Federal Register Notice Correspondence……………………

Appendix E: Follow-up Letter to Respondent……………………………………………….

Appendix F: Letter to Associations and Unions Requesting Help Completing Survey……..