0920-07AL_Supporting Statement_Sect B_10 22 07

Section B

of the

Supporting Statement

for

Evaluation of the Successful Business Strategies to Prevent Heart Disease and Stroke Toolkit

(0920-NEW)

CDC Task Order Officer (TOO): Dyann Matson Koffman

National Center for Chronic Disease Control and Health Promotion

TOO Contact Information: 770-488-8002 (phone), 770-488-8151(fax)

August 20, 2007

Revised October 2007

Section B. Statistical Methods

B1. Respondent Universe and Sampling Methods

The sample is the entire universe of 51 Heart Disease and Stroke Prevention programs within State Health Departments and the District of Columbia. Surveying the entire universe is preferable to using a sample because each program’s experience may differ from others in many respects. We cannot rely on a sample because we do not know in advance the extent and nature of activities carried out with employers and variations in the use of the Toolkit, which is the purpose of this evaluation. The expected response rate for this survey is at least 50 percent. AIR expects to achieve this rate as we have regular communication with target respondents, through Webinars and conference calls for trainings and consultations regarding the Toolkit. We will also provide an incentive of an additional copy of the Toolkit to the first 15 respondents to the survey.

B2. Procedures for the Collection of Information

Procedures for the collection of information do not include stratification and sample selection. We will survey the entire universe as described above in B1. This survey is a one-time data collection. The survey will be posted on the Web using the “SurveyMonkey” software. We will send the link to the Web-based survey via e-mail to a list of potential respondents from State Health Departments who work in cardiovascular health promotion and/or heart disease and stroke prevention (Attachment 3 for a copy of the initial and reminder e-mail message). The list will be provided by the CDC. The e-mail will instruct them that only one person from each SHD’s HDSP Program should respond to the survey. The e-mail text will also ask interested respondents to complete the survey within two weeks. Completion of the survey is voluntary. All potential respondents will also receive another email two weeks after the initial email thanking those who have completed the survey and reminding others that they may do so in the following 2 weeks and that the survey will not be available after that time. Thus, the survey will be available on the Web for a total of 4 weeks. We are also offering an extra copy of the Toolkit to the first 15 States that complete the survey.

As a means of quality control, copies of the e-mail messages to potential respondents will be seeded to an AIR researcher. An AIR research team member will also check the Web site for the survey for functionality on a daily basis. Validity checks of the data with respect to skip patterns are automatically programmed during the survey administration. Data coding is automatically done as part of the Web-based survey data collection using the software “SurveyMonkey”. The database of responses to the survey will be downloaded into an SPSS file and analyzed by AIR staff as described in section B1. Individual State results will not be reported; results will only be reported in the aggregate. All data will be treated in a confidential manner, unless otherwise compelled by law.

The power calculations in the table below were estimated to determine the scientific viability of the proposed study. Based on this information, we believe that there is adequate sample if we have at least 25 States respond to this survey.

Table B2-1. Power Calculations

Power (for n=25)
Change Points out of 100	Alpha = .05 One-tailed test	Alpha = .05 Two-tailed test
25	99.99%	99.99%
20	98.0%	99.0%
15	85.0%	91.0%
10	53.0%	64.0%

The table shows the powers to detect a range of average change for any of the continuous variables for the study, including Likert scales, which constitute the majority of the items. This presumes that at minimum 25 subjects have data for the proposed survey and a previously conducted survey. Since there are no descriptive statistics available for all the items, we standardized the scales to 100 points and considered a standard deviation of 25 with a moderate correlation over time. This means that 1 point out of a 4-point scale is valued as 25 out of a 100 point scale and 1 point out of a 5-point scale is valued as 20 out of a 100 point scale, etc. This standard deviation assumption is conservative (i.e., by using this value we are more likely than not, to underestimate the power to detect differences).

B3. Methods to Maximize Response Rates and Deal with Nonresponse

As mentioned in Section B1, estimated response rate for this survey is at least 50 percent. We expect to achieve this because CDC and AIR have had frequent contact with the HDSP or CVH programs in State Health Departments as part of this project. In developing and promoting use of the Toolkit, we have provided them with trainings and consultations, building a rapport with potential respondents. Between March and October, 2006, 6 trainings and 1 consultation call took place. Another consultation call took place in January 2007 and a final consultation call took place place in April 2007. We are also offering an extra copy of the Toolkit to the first 15 States that respond. This incentive was shown to boost response rates to the training evaluation forms.

In addition, to contribute to achieving at least a 50 percent response rate and manage issues of nonresponse, we plan to send two email messages to potential respondents. Both the initial and the reminder email messages will include a direct link to the survey Web site, so those who desire to respond to the survey only have to click on the link. The reminder email will be sent 2 weeks after the original e-mail inviting respondents to complete the survey.

Although a response rate of 80 percent or above is usually desirable, we have estimated at least a 50 percent given the 40 percent response rate to the five training evaluation forms. A similar survey conducted previously (which we are going to use to compare results) had a 100 percent response rate. However, this rate was achieved at the beginning of the project, not at the end, which is the case for the proposed survey. Achieving 100 percent included distributing copies of the Toolkit to all respondents, multiple phone calls from the Project Coordinator to each potential respondent, and an extended period of data collection. These strategies are not possible for this survey given the time still available in the contract for the project and the fact that in this case, States already have at least once copy of the Toolkit. Given our power calculations above and the fact that we are surveying the entire universe, rather than a sample, a 50 percent response rate should be sufficient.

B4. Tests of Procedures or Methods to be Undertaken

A Web survey with most of the same items was implemented with SHD representatives from the Heart Disease and Stroke Prevention or Cardiovascular Health programs more than a year from the date of the proposed survey data collection. The survey, administered in February 2006, served as the preliminary test of the proposed survey, as it included similar content and structure. The February 2006 survey also was administered via the same method as the proposed survey (via the web). This initial survey will serve in place of a pilot survey.

B5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The CDC worked with a contractor, the American Institutes for Research, on statistical aspects of the design. The contractor is also responsible for collecting and analyzing the information for the agency as shown in the table below.

Table B5-1. Contractor Staff Responsible for Statistical Design, Data Collection and Analyses

Name	Agency/Organization	Telephone Number	Email	Role
Margarita Hurtado	American Institutes for Research	(301) 592-2215	mhurtado@air.org	Project Coordinator-- Data collection statistical design; data collection and analysis
Karen Gold	American Institutes for Research	(301) 592-3373	kgold@air.org	Data collection statistical design
Alec Ulasevich	American Institutes for Research	(301) 592-2156	aulasevich@air.org	Data collection statistical design and data analysis
Anna Levin	American Institutes for Research	(919) 918-2321	alevin@air.org	Data collection and analysis

2