13PR Part_B_01 16 2014

Evaluating the Implementation and Outcomes of Policy and Environmental Cancer Control Interventions

New

Supporting Statement – Part B

January 16, 2014

Primary Contact:

Angela Moore

Technical Monitor

National Center on Chronic Disease Prevention and Health Promotion

Centers for Disease Control and Prevention (CDC)

4770 Buford Highway NE, MS K-55

Atlanta, GA 30341-3724

Telephone: (770) 488-3094

Fax: (770) 488-4335

Email: cyq6@cdc.gov

Table of Contents

B. Collections of Information Employing Statistical Methods 1

B.1. Respondent Universe and Sampling Methods 1

B.2. Procedures for the Collection of Information 4

B.3. Methods to Maximize Response Rates and Deal with Nonresponse 5

B.4. Tests of Procedures or Methods to be Undertaken 5

B. 5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 5

List of Attachments

Attachment A: Authorizing Legislation

Attachment B.1: Federal Register Notice

Attachment B.2: Summary of Public Comments

Attachment C.1a: Program Director Web Survey Questionnaire (.DOC version)

Attachment C.1b: Program Director Web Survey Questionnaire (screen shots)

Attachment C.2: Program Director Advance Letter

Attachment C.3: Program Director Advance E-Mail

Attachment C.4: Program Director Reminder E-Mail

Attachment C.5: Program Director Thank You E-Mail

Attachment D.1: Key Informant Interview Guide

Attachment D.2: Initial Site Contact E-Mail

Attachment D.3: Key Informant Identification and Selection

Attachment D.4: Initial Key Informant Contact E-Mail

Attachment D.5: Informed Consent for Key Informant Interview

Attachment E.1: TA Providers Focus Group Instrument

Attachment E.2: TA Providers Focus Group Invitation

Attachment F.1: Coalition Survey

Attachment F.2: Coalition Survey Advance E-Mail

Attachment F.3: Coalition Survey Recruitment E-Mail

Attachment F.4: Coalition Survey Reminder E-Mail

Attachment F.5: Coalition Survey Thank You E-Mail

Attachment G: Battelle IRB Approval Letter

B. Collections of Information Employing Statistical Methods

B.1. Respondent Universe and Sampling Methods

Through the National Comprehensive Cancer Control Program (NCCCP), CDC supports 65 cancer prevention and control programs to reduce cancer-related morbidity, mortality, and health disparities. The focus of this evaluation is the 13 comprehensive cancer control (CCC) programs that were awarded funds by CDC (1017 program) to increase their focus on Policy, System, and Environmental (PSE) change strategies. The premise of the 1017 program is that providing additional funding and convening a targeted taskforce will increase the ability of CCC programs to implement PSE change. To assess whether the 1017 pilot is meeting its goals and to inform CDC’s future support of this program and similar efforts, CDC is conducting an evaluation of the 1017 program. Data available through existing monitoring systems are insufficient to support a thorough evaluation. The data collection activities included in this request are essential for a comprehensive evaluation of the 1017 program. The mixed-method evaluation will include (A) surveys of all 65 CCC programs, (B) longitudinal case studies of selected 1017 programs, (C) focus groups with Technical Assistance (TA) providers, and (D) a network analysis of 1017 program participants.

1. The Program Director Survey will be administered to the universe of Comprehensive Cancer Control Programs funded by the CDC. There will be no sampling. All 65 programs will be invited to participate.

2. Longitudinal case study of selected 1017 programs. There are 13 funded programs. Six of these will be selected for case study based on six criteria (four primary, two secondary) representing common and shared features of the programmatic, social, and historical contexts that are expected to influence PSE change processes and outcomes. The general approach for selecting cases was to look for diversity across the different dimensions of the criteria so that the role of significant characteristics or contextual factors can be carefully examined, and so that other programs that share those characteristics or factors can find some applicable lessons for their own PSE change efforts. We gave more weight to the primary criteria in the selection process. The primary case selection criteria are:

• Structure of Public Health System: As a reflection of the overall structure of the public health system in a jurisdiction, we will consider the model of state and local public health relationships and the centralization of those relationships, including (1) centralized organizational control; (2) decentralized organizational control; or (3) hybrid of centralized or decentralized models. The model of relationship between state and local health agencies may influence how the 1017 grantees work with community partners to inform PSE change at the local level. In addition, to the extent possible, we will consider the extent of coordination among chronic disease programs within health agencies. Greater coordination may facilitate collaboration across disease categories to achieve cross-cutting PSE changes that influence multiple chronic conditions.

• PSE Change Climate: This refers to whether decision-makers at state or local levels are amenable to and or support PSE changes. This also refers to environments and/or contextual factors that inform implementation of PSE change.

• Capacity for PSE Change: This refers to whether there is a history of public health programs that design and implement PSE change approaches that prevent chronic diseases. A history of support may imply enhanced capacity to support PSE change initiatives undertaken within the CCC program context.

• Focus Areas: We will also consider the areas of focus for the PSE change strategies adopted by the 1017 grantees, including general PSE approaches (legislative policy or systems change efforts), and how they have focused on different aspects of the cancer prevention and control continuum (primary, secondary, and/or tertiary).

The secondary case selection criteria are:

• Demographics: We will consider the proportion of a jurisdiction’s population that lives in rural areas (% state population in non-metropolitan areas) and % of population below 100% federal poverty level. Both of these demographic factors influence availability and access to a wide array of cancer-related services and care, and may influence the implementation and success in achieving PSE change.

• Population Health: While there are many relevant ways to characterize cancer-related population health in any given jurisdiction, we will consider the annual percentage of adults age 50 and older who receive recommended screenings and preventive services as an overall indicator.

Selection of key informants (staff, partners, and community members) in selected case study sites will be based on consultation with the Program Director and PSE Coordinator in that site. Selection will be purposive, based on position and role in the program. The case study coordinators will provide the program contact person instructions and a tool for identifying and listing potential key informants (see Appendix D3: Key Informant Identification and Selection). The program contact will be asked to use the tool to list 10-15 potential key informants based on their involvement in PSE change strategies supported by the 1017 program. For each person listed, they will be asked to provide a brief description or rationale for why they are relevant to the case study. The following categories will guide the identification of key informants:

• CCC Program Directors and program managers/coordinators;

• PSE Coordinator hired and/or supported by 1017 funds;

• CCC program staff with a high degree of involvement in PSE change initiatives;

• Partners who are members of the PSE Workgroup;

• Community members who play an important role in implementation of the PSE change strategies, including decision makers, educators, stakeholders, and allies.

We anticipate that the 1017 programs may identify more than 12 potential key informants, and that not all potential key informants will be able to participate in the site visit interviews. After receiving the list of potential key informants, the case study coordinator will work with the other site visit team members (Battelle and CDC staff) to select approximately 10-12 individuals to participate in the interviews based on their fit into one of the categories described above and the description/rationale for their selection.

During the site visits, additional names of key informants may surface. The site visit teams will be attentive to these suggestions, and if merited, one or two additional interviews will be scheduled during the visit (or later by telephone if logistic considerations prevent that individual from being interviewed in person).

For Wave 2 site visits, we will attempt to schedule interviews with the same group of key informants who participated in the first site visits. However, we will be flexible and will add or drop key informants for Wave 2 to ensure that the appropriate stakeholders are invited to provide the needed data.

3. TA Focus Groups will be held with all available CDC staff and national partners that provide Technical Assistance to the 1017 grantees. Thus it is a purposive and inclusive group, not a sample. We estimate that 15 individuals will participate in an annual focus group.

4. Coalition survey of 13 1017 programs. Survey will be completed by individuals connected to the program through formal affiliation (staff or coalition members) or through informal association (strategic partners). The survey will not be fielded until 2015. Based on an estimated membership of 15-20 partners in each site, the total N is estimated at 13 x 20 = 260 responses. We will work with the 1017 grantee Project Directors to identify and invite members of the PSE Workgroup to participate in the survey. Selection of participants will be purposeful, based on each individual’s role in the Workgroup or with the program’s strategic objectives. Participants will be selected by CDC and Battelle with input from the local program directors.

Recruitment Procedures

Recruitment of survey participants, key informants, and focus group participants will be conducted by the contractor (Battelle).

1. All 65 program directors will be invited to participate in the Program Director Survey. The first invitation letter will be signed by CDC. Battelle will send an invitation via email with a link to the on-line survey. The text proposed for each of these communications along with the text for a follow up reminder email and a thank you email are attached (see Attachments C2-C5).

2. Programs selected for case study. First, the relevant CDC Program Consultants will make an initial contact via email with the Directors of the selected 1017 programs to introduce the study, the CDC Technical Monitor for the 1017 evaluation, and the Battelle case study coordinator. After the introductions, the case study coordinators will contact the Program Directors to provide more information about the 1017 evaluation, the case studies in general, and the site visits in particular. Text for the introductory e-mail message is included in Attachment D2. The case study coordinators will provide details about the site visits so that the 1017 program staff will know exactly what to expect and what their role will be. If appropriate, the case study coordinators will ask the Program Directors to assign another program staff person (e.g., program managers or coordinators) to act as the main site visit liaison. For Wave 2, the case study coordinators will contact the Program Directors (or liaison) directly to coordinate the second site visit for each site. Once sites have been recruited and key informants identified, recruitment of the key informants will be done by the contractor using the email text provide in Attachment D4.

3. Participants in the TA Focus Groups will be identified by CDC and invited to participate via email. The email invitation will be from CDC and Battelle jointly. All invitees provide technical assistance to grantees through formal arrangements with CDC (see Attachments E1-E2).

4. Participants in the Coalition Survey will be identified in consultation with local program staff and invited to participate via an email from Battelle.

Information will be collected over a three year period. Table B-1.1 provides information about the total number of respondents associated with each data collection instrument or supplementary document. Table B-1.1 also shows how we estimated the annualized number of respondents for each data collection activity.

Table B-1.1: Number of Respondents and Implementation Schedule for Information Collection

			Number of Respondents
Type of Respondent	Form Name	Att. #	Wave 1	Wave 2	Wave 3	Total Respondents over 3 years	Annualized # of respondents
CCC Program Directors	Program Director Web Survey advance letter, reminder email, and thank you email	C.2-C.5
	Program Director Web Survey Questionnaire	C.1a/b	65	65	0	130	43
CCC Staff	Key Informant Selection	D.2-D.3	6	0	0	6	2
	Key Informant Recruitment/Scheduling	D.4	18	18	0	36	12
	Key Informant Consent	D.5
	Key Informant Interview Guide	D.1	18	18	0	36	12
CCC Partners	Key Informant Recruitment/Scheduling	D.4	72	72	0	144	48
	Key Informant Interview Guide	D.1	72	72	0	144	48
	Coalition Survey advance email, recruitment email, reminder email, and thank you email	F2.-F.5
	Coalition Survey	F.1	0	260	0	260	87
	TA Provider Focus Group Invitation	E.2
	TA Provider Focus Group Guide	E.1	15	15	15	45	15

B.2. Procedures for the Collection of Information

1. The Program Director Survey will be administered online. The survey will be conducted twice, once upon approval from OMB in 2013 and again 18-24 months later.

2. Key informant interviews that will be done as part of the case studies will be done in person during Wave 1 (2014) and by telephone in Wave 2 (2015). If any individuals are unavailable for an in-person interview, the interview may be conducted by telephone.

3. To the extent possible, we will also observe meetings held by the CCC programs with their respective PSE Workgroups and other key partners involved in the various PSE change strategies during Wave 1 in-person data collection. Our ability to observe meetings depends on whether permission is granted from the CCC program leaders (e.g., Program Directors or managers/coordinators) and the willingness of the partners to allow our observers to listen in on the meetings. The meeting facilitators will announce the participation of the Battelle observer at the beginning of the meeting during introductions. There will not be a formal data collection instrument for the meeting observations. The meeting notes (and official meeting minutes, if available) are eligible for incorporation into the case study database for each selected 1017 program. No identifying information for the meeting participants will be recorded in the notes. There is no burden on programs associated with this data collection activity.

4. The TA Focus Groups will be conducted via telephone annually.

5. The Coalition Survey for the network analysis will be administered online once in 2015.

B.3. Methods to Maximize Response Rates and Deal with Nonresponse

We expect a high degree of cooperation by 1017 program staff and partners for the Program Director Survey, the key informant interviews and the TA focus groups due to the high degree of commitment to comprehensive cancer control among these stakeholders. However, Battelle will employ several methods to facilitate high response rates:

• Initial invitation to participate in the Program Director Survey will come from the staff person at CDC who has a long standing relationship with that program.

• A hyperlink to the web survey will be included in all of the follow-up e-mail correspondence (including initial invitation and up to 2 email reminders).

• The survey can be completed at a time convenient to the director and can be stopped and started as needed before final submission.

• A multi-pronged approached will be utilized for recruiting the key informants. Initial contact will be with the key program staff, all of whom are highly vested in this pilot program. They in turn will identify potential key informants among other staff and partners.

• A study coordinator will be assigned to each site and will be responsible for individual invitations and scheduling.

• Telephone option will be provided for individuals unavailable during the site visit

• TA focus group participants will be invited by CDC staff with whom they have a relationship.

• The invitation to partners to complete the coalition survey will be preceded by an email from the local staff alerting them to expect the survey from Battelle and encouraging them to participate.

B.4. Tests of Procedures or Methods to be Undertaken

The survey instrument was reviewed by an Evaluation Consultant Group for content and clarity. It was further tested with three directors or closely related CDC programs for the purpose of clarity of instructions and response items and for estimating time burden. The procedures are based on those used in a recent survey of CCC program directors that achieved a 92% response rate.

Interview and focus group procedures are modeled after similar studies that achieved high response rates.

B. 5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Battelle Centers for Public Health Research and Evaluation (CPHRE) staff worked with staff from CDC to design the data collection plan and instruments. Carlyn Orians, MA, led the Battelle team (206-528-3320). John Rose, PhD (919-544-3717), Marilyn Sitaker, MPH (206-528-3365), and Gary Chovnick, DrPH (206-528-3013) assisted in the Battelle effort to design and test survey instruments, interview and focus guides; develop recruitment procedures; and prepare plans for analyzing and reporting the data. In addition, an Evaluation Consultant Group was convened to provide theoretical, methodological, and interpretive guidance throughout the project. The Evaluation Consultant Group is comprised of members external to CDC who collectively have expertise in coalitions, PSE change, cancer, and evaluation, as well as experience in community-based implementation of PSE change istrategies. To date, the Evaluation Consultant Group has provided input on the logic model, evaluation plan, protocols, and instruments.

Battelle will collect and analyze the data for CDC. The overall data collection, analysis, and reporting effort will be directed by Battelle’s Project Director, Ms. Orians.