CCT Data Collection Plan

DFC Evaluation Systems, Measures, and Tools

Attachment 7:

CCT Data Collection Plan

CCT Data Collection Plan

CCT Table of Measures¹

Field Name	Data Type	Description	Keep (Keep, Delete, or New)	Std. Corr.	Rationale
CCT – Coalition Development & Management Global Rating	Likert	Section I: Coalition development and management involves designing and implementing your coalition’s organizational structure and operating procedures. These elements are fundamental and create the delivery mechanism for the array of prevention strategies that will be undertaken.	Keep		This is one of four function areas that comprise the coalition typology. Past evaluation reports have extensively used the typology to classify/categorize coalitions (i.e., Establishing, Functioning, Maturing, & Sustaining) and assess if they achieve different results on core outcomes. The typology should be retained in order to compare future findings to historical data. Additional typologies or other methods to categorize coalitions in developmental stages will also be investigated in future analyses. Overall reliability = .88.
		Assessing needs*	Keep	.72
		Mobilizing and building capacity in the coalition*	Keep	.73
		Developing a comprehensive plan*	Keep	.74
		Implementing*	Keep	.70
		Evaluating*	Keep	.65
		Planning for sustainability*	Keep	.63
Coalition Type	Check one	Item stem - Which of the following best describes your coalition?	Keep		This is basic and necessary descriptive information which should be retained.
		A loosely organized group whose main goal is information sharing.	Keep
		A semi-formal group of organizations who have begun to work together on prevention programs and strategies.	Keep
		A formal group or organization who plan and act together to implement prevention programs and strategies.	Keep
		A highly formal arrangement with most organizations having a clear role in the planning and implementation of community wide prevention strategies.	Keep
Organizational Structure	Check One	How would you describe your coalition’s organizational status?	Keep		This is another basic and necessary item to obtain some descriptive information about coalition structure and functioning.
		Our coalition reports to a larger organization on programmatic and fiscal matters.	Keep
		Our coalition reports to another organization on fiscal matters only, but we are able to operate otherwise independently, including choosing our own staff.	Keep
		Our coalition is a totally independent organization. We make decisions regarding finances, personnel, and programming.	Keep
	Text	Other, please specify:	Keep
Confidence in Task Completion	Likert	Item stem - How confident are you that your coalition can…?	Keep		This is a reliable scale that asks members if they are confident that their coalition can complete a number of tasks, from agenda setting to developing new leaders. This scale can be used to create a measure of collaborative formalization and could possibly be re-worded along strongly disagree to strongly agree anchors, reducing the need for a separate section and decreasing response burden. Factor Analysis – Good factor loadings (all >.50, except “develop agendas” which cross-loads with another factor). Overall reliability = .94.
		Develop agendas and stick to them in meetings.	Keep	.48
		Recruit new members who have the ability to take action in the community	Keep	.67
		Recruit new members who are accountable for reporting to their organization or constituency.	Keep	.69
		Follow up on decisions made at meetings.	Keep	.66
		Recruit members from the different sectors needed to address your Coalition’s goals.	Keep	.68
		Provide direction and vision for the Coalition through its leadership.	Keep	.77
		Maintain support of Coalition members through its leadership.	Keep	.78
		Share leadership among Coalition members.	Keep	.73
		Maintain stable leadership.	Keep	.75
		Develop new leaders.	Keep	.75
		Delegate responsibilities to committees.	Keep	.64
		Engage members of target populations (e.g. youth) and diverse cultural groups as active members and leaders.	Keep	.61
		Set and achieve annual goals.	Keep	.70
		Recruit “champions” to act on Coalition’s behalf.	Keep	.68
		Hold each other accountable.	Keep	.73
Coalition Governing Body	Yes/No	Item stem - We have a board or governing body that sets the direction of the coalition (A board or governing body is defined as a formal group or body that makes decisions for the coalition)?	Keep		This item precedes all the questions regarding written procedures and provides some basic descriptive information which is needed. This item should be retained.
Written Procedures	Yes/No Likert ** 3-point	Item stem - Please Indicate your agreement by selecting “Yes” or disagreement by selecting “No” with each of the following statements as they apply to your coalition.	Keep		Yes/No items on collaborative procedures are pretty common and standard items. There are currently eleven items regarding if there are written procedures around coalition activities and other items meant to tap into the “formal” nature of the collaboration. These items can be used to create scales (using a cut point to identify formalized coalitions versus not formalized) or looked at individually. There are some disadvantages to this method though, such as giving equal weighting to each activity and possible lack of variance across coalitions.
		We have a governing body that sets the direction of the coalition.	Keep
		We have a written policy for leadership rotation.	Keep
		We have written expectations for member participation (e.g., policy on missed meetings).	Keep
		We have a written description of procedures for leader selection.	Keep
		We have a written description of the procedures for decision making (e.g., majority rule, etc.).	Keep
		We hold regularly scheduled meetings (i.e., on a specific date/time).	Keep
		We prepare a written agenda for each Coalition meeting.	Keep
		We prepare and distribute written minutes of Coalition meetings.	Keep
		We have a current organizational chart showing Coalition structure and relationships.	Keep
		We have established subcommittees.	Keep
		We have paid staff members or in-kind staff (in-kind staff are staff who are paid by someone else to work at your coalition).	Keep
Identification of Community Leaders	Likert	Item stem - Please indicate how much you agree or disagree with each of the following statements.	Keep		Adequate factor loadings. This scale has adequate reliability and seems to tap into the grassroots and community-based nature of DFC. Thus, we recommend retaining this scale. Overall reliability = .70.
		Our coalition has identified all of the community leaders that can help in the coalition’s activities.	Keep	.45
		Our coalition regularly works with the necessary community leaders and they work on coalition activities or actively support strategies.	Keep	.56
		The most important community leaders are members of the coalition and work on coalition committees and activities.	Keep	.53
Community Leadership in Coalition Efforts	Likert ** 3-point	Please indicate the degree to which each statement below describes the role of the community leadership (not coalition staff) of the coalition. The coalition community leadership…	Keep		This scale follows-up on the previous scale and asks about the extent of community leaders’ involvement in collaborative activities.
		Takes responsibility for coordinating the coalition.	Keep
		Sets the agenda for coalition meetings.	Keep
		Keeps coalition members accountable to tasks.	Keep
		Promotes cohesiveness and team spirit.	Keep
		Creates an environment where opinions can be voiced.	Keep
		Are recognized leaders in the larger community on issues related to substance abuse prevention and related issues.	Keep
		Are very involved in regional, state or national prevention organizations or coalitions.	Keep
Coalition Leadership	Likert	Item stem - Coalition leadership refers to the official chair or co-chairs of your coalition. Below are several statements about leadership within coalitions.	New		In the current CCT, there are two good scales around community leadership – but there is no specific scale for coalition leadership. This may be because one of the co-chairs fills out the survey. If this is often the case, we propose at least including short scales on committee leadership and coalition coordinator leadership. Practitioners stress the importance of leadership and empirical research has demonstrated the strong links between leadership and collaborative effectiveness. These scales were originally adapted from Butterfoss (1998). Overall reliability is .97.
		Coalition leadership is committed to the coalition’s mission.	New
		Coalition leader(s) provide leadership and guidance in maintaining the coalition.	New
		Coalition leader(s) have appropriate time to devote to the coalition.	New
		Coalition(s) leaders promote equality and collaboration among members.	New
Committee Leadership	Likert	Item stem - Committee leadership refers to the chair or co-chairs of subcommittees.	New
		Committee leader(s) plan meetings effectively and efficiently.	New
		Committee leader(s) are flexible in accepting different viewpoints.	New
		Committee leader(s) is/are adept in organizational and communication skills.	New
		Committee leader(s) are adept in obtaining resources.	New
Coalition Coordinator Leadership	Likert	Item stem - Coalition coordinator refers to the paid staff on your coalition.	New
		Does funding from DFC support a part-time or full-time coalition coordinator?	Yes/No
		Does funding from local or another source support a part-time or full-time coalition coordinator?
		Our coalition coordinator plays a vital role in organizing coalition activities and efforts.	New
		The coalition coordinator facilitates communication across coalition participants.	New
		The coalition coordinator supports the coalition’s goals.	New
Collaborative Decision Making	Likert ** 3-item	Item stem - We would like to understand the effectiveness of your coalition in making decision. Please indicate the degree to which you either agree or disagree with each of the following statements describing how your coalition makes decisions. If you have trouble deciding, please select the answer that describes your feelings most of the time.	Keep		Decision making processes are key to coalition functioning and member satisfaction. This scale has adequate reliability if the reverse coded item is deleted. Reliability is somewhat low because a 3-point scale is used and it seems there are two issues being asked about: (1) if decisions are made and (2) if the decision making process is democratic and participatory. It might be beneficial to add a few items from pre-existing scales to increase reliability or use a 5-point agreement response set.
		The coalition makes decisions when they are needed.	Keep
		The general membership has real decision-making control over the policies and actions of the coalition.	Keep
		Decisions are made by a small group.	Keep
		There is formal process for making decisions (e.g., a voting system).	Keep
		Decisions on the allocation of coalition resources are made in an open and participatory manner.	Keep
		We effectively use the coalition process to plan and make decisions	Keep
Shared Vision & Cohesion	Likert	Item Stem - Please indicate the extent to which you agree with each statement from (1) Strongly Disagree to (5) Strongly Agree.	New		According to reviews of collaborative research, shared vision is one of the key dynamics that explains variability of coalition functioning and effectiveness. 5 items with .87 reliability.
		The coalition has a feeling of cohesiveness and team spirit.	New
		There is a shared vision for desired outcomes of DFC coalition work (e.g., reduce use, increase protective factors).	New
		Coalition members feel valued and important.	New
		There are clearly defined, attainable goals for the initiative.	New
		There is a shared vision of what the coalition should accomplish.	New
Frequency of Conflicts	Likert	How frequently do conflicts such as major disagreements and tensions among individual and groups occur in your coalition?	Delete		The items around frequency, cause, and impact of conflicts seem to provide less information than other items and they are recommended for deletion due to the need to include additional variables and items. For instance, check all that apply items are difficult to analyze in a meaningful fashion. If retained, CCT should be altered so communities could at least rank the top 2 or 3 causes for conflict.
Cause of Conflicts	Check all that apply	What are the causes of these conflicts? (Please check all that apply?)	Delete
		Different perspectives or philosophies	Delete
		Personality differences	Delete
		Turf battles	Delete
		Issues in the larger community	Delete
		Arguments over allocation of coalition resources	Delete
		Arguments over among or burden of work	Delete
	Text	Other, please specify	Delete
Impact of Conflicts	Check one	Overall, what has been the impact of these conflicts on the coalition? (Please check only one).	Delete
		The coalition is a lot worse off/Conflicts have hurt the coalition’s progress.	Delete
		The coalition is somewhat worse off.	Delete
		These conflicts have not made a difference.	Delete
		The coalition is somewhat better off because of the conflict.	Delete
		The coalition is a lot better off/very positive results.	Delete
Communication and Conflict Management	Likert	Item Stem - Please indicate the extent to which you agree with each statement from (1) Strongly Disagree to (5) Strongly Agree.	New		This was added because conflict management is an important coalition dynamic, but prior questions were not obtaining the necessary data. This is an abbreviated scale that we have used in the past which has established reliability. It can be used as one scale or broken out into communication management and conflict management (depending on results of factor analysis).
		There is a formal process for resolving conflicts among participating organizations.	New
		The communication procedures are clearly understood among collaborative members.	New
		Conflicts arise frequently among participating organizations in the collaborative.	New
		Communication between member organizations is closed and guarded.	New
		Differences among collaborative members are recognized and worked through.	New
Cultural Diversity	Likert	The following statements cover issues regarding how your coalition addresses cultural diversity and cultural competence. Please read each statement carefully and check the one response that comes closest to your belief. With regard to cultural diversity, your Coalition has....?	Keep		Good factor loadings (>.5). This scale has seven items and good reliability. Also, given the defining role of cultural competence or diversity in DFC, we recommend retaining this scale. Overall reliability = .85.
		Staff members that are representative of the demographic and cultural diversity in your community.	Keep	.54
		Culturally and language appropriate materials relevant to the target population.	Keep	.62
		Materials examined by diversity experts or target population members.	Keep	.56
		Developed a culturally appropriate outreach action plan.	Keep	.61
		Activities and decision-making processes that are designed to be inclusive.	Keep	.61
		Meetings and activities scheduled at times that are convenient and at locations that are accessible to the target population	Keep	.57
		Coalition members that are representative of the demographic and cultural diversity in your community.	Keep	.69
		Targeted youth are involved in coalition meetings and activities.	New		This was added because of the importance of engaging and involving the target community of DFC efforts.
Performance Evaluation	Yes/No	Item stem - Please respond Yes or No for the following statements. Does your coalition …?	Keep		This Yes/No list of performance evaluation and regular evaluation items should be retained given the need for GPRA data and substance use data. Items in the CCT can only serve to reinforce the importance of data collection for coalition leaders. Possibly create a Likert scale to provide a reliability estimate for the scale.
		Record coalition decisions in minutes or otherwise keep track of decisions.	Keep
		Have a system for monitoring and tracking coalition activities and other efforts.	Keep
		Monitor on a regular (annual or more frequently) basis community indicators of substance abuse or related information.	Keep
		Have someone on staff at the coalition or a member organization that assists the coalition with collecting and analyzing data on coalition activities and community indicators.	Keep
		Use relations with local universities or others to get assistance on evaluating coalition efforts.	Keep
		Typically hold meetings to reflect on the result of monitoring or evaluation activities in order to make adjustments to implementation.	Keep
		Collect school survey data that matches the coalition’s geographic boundaries at least once every two years.	Keep
Coalition Sustainability	Likert	There are many factors that may contribute to the developmental of a sustainable coalition. A sustainable coalition is one that can continue to work in the community without DFC funding and share ownership and capacity to collaboratively prevent substance abuse. We would like your assessment of the extent to which your coalition has accomplished each of the following. Your coalition has ….	Keep		Sustainability is a major focus for DFC and is even factored in the typology ratings assigned to coalitions, yet we believe it is not being fully assessed in the current version of the CCT. Of the thirteen sustainability items, we propose retaining six and adding an additional five items for an overall reduction in the number of items but expansion in assessing the quality of coalitions’ sustainability efforts. In addition to simply asking if the coalition itself will keep meeting, the items tap into other mechanisms through which DFC has been embedded/ institutionalized in the community (via MOUs, procedures for sharing relevant information, strategies that combine agency resources, such as blended funding, etc.). Factor analysis is OK with a couple items below .5. Good overall reliability (.77). The last six items utilize CADCA’s
		Established a reputation for ‘being able to get things done’ related to at least one initiative or practice.	Keep	.57
		Adopted an entrepreneurial spirit in seeking additional support.	Keep	.54
		Established on-going jobs in your organization.	Keep	.47
		A mature and stable lead organization or has all functions (501c3 status, etc.) to operate independently.	Keep	.45
		Aligned the coalition’s goals and priorities well with past work done by the coalition.	Keep	.57
		A plan for continued leadership.	Keep	.52
		The DFC community coalition plans on continuing meeting after federal grant funding ends.	New
		Youth-serving organizations have begun to or are already taking on coalition efforts in order to institutionalize them and provide long –term sustainability	New
		The coalition has developed strategies to continue to combine agency resources to better serve youth and families (e.g., blended funding, identification of alternative funding, etc.).	New
		The coalition has secured funding to continue its substance abuse reduction efforts when federal funding ends.	New
		Procedures for continuing to share relevant information across agencies have been established.	New
These items correspond to CADCA’s 6 steps of sustainability planning		Identified what must be sustained to continue our community-based efforts toward drug abuse prevention.	New
		Identified the resources that are required to continue our community-based efforts toward drug abuse prevention.	New
		Created case statements that explained our need to continue community-based efforts toward drug abuse prevention efforts.	New
		Determined funding strategies to continue our community-based efforts toward drug abuse prevention.	New
		Identified potential partners to continue our community-based efforts toward drug abuse prevention.	New
		Created an action plan to contact and present to potential partners in our efforts to continue community-based drug abuse prevention.	New
Sustainability	Choose one (Yes/No/Maybe or Don’t Know)	Item stem - Do you think your coalition will be able to sustain itself in your community for the next ten years?	Delete
	Check all that apply	Why don’t you think your coalition will be able to sustain itself in your community for the next ten years?	Delete
		Changes in circumstances, staff, and resources.	Delete
		DFC is no longer a good “fit” in our community.	Delete
		Desired outcomes were not achieved.	Delete
		Changes in population, place, or policy of interest reduced the need for DFC.	Delete
		Funding and resources are not available to continue	Delete
	Text	Other, please specify.	Delete
Coalition Focus 1	Check one	Some coalitions focus on coordinating programs or services delivered by their partners (e.g., curriculum for youth or parents, provision of after school programs); others focus on changing aspects of their community environment (e.g., availability, policy, enforcement). Which of the following do you think best describes the current status of your coalition?	Keep		Keep for identification of degree of coalition focus on direct services and prevention programming compared to environmental focus.
		Our coalition focuses exclusively on coordinating the provision of prevention programs or services.	Keep
		Out coalition focuses primarily on coordinating the provision of prevention programs/services, though occasionally attention is paid to environmental or policy change.	Keep
		Our coalition’s focus is about evenly split between coordinating prevention programs/services and environmental or policy change.	Keep
		Our coalition focuses primarily on environmental or policy change, though occasionally attention is paid to coordinating prevention programs and services.	Keep
		Our coalition focuses exclusively on environmental or policy change.	Keep
CCT – Coordination of Prevention Programs/Services	Likert	Section II: Coordination of Prevention Program/Services. A community uses community data and lists of evidence-based programs and services to identify and coordination an array of “best fit” prevention programs and services to be delivered by its partners. A coalition then aligns and coordinates the integration of these programs and services across the community and evaluates their impact.	Keep		This is one of four functional areas that comprise the Coalition Classification Tool. Classifications have been extensively used in data analyses and should be kept for historical comparative purposes. Overall reliability = .91.
		Assessing needs*	Keep	.73
		Mobilizing and building capacity of partners*	Keep	.80
		Developing comprehensive Strategic and Action Plans*	Keep	.78
		Implementation of Strategic and Action Plan*	Keep	.74
		Conducting Process & Outcome Evaluations*	Keep	.72
		Planning for sustainability*	Keep	.69
Action Plan Activities	Likert	Please choose one response for each of the following statements to characterize your coalition. Your coalition has ……	Keep		Factor Analysis - All greater than .50 except 1 (evidence-based, which approaches .50 and loads on another factor). This is a reliable scale which obtains good information on the degree of follow-up action planning. Overall reliability = .87.
		Identified specific strategies and activities to reach its goals.	Keep	.61
		Created a realistic timeline for completing activities.	Keep	.71
		Identified responsible person(s)/agencies for each activity.	Keep	.71
		Developed a strategy to recruit participants for activities or events.	Keep	.61
		Developed a budget that outlines the funding required for each coalition related activity, training, or event.	Keep	.65
		Identified other resources needed for each activity.	Keep	.69
		Planned to use primarily evidence-based strategies.	Keep	.48
		Plans to evaluate strategies and activities.	Keep	.56
		Revisited and updated your action plan.	New		Added to track a DFC requirement to update action plans annually (after creating the strategic plan in the first year).
CCT - Environmental Strategies	Likert	Section III: Environmental strategies (e.g., reducing access, policy change, increasing enforcement) involve the coalition assessing the community environment to select and implement an array of “best fit” environmental initiatives. The coalition then aligns and integrates them across the community and evaluates their impact.	Keep		This is one of four functional areas that comprise the coalition typology. Classifications have been extensively used in data analyses and should be kept for historical comparative purposes. Overall reliability = .93.
		Assessing needs*	Keep	.79
		Mobilizing and building capacity*	Keep	.83
		Developing a comprehensive plan*	Keep	.85
		Implementing strategies*	Keep	.83
		Evaluating environmental strategies*	Keep	.77
		Planning for sustainability*	Keep	.64
Environmental Strategies	Likert	Please indicate how much you agree or disagree with each of the following statements about your coalition.	Keep		Factor analysis – there is some cross-loading for this construct, mainly on two constructs but a few on other constructs as well. Given the importance of this variable for DFC, it may need to be modified. Overall reliability = .84.
		Our members are just learning the importance of environmental strategies.	Keep	.46
		We have had past success implementing policy and other environmental strategies.	Keep	.67
		The majority of our members are supportive of the coalition pursuing environmental strategies.	Keep	.59
		The majority of our members participate in implementing environmental strategies.	Keep	.61
		Our coalition does not have sufficient ability to advocate for policy changes.	Keep	.54
		Our coalition does not currently have the ability to implement an effective media campaign.	Keep	.45
		We have identified the policy and environmental changes relevant to our community.	Keep	.59
		We have the appropriate members to launch the needed policy change and environmental strategies.	Keep	.58
		We belong to other coalitions and networks in order to have a larger impact on policy and environmental strategies.	Keep	.43
		Our coalition uses an appropriate mix of environmental change strategies from providing information to changing consequences and physical design.	New		Added in to obtain information regarding using a mix of strategies targeting multiple levels.
Coalition Focus 2	Check one	Whatever your coalition’s focus, some coalitions act directly in the community (e.g., coordinating prevention programs and services or advocating for an environmental/policy change). Other coalitions act indirectly by building capacity of other organizations (through convening, incubating, training, or technical assistance). Which of the following do you think best describes the current status of your coalition?	Keep		This is another item that is attempting to assess the coalition’s specific focus – direct or indirect. Recommend keeping item.
		Our coalition exclusively acts directly (coordinates programs/services and pursues policy change ourselves).	Keep
		Our coalition primarily acts directly (coordinates programs/services and pursues policy change), though occasional attention is paid to building the capacity of other organizations.	Keep
		Our coalition is evenly split between acting directly and building the capacity of others to act (acting indirectly).	Keep
		Our coalition primarily acts indirectly (building the capacity of other organizations) though occasional attention is paid to acting directly to coordinate programs/services or pursue policy change.	Keep
		Our coalition exclusively acts indirectly (we build the capacity of other organizations to act).	Keep
CCT – Intermediary or Community support Org	Likert	Section IV: Some coalitions evolve into “intermediary or community support organizations.” The coalitions act mainly indirectly in the community by building the capacity of other organizations. They assess training and technical assistance needs and resources, build capacity where needed, develop a capacity building plan, provide an array of support services and evaluate impact on recipient organizations and the community.	Keep		This is one of four functional areas that comprise the coalition typology. Classifications have been extensively used in data analyses and should be kept for historical comparative purposes. Overall reliability = .92.
		Assessing needs*	Keep	.77
		Mobilizing and building capacity of other organizations*	Keep	.80
		Developing a comprehensive plan for capacity building*	Keep	.80
		Implementing or integrating capacity building strategies*	Keep	.78
		Evaluating capacity building efforts*	Keep	.75
		Planning for sustainability*	Keep	.71
Capacity Building Capacity	Check All That Apply	Please check all of the following resources your coalition has available for capacity building?	Modify		This is a decent checklist on some of the technical assistance and capacity building that coalitions could be providing to community-based organizations; however, need to put on a Likert Scale.
		Information on research-based prevention programs and strategies.	Keep
		Staff and consultants who are knowledgeable and skilled in capacity building methods (e.g., adult education, consulting).	Keep
		Staff and consultants are knowledgeable of evidence-based prevention programs, services and environmental strategies undertaken by the coalition.	Keep
		Ability to produce educational materials.	Keep
		Resources to send coalition members to workshops, conferences, or to visit other communities.	Keep
		Ability to distribute materials and other educational resources.	Keep
Synergy	Likert	What makes coalitions unique is their potential to combine the different perspectives, knowledge, and skills of a group of people and organizations. This unique combining power has been called synergy. We would like to know the extent to which you feel synergy is being created in your coalition. How much do you agree or disagree with each of the following statements.	Keep		Inspection of individual item content suggests that some of the items may be capturing related constructs (common language and common goals seems to be getting at shared vision). Depending on stakeholder feedback, this variable may be able to be deleted or modified. Factor analysis – good results (all items >.50). Overall alpha = .84.
		Our coalition has developed a common language for communication among diverse partners.	Keep	.62
		Our coalition has developed common goals that are understood and supported by all partners.	Keep	.63
		Our coalition is better able to carry out its work because of the contributions of diverse partners.	Keep	.63
		Our coalition has clearly communicated how its action will address problems that are important to people in the community.	Keep	.65
		Our coalition has committed the perspectives, resources and skills of partners.	Keep	.68
Perceived Effectiveness	Likert	Collective self-efficacy refers to a group’s shared belief in its joint capabilities to attain goals and accomplish desired tasks. We are interested in your assessment of the collective self-efficacy of the coalition. How much do you agree or disagree with each of the following statements.	Keep		In the CCT, this scale is called collective self-efficacy. Two items, which reflect connections to other organizations and larger entities, are proposed to be kept while three other more general items (e.g., “can work together as a group”) are proposed to be cut. Seven additional items, specific to the goals of the DFC community change initiative, are proposed to be included. Factor Analysis – Poor results, majority of items want to load on synergy, but display poor loadings.
		Our coalition has the support of other organizations and influential community leaders.	Keep
		Our coalition participates in larger coalitions and groups that can affect change at a higher level (e.g., State).	Keep
		Our coalition has a sufficient combination of skills within its membership to achieve the coalition’s goals.	Delete
		Our coalition can work together as a group and take collective action to achieve the coalition’s goals.	Delete
		Whenever our coalition has to move forward towards a goal, almost everyone is willing to join in and do their share of the work.	Delete
		Coalition activities and efforts have been successful in reducing substance abuse rates among youth in the community.	New
		Coalition activities and efforts have been successful in increasing community awareness of substance abuse issues in the community.	New
		The coalition has established ‘credibility’ in the community by their on-going efforts to reduce substance use rates among youth in the community.	New
		Coalition activities and efforts have been successful in increasing community-based approaches to reducing substance use.	New
		Coalition activities and efforts have been successful in increasing interagency collaborative efforts to address substance abuse issues.	New
		Coalition activities and efforts have been successful in increasing protective factors among youth in the community.	New
		Coalition activities and efforts have been successful in decreasing risk factors among youth in the community.	New
Coalition Self-Assessment	Check one	Coalitions develop as they increase their mastery of key functions needed to prevent substance abuse… For the work you need to do in order to be a successful coalition, how would you rate your coalition’s capacity to perform the key functions presented above?	Keep		Overall assessment of coalition functioning – keep.
		My coalition is learning how to perform most of these key functions.	Keep
		My coalition is achieving proficiency (competency) in performing most of the key functions to prevent substance abuse, though some we may be learning or have mastery (expertise).	Keep
		My coalition has achieved mastery (expertise) in performing most of the key functions needed to prevent substance abuse, however we could not sustain our coalition sufficiently without DFC funding	Keep
		My coalition has achieved mastery in most of the key functions we perform in the community. The capacities are sustainable and institutionalized (i.e., they can be supported without DFC funding and the capacities and mission are shared among many organizations).	Keep
	Text	Additional Comments	Keep
Member Satisfaction	Likert	Please indicate how much you agree or disagree with each of the following statements about your coalition.	New		This new variable is proposed to obtain an idea of how member satisfaction contributes to coalition functioning and could even be modeled as a basic output or outcome variable as well. Could be strengthened even further if we move to a multiple-informant data collection format.
		Members are satisfied with the coalition’s progress in addressing youth substance abuse issues.	New
		Members are satisfied with the degree of collaboration that has occurred as a result of the coalition’s activities and efforts.	New
		Members are satisfied with the progress made toward coalition goals and objectives.	New
Interorganizational Coordination	Likert	Please indicate how much you agree or disagree with each of the following questions about your coalition.	New		Given that one of the major goals of DFC is to increase collaboration, this variable is proposed to be included so we can capture whether or not organizations and agencies are indeed coordinating and collaborating to a greater extent.
		To what extent has the efforts of your coalition resulted in organizations and agencies working together more efficiently?	New
		To what extent has the efforts of your coalition resulted in members seeing their organization/agency as part of a broader system responding to youth drug abuse?	New
		To what extent has the efforts of your coalition increased the ability of organizations/agencies to coordinate their efforts?	New
		To what extent have the efforts of your coalition increased members’ knowledge of the strengths, as well as limitations, of each other’s organizations and agencies?	New
Organizational Impact	Likert	For my organization/agency, participation in the coalition has led to:	New		This variable assesses the degree of impact of coalition efforts and activities on member organizations. This could be a good “systems and community change” outcome variable – how has DFC efforts impacted the system -- that is, changed organizational/agency/community work for the benefit of youth and families?
		The generation of new ideas for improving our practices and/or services.	New
		The acquisition of useful knowledge about services, programs, or people in the community.	New
		A decrease in the number or severity of barriers we face in accomplishing our mission.	New
		A greater ability to identify the source of problems we encounter in order to come up with more effective solutions.	New
		A heightened public profile for my organization/agency.	New
		Increased our ability to affect public policy.	New
		Increased access to tools, best practices, and/or other information that has informed the work of my organization.	New
		Greater knowledge about how the system works and how organizations and agencies affect one another.	New
		An increase in our ability to find the answers to questions or problems that arise.	New
		An improvement in our ability to compete for grants and/or other funding opportunities.	New
		Increased utilizations of my organizations/agency’s expertise or services.	New
Community Social Organization	Likert	Please indicate how much you agree or disagree with each of the following statements about your coalition.	New		This variable is proposed in order to capture more about the community context and is an indicator of a community asset (social organization). Past research has demonstrated how this variable transmits neighborhood disadvantage to outcomes like substance use and delinquency.
		People in this community know each other.	New
		People in this community participate in social activities.	New
		People in this community feel connected to each other.	New
		People who live here feel they are part of a community.	New
		People who live here never do things to improve the community.	New
		People talk to each other about community problems.	New
		People in this community have a voice regarding important issues.	New
		Together, people in this community can persuade the city to respond to their needs and concerns.	New
		It is fairly safe to walk in this community at night.	New
		People in this community don’t trust each other.	New
		Residents don’t care about the community’s future.	New
		People in this community make it a safer place to live.	New
Collective Efficacy	Likert	Please indicate how much you agree or disagree with each of the following statements about your coalition.	New		Similar to community social organization, this variable, made famous by the project of human development in Chicago neighborhoods, obtains data on whether or not people feel comfortable in the neighborhood to intervene when kids are misbehaving and has been linked to lower levels of neighborhood violence.
		Children are skipping school and hanging out on a street corner.	New
		Children are spray-painting graffiti on a local building.	New
		Children are showing disrespect to an adult.	New
		Children are fighting in front of your house.	New
		Fire station closest to your home was threatened with budget cuts.	New
		People around here are willing to help their neighbors.	New
		This is a close knit neighborhood.	New
		People in this neighborhood can be trusted.	New
		People in this neighborhood generally don’t get along with each other.	New
		People in this neighborhood don’t share the same values.	New
Coalition Readiness	Likert	Please indicate how much you agree or disagree with each of the following statements about your coalition.	New		As discussed in the analysis plan, this variable obtains data on the degree to which coalition participants are ready and able for a comprehensive community change initiative like DFC.
		There is widespread knowledge about preventing drug abuse among participating agencies, organizations, and individuals.	New
		There is widespread support for the prevention of drug abuse among participating agencies, organizations, and individuals.	New
		There is a history of productive interaction among the stakeholders involved in designing and implementing the coalition’s drug abuse prevention efforts.	New
		Leaders of participating organizations are willing to commit resources, including staff time, for the coalition’s drug abuse prevention efforts.	New
		Financial resources (in addition to grant funds) are readily available to support the coalition’s drug abuse prevention efforts.	New
		Services and supports – formal and informal – are readily available in the community to support the coalition’s drug abuse prevention efforts.	New
Community Readiness	Likert	Please indicate how much you agree or disagree with each of the following statements about your coalition.	New		As discussed in the analysis plan, this variable obtains data on the degree to which community residents are ready and able for a comprehensive community change initiative that seeks to reduce substance use rates.
		There is widespread knowledge about the coalition’s drug abuse prevention efforts in the community.	New
		There is widespread support for the coalition’s drug abuse prevention efforts in the community.	New
		Community leaders are concerned about reducing substance abuse rates in the community.	New
		Community members and stakeholders are concerned about reducing substance abuse rates in the community.	New
		Existing programs within the community are conducive to developing interagency collaborative relationships.	New

1 Items marked with a “*” were used in the development of the typology of coalition maturity (i.e., Establishing, Functioning, Maturing, Sustaining). These items are being preserved in order to facilitate historical comparisons.

Office of National Drug Control Policy Page 0