Feb-2023 Climate Change Bundle Evaluation Instruments for AmeriCo

Climate Change Bundle Evaluation Instruments for AmeriCorps
Climate Change Evaluation Bundling

Submitted to:
AmeriCorps, 250 E St., SW

Washington, DC 20024

Submitted by:
ICF Incorporated, LLC 1902 Reston Metro Plz

Reston, VA 20190
https://www.icf.com

November 2022

Contents

AMERICORPS MEMBER SURVEY PROTOCOL 2

AMERICORPS PROJECT DIRECTOR SURVEY 10

AMERICORPS COMMUNITY PARTNER SURVEY 17

GRANTEE/SPONSOR ORGANIZATION – PROJECT DIRECTOR IN-DEPTH INTERVIEW (60 minutes) 23

GRANTEE/SPONSOR ORGANIZATION – PROGRAM STAFF IN-DEPTH INTERVIEW (60 minutes) 29

AMERICORPS MEMBER FOCUS GROUP (90 minutes) 34

COMMUNITY MEMBER FOCUS GROUP (60 minutes) 39

PARTNER ORGANIZATION IN-DEPTH INTERVIEW (60 minutes) 42

EVALUATION CAPACITY BUILDING PRE-/POST-PROGRAM ASSESSMENT 46

EVALUATION CAPACITY BUILDING PRESENTATION FEEDBACK SURVEYS 51

EVALUATION CAPACITY BUILDING OBSERVATION FORM 74

AMERICORPS MEMBER SURVEY PROTOCOL

AmeriCorps Member Survey Consent

Participation

Thank you for taking the time to participate in this survey. This survey is part of a study being conducted by ICF to help AmeriCorps and [program name] better understand how their programs are working. More specifically, we are interested in your program’s activities and effect on the community you serve. The survey will take about 20 minutes to complete.

Risks

We do not anticipate any risks in participating in this survey. Participation in this survey is voluntary. You can skip any question or stop answering questions at any time.

Your responses to the survey will be kept confidential. Your answers will not affect your current or future work with AmeriCorps. A report summarizing feedback across different organizations and AmeriCorps members will be shared with AmeriCorps and with grantees. The information that we use from this survey will not be identified with any one individual. Only ICF staff and the AmeriCorps Office of Research and Evaluation will have access to the de-identified survey data.

Questions

Should you have any questions about the survey, you may contact study representative Dr. Kimberly Good at Kimberly.Good@icf.com or Institutional Review Board representative [name].

If you agree to participate in the survey, please acknowledge below by selecting, “I agree to participate.”

• I agree to participate.

• I do not agree to participate. [If selected, will use skip logic take respondent to the end of the survey]

Background

1. This is a list of reasons people have for giving their time and efforts to others. Please indicate how important each of the following reasons are for you personally to volunteer.

	Not important	Slightly important	Important	Very important	Don’t know
It provides an opportunity to use your knowledge, skills, and experience.
It is something that will help your own neighborhood or community.
It is a way to make a difference for a cause or issue that is important to you OR in which you are interested.
It is a way to meet people or gather with people.
Because you feel a personal responsibility to help others when they need it.
It makes your own life more satisfying.
It lets you learn things through direct, hands-on experience.
It helps you to explore possible career options.

Source: These items are adapted from the Civic Engagement Volunteer Experience Survey and the Volunteer Motivation Scale.

Current Role and Services Provided

2. How long have you been with [grantee/sponsor name] as a national service member?

• Less than six months

• Six months – less than 1 year

• 1–5 years

• 6–10 years

• 11–15 years

• 16+ years

3. About how many hours per week do you serve as a member?

• Less than 5 hours

• 5–10 hours

• 11–20 hours

• 21–30 hours

• More than 30 hours

4. What types of services do you provide that are designed to impact climate change? Please select all of the services that your organization provides and indicate how long you have been providing the service.

		Length of Time (in months)
	Education or training (e.g., public outreach and education on environmental issues)
	Disaster response (e.g., clearing storm debris, distributing supplies)
	Community resilience (e.g., efforts to strengthen communities, communication, services)
	Energy efficiency (e.g., installing energy-efficient equipment in homes)
	Environmental stewardship/conservation (e.g., erosion control, invasive species removal)
	Mitigation activities (e.g., wildfire, flood)
	Other services (please specify): ________________________
	None
	Don’t know

5. For services connected to climate change that were selected in item 4, please describe (in greater detail) the activities that are being implemented to address climate change? ________________

________________________________________________________________________________

6. Why did you choose to serve in [program name]? _______________________________________

7. How would you describe the community that you are serving?

• City

• Suburb

• Town

• Rural area

8. Are you a member of the specific geographic community that you are serving?

• Yes

• No

Training

9. In general, how many hours of training have you received to prepare you to serve as a member?

• Less than 1 hour

• 1–4 hours

• 5–8 hours

• 9–16 hours

• 17+ hours
  
  
  

10. What was the content of the training? (Select all that apply.)

• Environmental sustainability (e.g., water conservation, reducing energy consumption)

• Renewable energy (e.g., wind or solar power)

• Technical skills (e.g., using new technology or tools)

• Communication skills (e.g., verbal communication, active listening)

• Problem solving skills (e.g., identifying a problem, exploring alternative solutions)

• Teamwork skills (e.g., team building, conflict resolution)

• Career preparation

• Other (please specify): ________________________

11. How helpful was the training for your role as a national service member in this program?

• Very helpful

• Helpful

• Somewhat helpful

• Not at all helpful

Program Outcomes

12. In the last 6 months, how often, if at all, have you engaged in the following climate change activities and actions. Please indicate the level to which you have participated on a scale from “Never” engaged to “Frequently” engaged.

	Never	Rarely	Sometimes	Frequently
Became involved with an environmental group or political party (e.g., volunteer, summer job, etc.).
Participated in an educational event (e.g., workshop) related to the environment.
Participated in a community event that focused on environmental awareness.
Participated in nature conservation efforts (e.g., planting trees, restoration of waterways).
Spent time working with a group/organization that deals with the connection of the environment to other societal issues such as justice or poverty.

Source: These items are adapted from the Environmental Action Scale.

13. Listed below are actions that one could take in the future to help reduce climate change. Please use the rating scale below to indicate how likely you would be to take the action.

	Definitely won’t	Probably won’t	Probably will	Definitely will
Vote on issues related to climate change.
Organize an educational event (e.g., workshop) related to climate change issues.
Use social media to raise awareness about climate change issues.
Personally write to or call a politician/ government official about climate change issues.
Organize a petition for an environmental cause.
Organize a community event to fight climate change.
Organize a community event to promote environmental justice.

Source: These items are adapted from the Environmental Action Scale.

14. The following items pertain to the impact of the program on your future education and career goals. Please indicate the extent to which participation in the program affected the following.

	Not at all	Very little	Somewhat	A great deal
Participation in this program enhanced my general employment skills (e.g., communication, problem solving, teamwork).
Participation in this program increased my ability to use new technology/tools.
Participation in this program improved my ability to work with people from different racial, ethnic, social, or educational backgrounds.
Participation in this program impacted the career/occupation I will pursue in the future.
Participation in this program influenced the degree/major I will pursue.

15. Do you have any interest in obtaining a job related to climate change?

• Yes

• No If no, go to item 17.
  
  
  

16. If yes, please rate your willingness to pursue a career in the following job categories?

	Definitely won’t	Probably won’t	Probably will	Definitely will
Agriculture and natural resource conservation
Clean transportation and fuels
Disaster services
Education, compliance, and awareness
Energy efficient product manufacturing, distribution, construction, installation, or maintenance
Natural and sustainable product manufacturing
Pollution prevention or environmental cleanup
Other (explain below)

If other was selected in item 16, please specify: ___________________________________________

17. The items in this section are designed to measure your perceptions of the effectiveness of [program name]. Please indicate the level at which you think the program is effective at addressing the following in the community. (If you do not have enough information to respond to the item, please select “Don’t know.”)

	Not Effective	Somewhat Effective	Effective	Very Effective	Don’t Know
Increase the use of renewable energy sources
Improve energy efficiency
Prepare for extreme weather and climate
Prepare for, respond to, or recover from natural disasters
Preserve public lands and/or waterways
Protect and/or restore biodiversity
Help the community become more resilient to a changing climate
Increase community engagement in climate change activities
Increase self-sufficiency among community members
Focus on the needs of vulnerable populations (economic, health, housing, transportation, social)
Increase community members’ awareness of resources to meet their basic needs (e.g., food, shelter, medical)
Increase community members’ knowledge about climate change
Increase community members’ positive attitudes toward activities that mitigate climate change
Increase community members’ behaviors around climate change
Recruit diverse program participants
Support local workforce development needs
Increase skills development toward future employment
Increase skills development toward green jobs

Demographics

18. What is your age?

• Under 18

• 18–29

• 30–39

• 40–49

• 50–59

• 60–69

• 70–79

• 80+

19. How do you describe your gender? (Select all that apply.)

• Male (including transgender men)

• Female (including transgender women)

• Nonbinary/nonconforming

• Prefer to self-describe as ____________

• Prefer not to say

20. Which one of these groups would you say best represents your race?

• White

• Black or African American

• American Indian or Alaska Native

• Asian

• Native Hawaiian or Pacific Islander

• Multi-racial or multi-ethnic (2+ races/ethnicities)

• Other (please specify): ____________

• Don’t know

• Prefer not to say

21. Are you of Hispanic, Latino/a, or Spanish origin?

• Yes

• No

• Don’t know

• Prefer not to say

22. What is the highest grade or year of school you completed?

• Never attended school or only kindergarten

• Elementary school

• Some high school

• High school graduate or equivalent

• Some college or technical school (If selected, go to item 23)

• College graduate (If selected, go to item 23)

• Some graduate school (If selected, go to item 23)

• Completed graduate school (If selected, go to item 23)

• Prefer not to say

23. What was your major area of study? ____________________________________________________

24. Did you grow up in an area that is similar to the community that you are serving?

• Yes If yes, go to item 26.

• No

25. How would you describe the community where you grew up?

• City

• Suburb

• Town

• Rural area

26. Do you have any other comments? _____________________

AMERICORPS PROJECT DIRECTOR SURVEY

Grantee/Sponsor Project Director Survey Consent

Participation

Thank you for taking the time to participate in this survey. This survey is part of a study being conducted by ICF on behalf of AmeriCorps and [program/project name] to better understand how their programs/projects are working. More specifically, we are interested in your program/project’s activities and effect on the community you serve. The survey will take about 20 minutes to complete.

Risks

We do not anticipate any risks in participating in this survey. Participation in this survey is voluntary. You can skip any question or stop answering questions at any time.

Your responses to the survey will be kept confidential. Your answers will not affect your current or future work with AmeriCorps. A report summarizing feedback across different organizations and AmeriCorps members will be shared with AmeriCorps. The information that we use from this survey will not be identified with any one individual. Only ICF staff and the AmeriCorps Office of Research and Evaluation will have access to the de-identified survey data.

Questions

Should you have any questions about the survey, you may contact study Dr. Kimberly Good at Kimberly.Good@icf.com or Institutional Review Board representative [name].

If you agree to participate in the survey, please acknowledge below by selecting, “I agree to participate.”

• I agree to participate.

• I do not agree to participate. [If selected, will use skip logic take respondent to the end of the survey]

Background

1. What is your current role at [organization name]?

• Program assistant

• Program manager

• Director

• Other (please specify): ____________

2. How long have you been with [organization name]?

• Less than 1 year

• 1–5 years

• 6–10 years

• 11–15 years

• 16+ years

3. How long have you been with [organization name] in your current role?

• Less than 1 year

• 1–5 years

• 6–10 years

• 11–15 years

• 16+ years

4. How many national service members does your program oversee? ____________

National Service Member Training

5. How many hours of training do national service members receive?

• Less than 1 hour

• 1–4 hours

• 5–8 hours

• 9–16 hours

• 17+ hours

6. What is the content of the training? (Select all that apply.)

• Environmental sustainability (e.g., water conservation, reducing energy consumption)

• Renewable energy (e.g., wind or solar power)

• Technical skills (e.g., using new technology or tools)

• Communication skills (e.g., verbal communication, active listening)

• Problem solving skills (e.g., identifying a problem, exploring alternative solutions)

• Teamwork skills (e.g., team building, conflict resolution)

• Career preparation

• Other (please specify): ________________________

Services and Activities

7. In general, what types of services does your organization provide? (Select all that apply.)

• Education (e.g., tutoring, literacy, education for children and youth)

• Health (e.g., nutrition, access to care, prevention, awareness, mental health)

• Services to veterans, members of the armed forces, or their families

• Environment (e.g., education on environmental issues, land conservation, energy conservation, ecosystem development/maintenance)

• Disaster services (e.g., disaster education, preparation, mitigation, response)

• Economic opportunity (e.g., housing services, employment counseling, job training, financial literacy, financial assistance)

• Other services (please specify): ____________________________________
  
  

8. Who does your organization serve? Please select the populations that your organization serves. (Select all that apply.)

• Formerly Incarcerated

• Homeless

• Immigrant (documented and undocumented)

• Individuals with disabilities

• Low income

• Non-English speaking or limited English proficiency

• Older adults

• Racially/ethnically diverse

• Refugee

• Substance dependent

• Veterans

• Other: ________________________

9. What types of services do you provide that are designed to impact climate change? Please select all of the services that your organization provides and indicate how long you have been providing the service (e.g., 2 years, 6 months).

		Length of Time (in months)
	Education or training (e.g., public outreach and education on environmental issues)
	Disaster response (e.g., clearing storm debris, distributing supplies)
	Community resilience (e.g., efforts to strengthen communities, communication, services)
	Energy efficiency (e.g., installing energy-efficient equipment in homes)
	Environmental stewardship/conservation (e.g., erosion control, invasive species removal)
	Mitigation activities (e.g., wildfire, flood)
	Other services (please specify): ________________________
	None
	Don’t know

10. For services connected to climate change that were selected in item 9, please describe (in greater detail) the activities that are being implemented to address climate change? ___________________

__________________________________________________________________________________

__________________________________________________________________________________

11. How has the COVID-19 pandemic impacted the services and activities provided by your organization? ______________________________________________________________________

__________________________________________________________________________________

__________________________________________________________________________________

Partnerships

12. Does your organization partner with other organizations to provide activities or services for this program?

• Yes

• No

If yes is selected, go to item 13. If no is selected, go to item 22.

13. Please specify the number of organizations that your organization works with to provide activities or services for this program: ____________

If you worked with more than one partner, please think about one partnership that was successful and another partnership that was challenging when responding to the following questions. You will have the opportunity to answer each question twice to provide information about each partnership.

(Questions 14-21 will be shown twice so the grantee/sponsor can provide information about two organizations; they will have the option to skip the second series of questions if they have only one partner example.)

14. Partner organization name: ­­­­­­­­­­­­­­­­­­­____________________________________________________

15. What types of activities or services does your partner organization provide for this program? (Select all that apply.)

• Recruiting national service members or volunteers

• Curriculum design for training

• Identifying knowledge and skills for job requirements

• Training national service members or volunteers to develop required skills

• Providing facilities or equipment for training activities

• Providing equipment or software to support program success

• Coordinating program activities

• Supporting national service members or volunteers while on the job

• Other (please specify): ________________________

16. Prior to the current program, how often did your organization work with your partner organization?

• Never

• Rarely

• Occasionally

• Frequently

• Always

• Don’t know

17. Overall, how satisfied are you with the relationship with your partner?

• Very satisfied

• Satisfied

• Neither satisfied nor dissatisfied

• Dissatisfied

• Very dissatisfied

18. How would you classify your organization’s relationship with your partner organization?

• We have a true partnership; we share strategies, resources, and information and jointly make decisions about the program.

• We coordinate with the organization and make some of our decisions together about the program but the roles and responsibilities are clearly defined.

• We cooperate with the organization to implement the program; we consider this organization when making decisions, but do not make decisions together.

• We have limited communication; the organization does not influence our decisions and we do not influence their decisions about the program.

19. How would you describe your relationship with this partner organization?

• Challenging

• Successful

20. Please elaborate on the description that you gave to the relationship with your partner. Why was the relationship challenging or successful? ______________________________________________

_________________________________________________________________________________

21. The partnership between my organization and our partners will continue beyond the life of the grant period. (Using the selections below, please rate how strongly you agree with this statement.)

• Strongly agree

• Agree

• Neither agree nor disagree

• Disagree

• Strongly disagree

Program Outcomes

The items in this section are designed to measure your perceptions of the effectiveness of [program name].

22. Please indicate the level to which you think the program is effective at addressing the following in the community. If you do not have enough information to respond to the item, please select “Don’t know.”

	Not Effective	Somewhat Effective	Effective	Very Effective	Don’t Know
Increase the use of renewable energy sources
Improve energy efficiency
Prepare for extreme weather and climate
Prepare for, respond to, or recover from natural disasters
Preserve public lands and/or waterways
Protect and/or restore biodiversity
Help the community become more resilient to a changing climate
Increase community engagement in climate change activities
Increase self-sufficiency among community members
Focus on the needs of vulnerable populations (economic, health, housing, transportation, social)
Increase community members’ awareness of resources to meet their basic needs (e.g., food, shelter, medical)
Increase community members’ knowledge about climate change
Increase community members’ positive attitudes toward activities that mitigate climate change
Increase community members’ behaviors around climate change
Recruit diverse program participants
Support local workforce development needs
Increase skills development toward future employment
Increase skills development toward green jobs

23. Please indicate the level to which you think the program is effective at addressing the following for national service members. If you do not have enough information to respond to the item, please select “Don’t know.”

	Not Effective	Somewhat Effective	Effective	Very Effective	Don’t Know
Attitudes and behavior toward civic engagement
Future career goals
Future education goals
Job skills development
Skills development toward green jobs
Knowledge of the impacts of climate change
Activities that reduce greenhouse gas emissions
Behavior to become more resilient to a changing climate
Behavior to prepare for extreme weather and climate
Behavior to become more resilient to extreme events

24. In your opinion, what is contributing to achieving the desired outcomes in your community? ________________________________________________________________________________

25. What are the barriers to achieving the intended outcomes in your community? ________________ _________________________________________________________________________________

26. What recommendations do you have for other organizations that are planning to implement a similar program? __________________________________________________________________ _________________________________________________________________________________

27. Do you have any other comments? _____­­­­______________________________________________

AMERICORPS COMMUNITY PARTNER SURVEY

Partner Organization Survey Consent

Participation

Thank you for taking the time to participate in this survey. This survey is part of a study being conducted by ICF to help AmeriCorps and [program name] better understand how their programs are working. More specifically, we are interested in your organization’s activities, partnership with [program name], and effect on the community you serve. The survey will take about 20 minutes to complete.

Risks

We do not anticipate any risks in participating in this survey. Participation in this survey is voluntary. You can skip any question or stop answering questions at any time.

Your responses to the survey will be kept confidential. Your answers will not affect your current or future work with [program name] or AmeriCorps. A report summarizing feedback across different organizations and AmeriCorps members will be shared with AmeriCorps. The information that we use from this survey will not be identified with any one individual. Only ICF staff and the AmeriCorps Office of Research and Evaluation will have access to the de-identified survey data.

Questions

Should you have any questions about the survey, you may contact study representative Dr. Kimberly Good at Kimberly.Good@icf.com or Institutional Review Board representative [name].

If you agree to participate in the survey, please acknowledge below by selecting, “I agree to participate.”

• I agree to participate.

• I do not agree to participate. [If selected, will use skip logic take respondent to the end of the survey]

Note: ICF will ask the grantee/sponsor to select 1–2 key partners to complete this survey.

Background

1. What is your current role at [organization name]?

• Organizational leadership (e.g., executive director, president, CEO)

• Office or program director

• Office or program staff

• Administrative staff (e.g., finance, development, human resources)

• Volunteer

• Other (please specify): ____________

2. How long have you been with [organization name]?

• Less than 1 year

• 1–5 years

• 6–10 years

• 11–15 years

• 16+ years

3. How long have you been with [organization name] in your current role?

• Less than 1 year

• 1–5 years

• 6–10 years

• 11–15 years

• 16+ years

4. What is your organization’s sector? (Select all that apply.)

• Private, for-profit business

• Public / government / Tribal organization, office, or agency

• Nonprofit organization

• Religious or other spiritual organization

• School (K–12)

• College or university

• Other: ________________________

5. In general, what types of services does your organization provide? (Select all that apply.)

• Education (e.g., tutoring, literacy, education for children and youth)

• Health (e.g., nutrition, access to care, prevention, awareness, mental health)

• Services to veterans, members of the armed forces, or their families

• Environment (e.g., education on environmental issues, land conservation, energy conservation, ecosystem development/maintenance)

• Disaster services (e.g., disaster education, preparation, mitigation, response)

• Economic opportunity (e.g., housing services, employment counseling, job training, financial literacy, financial assistance)

• Other services (please specify): ____________________________________
  
  
  

6. Who does your organization serve? Please select the populations that your organization serves. (Select all that apply.)

• Formerly Incarcerated

• Homeless

• Immigrant (documented and undocumented)

• Individuals with disabilities

• Low-income

• Non-English speaking or limited English proficiency

• Older adults

• Racially/ethnically diverse

• Refugee

• Substance dependent

• Veterans

• Other: ________________________
  
  
  

7. What types of services does your organization provide that are designed to impact climate change? Please select all of the services that your organization provides and indicate how long you have been providing the service (e.g., 2 years, 6 months).

	Service	Length of Time
	Education or training (e.g., public outreach and education on environmental issues)
	Disaster response (e.g., clearing storm debris, distributing supplies)
	Community resilience (e.g., efforts to strengthen communities, communication, services)
	Energy efficiency (e.g., installing energy efficient equipment in homes)
	Environmental stewardship/conservation (e.g., erosion control, invasive species removal)
	Mitigation activities (e.g., wildfire, flood)
	Other services (please specify): ________________________
	None
	Don’t know

8. For services connected to climate change that were selected in item 7, please describe (in greater detail) the activities that are being implemented to address climate change?

______________________________________________________________________________

______________________________________________________________________________

9. How has the COVID-19 pandemic impacted the services and activities provided by your organization? ______________________________________________________________________________

______________________________________________________________________________

Partner Services

10. What types of activities or services does your organization provide for this program? (Select all that apply.)

• Recruiting national service members or volunteers

• Curriculum design for training

• Identifying knowledge and skills for job requirements

• Training national service members or volunteers to develop required skills

• Providing facilities or equipment for training activities

• Providing equipment or software to support program success

• Coordinating program activities

• Supporting national service members or volunteers while on the job

• Other (please specify): ________________________

11. Prior to the current program, how often did your organization work with [grantee organization name]?

• Never

• Rarely

• Occasionally

• Frequently

• Always

• Don’t know

12. Overall, how satisfied are you with the relationship with [grantee organization name]?

• Very satisfied

• Satisfied

• Neither satisfied nor dissatisfied

• Dissatisfied

• Very dissatisfied

13. How would you classify your organization’s relationship with [grantee organization name]?

• We have a true partnership; we share strategies, resources, and information and jointly make decisions about the program.

• We coordinate with the organization and make some of our decisions together about the program but the roles and responsibilities are clearly defined.

• We cooperate with the organization to implement the program; we consider this organization when making decisions but do not make decisions together.

• We have limited communication; the organization does not influence our decisions and we do not influence their decisions about the program.

14. Please rate how strongly you agree with this statement: The partnership between my organization and [grantee organization name] will continue beyond the life of the grant period.

• Strongly agree

• Agree

• Neither agree nor disagree

• Disagree

• Strongly disagree

Program Outcomes

The items in this section are designed to measure your perceptions of the effectiveness of [program name].

15. Please indicate the level to which you think the program is effective at addressing the following in the community. If you do not have enough information to respond to the item, please select “Don’t know.”

	Not Effective	Somewhat Effective	Effective	Very Effective	Don’t Know
Increase the use of renewable energy sources
Improve energy efficiency
Prepare for extreme weather and climate
Prepare for, respond to, or recover from natural disasters
Preserve public lands and/or waterways
Protect and/or restore biodiversity
Helps the community become more resilient to a changing climate
Increase community engagement in climate change activities
Increase self-sufficiency among community members
Focus on the needs of vulnerable populations (economic, health, housing, transportation, social)
Increase community members’ awareness of resources to meet their basic needs (e.g., food, shelter, medical)
Increase community members’ knowledge about climate change
Increase community members’ positive attitudes toward activities that mitigate climate change
Increase community members’ behaviors around climate change
Recruit diverse program participants
Support local workforce development needs
Increase skills development toward future employment
Increase skills development toward green jobs

16. In your opinion, what is contributing to achieving the desired outcomes in your community? ____________________________________________________________________________________________________________________________________________________________________

17. What are the barriers to achieving the intended outcomes in your community? ____________________________________________________________________________________________________________________________________________________________________

18. What recommendations do you have for other organizations who are planning to implement a similar program? __________________________________________________________________________________ __________________________________________________________________________________

19. Do you have any other comments? ____________________________________________________________________________________________________________________________________________________________________

GRANTEE/SPONSOR ORGANIZATION – PROJECT DIRECTOR IN-DEPTH INTERVIEW (60 minutes)

(Research question addressed is in parentheses)

Grantee/Sponsor Project Director Interview Consent

Participation

Thank you again for taking the time to participate in today’s interview. This interview is part of a study being conducted by ICF for AmeriCorps. The reason for the study is to help AmeriCorps and [program name] better understand how their programs are working. More specifically, we are interested in your program/project’s activities and effect on the community you serve. We have prepared a number of discussion questions, but please feel free to offer any additional thoughts you have about your experience. The interview will take up to 60 minutes.

Risks

We do not anticipate any risks in being a part of this interview. Participation in today’s discussion is completely up to you. You can decide you don’t want to answer any question we ask, and you can stop participating in the discussion at any time. You can also decide not to participate at all and leave the interview at any point.

Anything you share with ICF will be kept confidential. By speaking with us, you will not affect your current or future relationship with AmeriCorps. A report summarizing feedback across different organizations and AmeriCorps members will be shared with AmeriCorps. Only ICF staff will see the data collected from this interview, and the information that we use from our discussions will not be identified with any one individual. However, since we are only talking to a small number of grantee staff at each organization, there is a chance that AmeriCorps personnel will be able to guess which individual shared certain information.

Do you have any questions about this study or this interview?

Before we begin, we would also like to check that you are 18 years or older and get your permission to participate in today’s discussion. If you do not want to participate, you may leave at this time.

Consent Statement

1. You are 18 years or older.

2. You understand being a part of this study is completely up to you and that you can stop being a part of the discussion at any time, with no penalty or risk.

3. You understand that only ICF staff will see your answers to these questions.

4. You understand that your name will not be included in any reports or presentations of the results and that what you share with us today will be treated as confidential.

5. You understand the possible risks and benefits of being a part of this study.

Questions

Should you have any questions about the interview, you may contact study representative Dr. Kimberly Good at Kimberly.Good@icf.com, or Institutional Review Board representative [name].

Background

1. Let’s begin with some brief introductions. Please tell me a little bit about yourself.

   1. What is your title and role at [organization name]?

   2. How long have you been in this position?

   3. What are your specific responsibilities?

2. Can you tell me a little about [organization name]? What types of services and activities does your organization provide?

3. Were you required to provide matching funds to receive AmeriCorps funding for the [program name]? If yes, to what extent were the required matching funds easily secured? Explain.

Program Operations and Context

I’d like to ask you a few questions about your program, [program name], and the community that you serve.

4. What are the mission and goals of [program name]? (1)

1. What are program priorities? (1.a)

2. In what ways has climate change affected your community and its members? (8)

• Probe for economic, health, housing, transportation, and social impacts

1. Can you discuss some of the services and activities that address climate change issues in your community? (1.b)

2. How are you measuring program impacts? (1.c)

• Use the probes below based on the specific program

• Capacity Building
  Number of organizations that received capacity building services/increased their effectiveness, efficiency, and/or program scale/reach

• Disaster Services
  Number of individuals served; number of individuals reporting increased disaster readiness; number of structures protected or restored/returned to regular use after a disaster

• Environmental Stewardship
  Number of housing units or public structures weatherized or retrofitted to improve energy efficiency/with reduced energy consumption or reduced energy costs; number of individuals receiving education or training in environmental stewardship and/or environmentally-conscious practices/with increased knowledge of environmental stewardship and/or environmentally-conscious practices; number reporting a change in behavior or the intent to change behavior to better protect the environment; number of acres of public parks or other public and Tribal lands that are treated/improved; number of miles of public trails or waterways that are treated and/or constructed/improved and/or put into use

1. What are some challenges that you’ve encountered with the AmeriCorps performance measures? This includes outputs and outcomes.

5. How does your program define equity? (2)

1. Does the program have goals around equity and inclusion? If yes, please tell us about them and how they are measured.

2. Can you describe the strategies that your program uses to increase equity?

6. What are the characteristics of the communities and individuals served through the program? (9.a)

1. Can you speak to the role of the community in this program?

1. How were community members notified about the program?

2. How are community members involved in program activities? (10.a) Are they actively involved in program activities? What role do they have in the program?

3. Were the community members involved in program planning? If yes, please tell us about the process.

7. Thinking about how this program was structured and implemented, is the program operating as intended? (3)

1. How did the COVID-19 pandemic affect program planning and implementation? (12)

1. How did your organization adapt during the pandemic? In what ways are you adapting program activities and services? (12.b)

2. Was the training of national service members adapted? If so, how? (12.a)

3. What adaptations will you retain moving forward? Why? (12.c)

8. What aspects of the program do you think are most effective? What do you think could be improved in how the program was structured and implemented? (4)

1. What practices have been most effective for engaging vulnerable and/or diverse communities? (4.a) Is there anything that you would do differently to increase community engagement?

2. What practices have been most effective for increasing equity? What gaps still need to be addressed?

9. What barriers has the program faced while trying to meet intended outcomes? (5)

10. What facilitators have assisted the program in meeting intended outcomes? (5)

11. Considering the goal of addressing climate change, what successful strategies or innovations did your program implement that could be transferable to other communities or organizations wanting to implement this type of program? (6)

Partnerships

12. What are the number and types of partners that your organization is working with on [program name]?

1. Do you have any partnerships that focus on resilience building for at-risk populations within your community? (Examples may include racial minority groups, populations with limited English proficiency, disabled populations, and any other identified populations within your community.) (14)

13. Can you please tell me how you started working with [partner organization name]? How long have you been working with them? (16)

1. How has [organization name] maintained a relationship with [partner organization name] during that time? (16)

2. How often do you engage with your partners? (14)

1. How often do you work with your partners on joint activities?

14. How would you describe the relationship between the organizations? (15)

15. How is your organization working with [partner organization name] to build community resilience? (16)

Program Outcomes

16. In what ways does your program improve member outcomes? (19 & 21)

• Probe for increased civic engagement (19.a)

• Probe for increased knowledge of the impacts and solutions to climate change, positive attitudes toward activities that mitigate climate change, and behaviors around climate change

• Probe for an increase in functional and technical job skills, interest/willingness to pursue a career in a green job (21.a, 21.b)

17. From your perspective, in what ways has [program name] helped the community address climate change?

• Probe on the following and follow up with how

• Energy efficiency/renewable energy sources (18.a)

• Disaster response (18.b)

• Environmental conservation (18.c)

• Improved knowledge, attitudes, and behaviors around climate change (18.d)

• Built community capacity to mitigate and adapt to climate change (18.e)

18. In what ways has your program increased equity in the community?

• Probes include increased education, engagement, self-sufficiency, and partnerships

Program Sustainability

19. From your perspective, how likely is it that you will be able to sustain the program after the grant-funding period? (7)

3. Which components will you be able to sustain?

20. Do you have any additional feedback or insights you would like to share with us regarding the program?

GRANTEE/SPONSOR ORGANIZATION –
PROGRAM STAFF IN-DEPTH INTERVIEW
(60 minutes)

Grantee Staff Interview Consent

Participation

Thank you again for taking the time to participate in today’s interview. This interview is part of a study being conducted by ICF for AmeriCorps. The reason for the study is to help AmeriCorps and [program name] better understand how their programs are working. More specifically, we are interested in your program’s activities and effect on the community you serve. We have prepared a number of discussion questions, but please feel free to offer any additional thoughts you have about your experience. The interview will take up to 60 minutes.

Risks

We do not anticipate any risks in being a part of this interview. Participation in today’s discussion is completely up to you. You can decide you don’t want to answer any question we ask you, and you can stop participating in the discussion at any time. You can also decide not to participate at all and leave the interview at any point.

Anything you share with ICF will be kept confidential. By speaking with us, you will not affect your current or future relationship with AmeriCorps. A report summarizing feedback across different organizations and AmeriCorps members will be shared with AmeriCorps. Only ICF staff will see the data collected from this interview, and the information that we use from our discussions will not be identified with any one individual. However, since we are only talking to a small number of grantee staff at each organization, there is a chance that AmeriCorps personnel will be able to guess which individual shared certain information.

Do you have any questions about this study or this interview?

Before we begin, we would also like to check that you are 18 years or older and get your permission to participate in today’s discussion. If you do not want to participate, you may leave at this time.

Consent Statement

1. You are 18 years or older.

2. You understand being a part of this study is completely up to you and that you can stop being a part of the discussion at any time, with no penalty or risk.

3. You understand that only ICF staff will see your answers to these questions.

4. You understand that your name will not be included in any reports or presentations of the results and that what you share with us today will be treated as confidential.

5. You understand the possible risks and benefits of being a part of this study.

Questions

Should you have any questions about the interview, you may contact study representative Dr. Kimberly Good at Kimberly.Good@icf.com, or Institutional Review Board representative [name].

Background

1. Let’s begin with some brief introductions. Please tell me a little bit about yourself.

   1. What is your title and role at [organization name]?

   2. How long have you been in this position?

   3. What are your specific responsibilities?

Program Operations and Context

2. How have you been involved with [program name]?

   1. In what ways has climate change affected your community and members of the community? (8)

• Probe for economic, health, housing, transportation, and social impacts

1. Can you discuss some of the services and activities that address climate change? (1.b)

2. Can you discuss how the program attempts to increase equity in terms of climate change outcomes? (2)

3. Do you work with the national service members? How are the members recruited? Can you tell us about their activities and your interactions with the members?

• Probe for information about the characteristics of the national service members

1. Do you work with members of the community? How are they involved in the program?

• Probe for information about the characteristics of the community and its members; probe for community members’ role in program activities

1. What practices have you found to be most effective at engaging vulnerable and/or diverse communities? (4.a)

1. How did your organization adapt during the COVID-19 pandemic? (12)

2. What aspects of the program do you think are most effective? What do you think could be improved in how the program was structured and implemented? (4)

3. How are outcomes related to climate change measured? (1.c)

• Use the probes below based on the specific program

  • Capacity Building
    Number of organizations that received capacity building services/increased their effectiveness, efficiency, and/or program scale/reach

  • Disaster Services
    Number of individuals served; number of individuals reporting increased disaster readiness; number of structures protected or restored/returned to regular use after a disaster

  • Environmental Stewardship
    Number of housing units or public structures weatherized or retrofitted to improve energy efficiency/with reduced energy consumption or reduced energy costs; number of individuals receiving education or training in environmental stewardship and/or environmentally-conscious practices/with increased knowledge of environmental stewardship and/or environmentally-conscious practices; number reporting a change in behavior or the intent to change behavior to better protect the environment; number of acres of public parks or other public and Tribal lands that are treated/improved; number of miles of public trails or waterways that are treated and/or constructed/improved and/or put into use

1. What are some challenges that you’ve encountered with the AmeriCorps performance measures? This includes outputs and outcomes.

6. From your perspective, what challenges did your organization encounter in meeting intended outcomes? (5)

7. What were the most successful strategies that you observed to meet intended outcomes? (5)

8. Considering the goal of addressing climate change, what successful strategies or innovations did your program implement that could be transferable to other communities or organizations wanting to implement this type of program? (6)

Working with National Service Members

9. Does [organization name] offer members any additional training? If so, what types of training? (20, 21.a)

• Probe for member skill development toward green jobs and willingness to pursue a career in conservation and environmental education fields (21.a, 21.b)

10. What have you found to be best practices for engaging members? What are the challenges? (4)

1. Do you have any specific practices to effectively support members from diverse backgrounds? (4.a)

• Probe for diversity, equity, and inclusion (DE&I) commitments, affinity groups, identity-specific outreach events, and so forth

Working with Partner Organizations

11. Does [organization name] work with community partners? If so, can you please share what type of collaboration you have had with these partners in your work on [program name]? (14, 15)

    1. How often do you engage with your partners? (15)

12. How often do you work with these partners on joint activities?

1. How has [organization name] built and maintained these partnerships? (16)

2. Do you think your work with partners helps build community resilience? If so, how? (17)

Program Outcomes

13. From your perspective, in what ways has [program name] helped the community address climate change?

• Probe on the following and follow up with how

• Energy efficiency/renewable energy sources (18.a)

• Disaster response (18.b)

• Environmental conservation (18.c)

• Improved knowledge, attitudes, and behaviors around climate change (18.d)

• Built community capacity to mitigate and adapt to climate change (18.e)
  
  
  

14. In what ways has your program increased equity in the community?

• Probes include increased education, engagement, self-sufficiency, and partnerships

15. In what ways has [program name] improved member outcomes? (19 & 21)

• Probe for increased civic engagement (19.a)

• Probe for members’ increase in functional and technical job skills, interest/willingness to pursue a career in a green job (21.a, 21.b)

• Probe for previous members’ job attainment and careers in green jobs after their service (21.c, 21.d)

16. Do you have any additional feedback or insights you would like to share with us regarding the program?

AMERICORPS MEMBER FOCUS GROUP
(90 minutes)

AmeriCorps Member Focus Group Consent

Participation

Thank you again for taking the time to participate in today’s focus group. This focus group is part of a study being conducted by ICF for AmeriCorps. The reason for the study is to help AmeriCorps and [program name] better understand how their programs are working. More specifically, we are interested in your program’s activities and effect on the community you serve. We have prepared a number of discussion questions, but please feel free to offer any additional thoughts you have about your experience. The focus group will take up to 60 minutes.

Risks

We do not anticipate any risks in being a part of this focus group. Participation in today’s discussion is completely up to you. You can decide you don’t want to answer any question we ask you, and you can stop participating in the discussion at any time. You can also decide not to participate at all and leave the focus group at any point.

Anything you share with ICF will be kept confidential. By speaking with us, you will not affect your current or future relationship with [program name]. A report summarizing feedback across different organizations and AmeriCorps Members will be shared with AmeriCorps. Only ICF staff will see the data collected from this focus group, and the information that we use from our discussions will not be identified with any one individual. However, since we are only talking to a small number of members at each organization, there is a chance that AmeriCorps or [program name] personnel will be able to guess which individual shared certain information.

We also request that you do not discuss what is disclosed in this focus group once the discussion ends.

Do you have any questions about this study or this focus group?

Before we begin, we would also like to check that you are 18 years or older and get your permission to participate in today’s discussion. If you do not want to participate, you may leave at this time.

Consent Statement

1. You are 18 years or older.

2. You understand being a part of this study is completely up to you and that you can stop being a part of the discussion at any time, with no penalty or risk.

3. You understand that only ICF staff will see your answers to these questions.

4. You understand that your name will not be included in any reports or presentations of the results and that what you share with us today will be treated as confidential.

5. You understand the possible risks and benefits of being a part of this study.

Questions

Should you have any questions about the focus group, you may contact study representative Dr. Kimberly Good at Kimberly.Good@icf.com, or Institutional Review Board representative [name].

Let’s begin with some brief introductions. Please tell us your name and a little bit about yourself.

Background

1. Why did you become a national service member?

2. To what extent were you engaged in community activities prior to your service?

3. Why did you choose to serve in [program name]? How did you become aware of the program?

4. Are you a member of the community that you are serving?

5. How did the COVID-19 pandemic affect you and your community? (13)

Program Context and Activities

6. How does climate change impact your life? In what ways has climate change affected your community and members of the community? (8)

• Probe for economic, health, housing, transportation, and social impacts

7. What is your understanding of the mission and goals of [program name]? In your own words, what is the program designed to accomplish?

   • What activities and services are intended to address climate change? (1.b)

8. Can you discuss how the program attempts to increase equity in terms of climate change outcomes? (2)

9. How have you been involved with [program name]?

   • What is your role? What services did you provide? What activities did you engage in during your service?

10. Do you work with members of the community? How are they involved in the program?

• Probe for information about the characteristics of the community and its members; probe for community members’ role in program activities (10)

11. How did the COVID-19 pandemic affect program activities and services? (12)

Training and Supports

12. Overall, how prepared were you for your position as a national service member?

    • Now please think about your specific job tasks—for example, conducting community outreach or retrofitting a building. How prepared did you feel to perform these tasks?

13. After the training, how knowledgeable were you about the impacts of and solutions to climate change?

14. Were you provided a living allowance and/or stipend for participating in the program? If yes, were these monetary supports sufficient to meet your living needs? Why or why not?

Program Outcomes

15. In what ways has [program name] affected you personally? (19 & 21)

• Probe for increased civic engagement (19.a)

• Probe for increased knowledge, attitudes, and behaviors around climate change

• Probe for an increase in functional and technical job skills, interest/willingness to pursue a career in a green job (19.a, 19.b)

16. Following your experience in the program, would you be interested in pursuing additional opportunities to positively engage with your community? (19)

    • Examples may include volunteer work, contributions to charitable organizations within your community, participation in discussions concerning social/environmental responsibility, and so forth.

17. Do you have any interest in obtaining a job related to climate change? (21.b)

• If yes, probe to find out what area (e.g., energy, conservation)

• If yes, probe to find out if they will have need to pursue more schooling and in what field of study

18. Can you tell me about how your experience has affected your preparedness for a career within the climate change field? Have you developed necessary skills for a career within climate change? (20, 21.a)

    • If not, has your experience within the program increased your awareness of pathways to obtain necessary skills for a career within climate change?

19. From your perspective, in what ways has [program name] helped the community address climate change?

• Probe on the following and follow up with how

• Energy efficiency/renewable energy sources (18.a)

• Disaster response (18.b)

• Environmental conservation (18.c)

• Improved knowledge, attitudes, and behaviors around climate change (18.d)

• Built community capacity to mitigate and adapt to climate change (18.e)

Final Thoughts

20. Do you have any suggestions to improve [program name]? (6)

• Probe for training/onboarding deficiencies, member-level challenges, etc.

21. Do you have any additional feedback or insights you would like to share with us regarding your experiences in the program or working with [organization name]?

COMMUNITY MEMBER FOCUS GROUP
(60 minutes)

Community Member Focus Group Consent

Participation

Thank you again for taking the time to participate in today’s focus group. This focus group is part of a study being conducted by ICF for AmeriCorps. The reason for the study is to help AmeriCorps and [program name] better understand how their programs are working. More specifically, we are interested in your program’s activities and effect on the community you serve. We have prepared a number of discussion questions, but please feel free to offer any additional thoughts you have about your experience. The focus group will take up to 60 minutes.

Risks

We do not anticipate any risks in being a part of this focus group. Participation in today’s discussion is completely up to you. You can decide you don’t want to answer any question we ask you, and you can stop participating in the discussion at any time. You can also decide not to participate at all and leave the focus group at any point.

Anything you share with ICF will be kept confidential. By speaking with us, you will not affect your current or future relationship with [program name]. A report summarizing feedback across different organizations and AmeriCorps Members will be shared with AmeriCorps. Only ICF staff will see the data collected from this focus group, and the information that we use from our discussions will not be identified with any one individual. However, since we are only talking to a small number of members at each organization, there is a chance that AmeriCorps or [program name] personnel will be able to guess which individual shared certain information.

We also request that you do not discuss what is disclosed in this focus group once the discussion ends.

Do you have any questions about this study or this focus group?

Before we begin, we would also like to check that you are 18 years or older and get your permission to participate in today’s discussion. If you do not want to participate, you may leave at this time.

Consent Statement

1. You are 18 years or older.

2. You understand being a part of this study is completely up to you and that you can stop being a part of the discussion at any time, with no penalty or risk.

3. You understand that only ICF staff will see your answers to these questions.

4. You understand that your name will not be included in any reports or presentations of the results and that what you share with us today will be treated as confidential.

5. You understand the possible risks and benefits of being a part of this study.

Questions

Should you have any questions about the focus group, you may contact study representative Dr. Kimberly Good at Kimberly.Good@icf.com, or Institutional Review Board representative [name].

Let’s begin with some brief introductions. Please tell us your name and a little bit about yourself.

• Participants introduce themselves

Community Context

1. Can you tell us about your community? What is it like? (9)

• Probe for history, infrastructure, community leaders

2. How long have you lived in the community? Has it changed in the past 5 years? If so, how?

3. How did the COVID-19 pandemic affect you and your community?

Perceptions of Climate Change

4. What do you know about climate change?

• Probe for causes of climate change, their beliefs about climate change

1. How does climate change impact your life? (8)

• Probe for economic, health, housing, transportation, and social impacts

6. What are you doing to respond to climate change?

Engagement with the Program

7. Why did you get involved with [program name]?

8. How have you been involved with [program name]? (10.a)

   1. Did you provide feedback on the program? Did you contribute to the development, content, or activities of the program?

9. Did you receive any training or education related to climate change? If so, what was the training about and how helpful was it?

Program Outcomes

10. In what ways has [program name] affected you personally?

• Probe on the following and follow up with how

• Energy efficiency/renewable energy sources (18.a)

• Disaster response (18.b)

• Environmental conservation (18.c)

• Improved knowledge, attitudes, and behaviors around climate change (18.d)

• Built community capacity to mitigate and adapt to climate change (18.e)

11. Do you have any suggestions to improve [program name]?

PARTNER ORGANIZATION IN-DEPTH INTERVIEW
(60 minutes)

Partner Organization Consent Statement

Participation

Thank you again for taking the time to participate in today’s interview. This interview is part of a study being conducted by ICF for AmeriCorps. The reason for the study is to help AmeriCorps and [program name] better understand how their programs are working. More specifically, we are interested in your organization’s activities, partnership with [program name], and effect on the community you serve. We have prepared a number of discussion questions, but please feel free to offer any additional thoughts you have about your experience. The interview will take about 30 minutes.

Risks

We do not anticipate any risks in being a part of this interview. Participation in today’s discussion is completely up to you. You can decide you don’t want to answer any question we ask you, and you can stop participating in the discussion at any time. You can also decide not to participate at all and leave the interview at any point.

Anything you share with ICF will be kept confidential. By speaking with us, you will not affect your current or future relationship with [program name] or AmeriCorps. A report summarizing feedback across different organizations and AmeriCorps members will be shared with AmeriCorps. Only ICF staff will see the data collected from this interview, and the information that we use from our discussions will not be identified with any one individual.

Do you have any questions about this study or this interview?

Before we begin, we would also like to check that you are 18 years or older and get your permission to participate in today’s discussion. If you do not want to participate, you may leave at this time.

Consent Statement

1. You are 18 years or older.

2. You understand being a part of this study is completely up to you and that you can stop being a part of the discussion at any time, with no penalty or risk.

3. You understand that only ICF staff will see your answers to these questions.

4. You understand that your name will not be included in any reports or presentations of the results and that what you share with us today will be treated as confidential.

5. You understand the possible risks and benefits of being a part of this study.

Questions

Should you have any questions about the interview, you may contact study representative Dr. Kimberly Good at Kimberly.Good@icf.com, or Institutional Review Board representative [name].

Background

1. Let’s begin with some brief introductions. Please tell me a little bit about yourself.

   1. What is your title and role at [partner organization name]?

   2. How long have you been in this position?

   3. What are your specific responsibilities?

2. Can you tell me a little about [partner organization name]? What types of services and activities does your organization provide?

Partnership and Program Operations

3. Can you please tell me how you started working with [organization name]? How long have you been working with them? (16)

   1. How has [partner organization name] maintained a relationship with [organization name] during that time? (16)

4. How would you describe the relationship between the organizations? (15)

5. What role does your organization play with [program name]? How have you been involved with planning/implementation of [program name]? (14)

   1. How did the COVID-19 pandemic affect program planning and implementation? (12)

6. How is your organization building community resilience? (17)

   1. In what ways has climate change affected your community and its members? (8)

• Probe for economic, health, housing, transportation, and social impacts

1. Can you discuss some of the services and activities that address climate change?

1. Can you discuss how the program attempts to increase equity in terms of climate change outcomes? (2)

2. Do you work with members of the community? How are they involved in the program? (10.a)

• Probe to identify if they were involved in planning and implementation

1. What practices have you found to be most effective for engaging vulnerable and/or diverse communities? (4.a)

9. What aspects of the program do you think are most effective? What do you think could be improved in how the program was structured and implemented? (4)

Program Outcomes

10. From your perspective, in what ways has [program name] helped the community address climate change?

• Probe on the following and follow up with how

  • Energy efficiency/renewable energy sources (18.a)

  • Disaster response (18.b)

  • Environmental conservation (18.c)

  • Improved knowledge, attitudes, and behaviors around climate change (18.d)

  • Built community capacity to mitigate and adapt to climate change (18.e)

11. From your perspective, how likely is it that you will be able to sustain the program after the grant-funding period? (7)

    1. Which components will you be able to sustain?

12. Do you have any additional feedback or insights you would like to share with us regarding [program name] or working with [organization name]?

EVALUATION CAPACITY BUILDING PRE-/POST-PROGRAM ASSESSMENT

On behalf of AmeriCorps (formerly known as the Corporation for National and Community Service), ICF is providing evaluation capacity building workshops throughout one year to grantees and sites that are participating in the [bundle name] evaluation. We are inviting you to complete this 15-minute survey at the beginning and end of your participation.

Participation in this survey is voluntary, but we hope you will respond because your experience and perspective are extremely valuable to assess the success of these workshops.

1. Awareness/Attitudes/Motivation:

To what extent do you agree with the following statements?

(1 – Strongly Disagree, 2 – Disagree, 3 – Neither Agree nor Disagree, 4 – Agree, 5 – Strongly Agree)

1. An evaluation…

1. will help me understand my program.

2. will inform the decisions I make about my program.

3. will justify funding for my program.

4. will help to convince managers that changes are needed in my program.

5. is absolutely necessary to improve my program.

6. should involve program participants in the evaluation process.

7. will help improve services to target populations.

8. takes away resources that should be used to provide services.

9. is incorporated into my daily work.

10. report is something I review regularly.

11. can have findings that affect the decisions I make in my work.

2. I am motivated to…

1. learn about evaluation.

2. start evaluating my program.

3. support other staff to evaluate their program.

4. encourage others to buy in to evaluating our program.

3. To what extent are the following factors important in your motivation to engage in program evaluation?
   (1 – Not at all, 2 – To a small extent, 3 – To a moderate extent, 4 – To a large extent, 5 – Always)

1. A need to meet accountability requirements

2. A desire to enlighten and support government policy making and planning

3. Changes in the organization (e.g., reorganization, new leadership, or vision)

4. A mandate from leadership to increase the learning function of evaluation

5. A perceived lack of internal evaluation knowledge and skills

6. A desire to seek new or increased funding

7. A desire to use evaluation to make program improvements

8. Other factors (specified)

2. Skills/Knowledge/Behaviors:

To what extent do you agree with the following statements?

(1 – Not at all, 2 – To a small extent, 3 – To a moderate extent, 4 – To a large extent, 5 – Always)

1. I know how to…

1. recognize key concepts in evaluation (evidence, data, performance measurement, evaluation).

2. recognize the components of an evaluation plan.

3. describe what a “theory of change” is.

4. recognize how theory of change connects to a logic model.

5. recognize the attributes of a good evaluation question.

6. identify strategies to collect information from participants.

7. recognize features of process and outcome indicators of a program.

8. recognize who should collect data.

9. recognize when data should be collected.

10. identify common data analysis terms.

11. recognize how quantitative and qualitative analysis is performed.

12. recognize how to use the evaluation findings to improve a program.

13. read an evaluation report and recognize its basic components.

2. In the past year, I helped my organization to…

1. provide funding to carry out an evaluation or hire an external evaluator (as part of a grant or through other means).

2. provide training or technical assistance to conduct an evaluation (by your staff, consultants, or other means).

3. conduct evaluations of programs funded by your organization.

4. use evaluation findings to improve programs funded by your organization.

5. use evaluation findings to demonstrate and communicate effectiveness of programs funded by your organization.

3. Evaluation Needs

1. Does your organization have any of the following?

1. Staff position(s) or group within your organization dedicated to evaluation

2. External evaluation partner(s), consultant(s), or organization(s) that provide your organization with evaluation services

3. Part of the organizations' budget dedicated to evaluation

1. Are you currently working with an evaluator for this program?

• Yes

• No

1. To what extent have you experienced the following barriers to conducting a program evaluation? (Select all that apply)

1. Not enough time

2. Not enough money

3. Not knowing how to conduct a program evaluation

4. Not having assistance with data collection

5. Not getting enough people to respond to surveys

6. Not knowing how to analyze data

7. Not having people to turn to for consultation and assistance

8. Not knowing what questions to ask

9. Not knowing how to write up results

10. Other factors (please specify):

1. If you were to attend a series of evaluation-related workshop, what would you like to learn most? (Select only one.)

• How to conduct an evaluation within my organization

• How to work with external evaluators

• What other organizations do with their evaluations

• Other (please specify)

1. If you were to attend a series of evaluation-related workshop that last 1 hour each, how much time would you like to spend in interactive activities (e.g., breakout sessions, chat, exercise)? (Select only one.)

• Less than 10 minutes

• 11–20 minutes

• 21–30 minutes

1. Which of the following approaches would you prefer:

• To have discussion-centered learning activities but with an expectation that you read evaluation learning materials and complete activities in advance.

• To have presentation-centered learning activities with little or no expectation to review or prepare materials in advance.

4. About the Respondent

A. Which one of these groups would you say best represents your race?

• White

• Black or African American

• American Indian or Alaska Native

• Asian

• Native Hawaiian or Pacific Islander

• Multiracial or multi-ethnic (2+ races/ethnicities)

• Other (please specify): ____________

• Don’t know

• Prefer not to say

2. Are you of Hispanic, Latino/a, or Spanish origin?

• Yes

• No

• Don’t know

• Prefer not to say

3. What is your gender?

• Male

• Female

• Nonconforming

• Prefer not to respond

• Prefer to self-describe (please specify): __________________________________

4. How long have you worked with your organization?

• Less than 1 year

• 2–5 years

• 6–10 years

• 11–20 years

• 21+ years

5. What is your position in the organization?

• Leader (CEO, executive Director)

• Other executive

• Manager/Supervisor

• Staff

• Board member

• Other (please specify): __________________________________

6. How long has your organization worked with AmeriCorps?

• Less than 1 year

• 2–5 years

• 6–10 years

• 11–20 years

• 21+ years

EVALUATION CAPACITY BUILDING PRESENTATION FEEDBACK SURVEYS

Module 1

Presentation 1: Introduction and Evaluation Basics

Learning Objectives

• Describe how evaluations can be useful

• Recognize key concepts in evaluation

• Recognize stages of evaluation

Knowledge Questions

1. What is not a good reason to conduct a program evaluation?

   1. Program A wants to demonstrate the benefits of the program to stakeholders

   2. Program C wants to understand what needs to be improved in the program

   3. Program B wants to generate a daily report for its employees

   4. Program D wants to make decisions about resource allocation

2. Please match the following terms with the right definition.

   1. Evidence: The reasons why you think your program will achieve its goals.

   2. Evaluation: Systematic investigation of your program.

   3. Quantitative data: Numerical information that can be counted, quantified, and mathematically analyzed.

   4. Qualitative data: Narrative information that often describes context, motivations, and “How?” and “Why?” answers.

3. Which of these statements best highlights the difference between program evaluation and performance measurement?

   1. Performance measurement uses a systematic approach while evaluation uses regular updates of changes over time.

   2. Performance measurement answers program-level research questions while evaluation answers routine questions.

   3. Performance measurement reports data that requires programmatic change while evaluation reports data that requires immediate change.

   4. Performance measurement indicates whether a project is moving according to plan while evaluation systematically examines the workings of a program.

Satisfaction Questions

1. Overall, how satisfied are you with this presentation?

   1. Very Dissatisfied, Dissatisfied, Unsure, Satisfied, Very Satisfied

2. How much do you agree or disagree with the following statements. (Strongly Disagree, Disagree, Neither Agree nor Disagree, Agree, Strongly Agree)

   1. The information I learned in this module was useful for me.

   2. The instructor(s) clearly explained the material covered in this module.

   3. I felt comfortable voicing any questions I had in class.

   4. The activities were helpful to my understanding of the material.

   5. I understand how to apply this material to my own program.

3. After taking this module, how do you rate your understanding of the following topics?

(Very Poor, Poor, Acceptable, Good, Very Good)

1. Why evaluations can be useful

2. The difference between evidence and data

3. The difference between evaluation and performance measurement

1. What did you enjoy about the learning experience?

2. Do you have any suggestions for how to improve the learning experience?

Module 1

Presentation 2: Getting to Know One Another

Satisfaction Questions

1. Overall, how satisfied are you with this presentation?

   1. Very Dissatisfied, Dissatisfied, Unsure, Satisfied, Very Satisfied

2. How much would you agree or disagree with the following statements? (Strongly Disagree, Disagree, Neither Agree nor Disagree, Agree, Strongly Agree)

   1. Getting to know my peers was useful to me.

   2. The instructor(s) facilitated the session well.

3. What did you enjoy about the module?

4. Do you have any suggestions for how to improve the learning experience?

Module 1

Presentation 3: Theory of Change

Knowledge Questions

1. Which of these best defines what a “theory of change” is?

   1. An explanation of any negative or positive impact your program has on the community

   2. An explanation of what your program’s ultimate goal is

   3. An example of how you’d like to change your role within your program

   4. A reflection on how you achieve your personal goals every day within your program

2. What is not a reason to develop a theory of change?

   1. It ensures people will not ask you to evaluate your program

   2. It forces your organization to articulate the assumptions underlying your program

   3. It highlights the goals your program is trying to achieve

   4. It shows the unique contribution your program makes to achieve its goals

Satisfaction Questions

1. Overall, how satisfied are you with this presentation?

   1. Very Dissatisfied, Dissatisfied, Unsure, Satisfied, Very Satisfied

2. How much would you agree or disagree with the following statements?

(Strongly Disagree, Disagree, Neither Agree nor Disagree, Agree, Strongly Agree)

1. The information I learned in this module was useful for me.

2. The instructor(s) clearly explained the material covered in this module.

3. I felt comfortable voicing any questions I had in class.

4. The activities were helpful to my understanding of the material.

5. I understand how to apply this material to my own program.

1. After taking this module, how do you rate your understanding of the following topics?

(Very Poor, Poor, Acceptable, Good, Very Good)

1. What a theory of change is

2. How to come up with a theory of change for my program

1. What did you enjoy about the learning experience ?

2. Do you have any suggestions for how to improve the learning experience?

Module 1

Presentation 4: Logic Models

Knowledge Questions

1. Select the statement that best describes how a logic model connects to your theory of change.

   1. A theory of change visually describes a logic model.

   2. A logic model visually shows how your program’s theory of change works.

   3. A theory of change visually represents the connections between program components, while a logic model is a statement describing the program’s goals.

   4. A logic model presents every theory of change that you can come up with in one chart.

2. Using what you’ve learned in class about what goes into a logic model and the following theory of change, select one item for each column in the logic model come up with one item for each column in the logic model.

Our state increasingly needs public lands maintenance. We believe this is because our state is facing an increase in natural and human-made disasters and does not have enough high-impact conservation efforts in place. Our theory is that if AmeriCorps members assist with high-impact conservation, disaster recovery, and mitigation projects, and reroute trails to avoid sensitive ecosystems, then public lands will become more resilient and resistant to future disasters as well as recover from past disasters. By engaging in high-impact conservation and treating various ecosystems, we will ease the burden of the disasters on our state. Ultimately, this will reduce the need for reactive public land maintenance.

1. An example of an input for this program:

   1. Funding for the program

   2. Rerouting trails to avoid sensitive ecosystems

   3. Number of trails that are constructed because of the program

   4. Changes in invasive species numbers in parks

   5. Reduced need for reactive public land maintenance

2. An example of activities:

   1. Funding for the program

   2. Rerouting trails to avoid sensitive ecosystems

   3. Number of trails that are constructed because of the program

   4. Changes in invasive species numbers in parks

   5. Reduced need for reactive public land maintenance

3. An example of outputs:

   1. Funding for the program

   2. Rerouting trails to avoid sensitive ecosystems

   3. Number of trails that are constructed because of the program

   4. Changes in invasive species numbers in parks

   5. Reduced need for reactive public land maintenance

4. An example of short-term outcomes:

   1. Funding for the program

   2. Rerouting trails to avoid sensitive ecosystems

   3. Number of trails that are constructed because of the program

   4. Changes in invasive species numbers in parks

   5. Reduced need for reactive public land maintenance

5. An example of intermediate outcomes:

   1. Funding for the program

   2. Rerouting trails to avoid sensitive ecosystems

   3. Number of ecosystems that have recovered from disasters

   4. Changes in invasive species numbers in parks

   5. Reduced need for reactive public land maintenance

6. An example of long-term outcomes:

   1. Funding for the program

   2. Rerouting trails to avoid sensitive ecosystems

   3. Number of trails that are constructed because of the program

   4. Changes in invasive species numbers in parks

   5. Reduced need for reactive public land maintenance

Satisfaction Questions

1. Overall, how satisfied are you with this presentation?

   1. Very Dissatisfied, Dissatisfied, Unsure, Satisfied, Very Satisfied

2. How much would you agree or disagree with the following statements?

(Strongly Disagree, Disagree, Neither Agree nor Disagree, Agree, Strongly Agree)

1. The information I learned in this module was useful for me.

2. The instructor(s) clearly explained the material covered in this module.

3. I felt comfortable voicing any questions I had in class.

4. The activities were helpful to my understanding of the material.

5. I understand how to apply this material to my own program.

1. After taking this module, how do you rate your understanding of the following topics?

(Very Poor, Poor, Acceptable, Good, Very Good)

1. What a logic model is

2. What the difference is between a logic model and a theory of change

3. How to figure out what goes into a logic model

1. What did you enjoy about the learning experience ?

2. Do you have any suggestions for how to improve the learning experience?

Module 1

Presentation 5: Evaluation Planning

Knowledge Questions

1. Which is an example of a good evaluation question?

   1. Does the climate change program work?

   2. Are the disaster mitigation strategies useful?

   3. To what extent does the likelihood of public lands maintenance differ for communities who participate in this program compared to other communities?

   4. Does the program improve its communities’ happiness?

2. What’s the difference between process and outcome evaluations?

   1. Process evaluations examine how a program works while outcome evaluations measure the results or effects of a program

   2. Process evaluations measure changes in attitudes while outcome evaluations examine whether the program has sufficient resources

   3. Process evaluations measure changes to beneficiaries while outcome evaluations measure what pieces of the program are working as expected

   4. Process evaluations measure the results of a program while outcome evaluations measure program activities

3. What is an example of a quantitative indicator for a process evaluation?

   1. Community leaders’ thoughts about what did or did not work well in their program

   2. AmeriCorps members’ opinions about what they liked and did not like about their training

   3. The number of communities who received disaster relief initiatives

   4. Attributes of communities who received all program components and those who did not

Satisfaction Questions

1. Overall, how satisfied are you with this presentation?

   1. Very Dissatisfied, Dissatisfied, Unsure, Satisfied, Very Satisfied

2. How much would you agree or disagree with the following statements?

(Strongly Disagree, Disagree, Neither Agree nor Disagree, Agree, Strongly Agree)

1. The information I learned in this module was useful for me.

2. The instructor(s) clearly explained the material covered in this module.

3. I felt comfortable voicing any questions I had in class.

4. The activities were helpful to my understanding of the material.

5. I understand how to apply this material to my own program.

1. After taking this module, how do you rate your understanding of the following topics?

(Very Poor, Poor, Acceptable, Good, Very Good)

1. The difference between a process and an outcome evaluation

2. The attributes of a good evaluation question

1. What did you enjoy about this learning experience?

2. Do you have any suggestions for how to improve the learning experience?

Module 2

Presentation 1: Preparing to Collect Data

Knowledge Questions

1. What is an example of primary data that can be collected to evaluate the impact of the recovery coaching program?

   1. Economic data on communities from the U.S. Census Bureau

   2. Geologic basin boundaries from the U.S. Environmental Protection Agency

   3. Survey responses from AmeriCorps members in the program

   4. Community actions data from the state

2. When collecting data from the population, what is a good reason to draw a sample? (Select all that apply.)

   1. It can be time-consuming to collect data from an entire population

   2. It costs less to collect data from a sample than a population

   3. One can get a higher response rate using follow-up efforts with data collected from a sample than a population

   4. A sample can be representative of a population

3. When should you collect the following types of data?

(Before the intervention / During the intervention / After the intervention)

1. Baseline data: __________ (Before the intervention)

2. Number of rivers that were treated: ___________ (During or after the intervention)

3. Changes in employment: ___________ (After the intervention)

Satisfaction Questions

1. Overall, how satisfied are you with this module?

   1. Very Dissatisfied, Dissatisfied, Unsure, Satisfied, Very Satisfied

2. How much would you agree or disagree with the following statements?

(Strongly Disagree, Disagree, Neither Agree nor Disagree, Agree, Strongly Agree)

1. The information I learned in this module was useful for me.

2. The instructor(s) clearly explained the material covered in this module.

3. I felt comfortable voicing any questions I had in class.

4. The activities were helpful to my understanding of the material.

5. I understand how to apply this material to my own program.

1. After taking this module, how do you rate your understanding of the following topics?

(Very Poor, Poor, Acceptable, Good, Very Good)

1. The difference between primary data and secondary data

2. When data should be collected to evaluate a program

1. What did you enjoy about this learning experience?

2. Do you have any suggestions for how to improve the learning experience?

Module 2

Presentation 2: Data Collection Techniques

Knowledge Questions

1. If you are intending to collect qualitative data from a participant, which data collection technique would you not utilize?

1. Structured interviews

2. Focus groups

3. Structured observations

4. Closed-ended survey questions

2. Read the questions below and identify the “common pitfall” (if any) associated with each of the survey questions below.

1. Do you feel that you have had a great experience with treating public lands in this program?

1. Double-barreled question

2. Leading question

3. Social desirability

4. Non-applicable question

5. No pitfall

1. Please rate the impact of removing hazard trees to mitigate wildland fire, with 1 being “very useful” and 5 being “not useful at all.”

1. Double-barreled question

2. Leading question

3. Social desirability

4. Non-applicable question

5. No pitfall

3. Do you have any recommendations or complaints about the river treatment process?

6. Double-barreled question

7. Leading question

8. Social desirability

9. Non-applicable question

10. No pitfall

Satisfaction Questions

1. Overall, how satisfied are you with this presentation?

1. Very Dissatisfied, Dissatisfied, Unsure, Satisfied, Very Satisfied

2. How much would you agree or disagree with the following statements?

(Strongly Disagree, Disagree, Neither Agree nor Disagree, Agree, Strongly Agree)

1. The information I learned in this module was useful for me.

2. The instructor(s) clearly explained the material covered in this module.

3. I felt comfortable voicing any questions I had in class.

4. The activities were helpful to my understanding of the material.

5. I understand how to apply this material to my own program.

2. After taking this module, how do you rate your understanding of the following topics?

(Very Poor, Poor, Acceptable, Good, Very Good)

1. When to use surveys for primary data collection

2. When to use focus groups for primary data collection

3. When to use interviews for primary data collection

4. When to use structured observations for primary data collection

5. The different types of data collection techniques

2. What did you enjoy about this learning experience?

3. Do you have any suggestions for how to improve the learning experience?

Module 2

Presentation 3: Data Analysis

Knowledge Questions

1. What kinds of questions would quantitative data be used to answer?

   1. How do AmeriCorps members interact with the organization when assisting with conservation efforts?

   2. Why do some communities become resilient while others do not?

   3. How did clearing trails affect erosion and flooding on public lands?

   4. How many miles of trails have been created by AmeriCorps members?

2. What is an example of qualitative analysis of data?

   1. Frequencies

   2. Counts

   3. Minimum and maximum

   4. Interpretation of direct quotes

3. What is an effective way to quantitively analyze data?

   1. Measure whether a difference between two groups is “statistically significant”

   2. Code participant responses to analyze themes in the data

   3. Interpret differences in categories of coded responses

   4. Taking narrative notes

Satisfaction Questions

1. Overall, how satisfied are you with this presentation?

   1. Very Dissatisfied, Dissatisfied, Unsure, Satisfied, Very Satisfied

2. How much would you agree or disagree with the following statements?

(Strongly Disagree, Disagree, Neither Agree nor Disagree, Agree, Strongly Agree)

1. The information I learned in this module was useful for me.

2. The instructor(s) clearly explained the material covered in this module.

3. I felt comfortable voicing any questions I had in class.

4. The activities were helpful to my understanding of the material.

5. I understand how to apply this material to my own program.

1. After taking this module, how do you rate your understanding of the following topics?

(Very Poor, Poor, Acceptable, Good, Very Good)

1. Quantitative analysis techniques

2. Qualitative analysis techniques

1. What did you enjoy about this learning experience?

2. Do you have any suggestions for how to improve the learning experience?

Module 3

Presentation 1: Evaluation Reporting

Knowledge Questions

1. Which of these questions should the discussion section of an evaluation report answer?

   1. What was the approach used to analyze the data?

   2. What was the theory of change for the program?

   3. Do the findings of the report support your theory of change?

2. Which of these questions should the methodology section of an evaluation report answer?

   1. What was the approach used to analyze the data?

   2. What was the theory of change for the program?

   3. Do the findings of the report support your theory of change?

3. Which of these questions should the introduction section of an evaluation report answer?

   1. What was the approach used to analyze the data?

   2. What was the theory of change for the program?

   3. Do the findings of the report support your theory of change?

Satisfaction Questions

1. Overall, how satisfied are you with this presentation?

   1. Very Dissatisfied, Dissatisfied, Unsure, Satisfied, Very Satisfied

2. How much would you agree or disagree with the following statements?

(Strongly Disagree, Disagree, Neither Agree nor Disagree, Agree, Strongly Agree)

1. The information I learned in this module was useful for me.

2. The instructor(s) clearly explained the material covered in this module.

3. I felt comfortable voicing any questions I had in class.

4. The activities were helpful to my understanding of the material.

5. I understand how to apply this material to my own program.

1. After taking this module, how do you rate your understanding of the following topics?

(Very Poor, Poor, Acceptable, Good, Very Good)

1. What goes into an evaluation report

1. What did you enjoy about this learning experience?

2. Do you have any suggestions for how to improve the learning experience?

Module 3

Presentation 2: Using Evaluation for Program Improvement

Knowledge Questions

1. Select all the ways you can use findings from a process evaluation for program improvement.

   1. To demonstrate improvements for communities after disaster recovery initiatives

   2. To plan further evaluations to examine long-term reductions in community resilience

   3. To demonstrate changes in land erosion after conservation efforts through the program

   4. To change the duration of maintenance the public lands receive

   5. To revise the intervention model

   6. To change the program activities

2. Select all the ways you can use findings from an outcome evaluation for program improvement.

   1. To demonstrate improvements for communities after disaster recovery initiatives

   2. To plan further evaluations to examine long-term reductions in community resilience

   3. To demonstrate changes in land erosion after conservation efforts through the program

   4. To change the duration of maintenance the public lands receive

   5. To revise the intervention model

   6. To change the program activities

Satisfaction Questions

1. Overall, how satisfied are you with this presentation?

   1. Very Dissatisfied, Dissatisfied, Unsure, Satisfied, Very Satisfied

2. How much would you agree or disagree with the following statements?

(Strongly Disagree, Disagree, Neither Agree nor Disagree, Agree, Strongly Agree)

1. The information I learned in this module was useful for me.

2. The instructor(s) clearly explained the material covered in this module.

3. I felt comfortable voicing any questions I had in class.

4. The activities were helpful to my understanding of the material.

5. I understand how to apply this material to my own program.

1. After taking this module, how do you rate your understanding of the following topics?

(Very Poor, Poor, Acceptable, Good, Very Good)

1. Findings from a process evaluation

2. Findings from an outcome evaluation

1. What did you enjoy about this learning experience?

2. Do you have any suggestions for how to improve the learning experience?

Module 3

Presentation 3: Developing a Long-Term Research Agenda

Knowledge Questions

1. What does not characterize a long-term research agenda?

   1. It should be progressive

   2. Once it is established, it cannot be changed

   3. It should build on what has been done in the past

   4. It should point towards future goals

2. What needs to be done to build evidence to support the program goals over time? Arrange the following tasks in order of complexity.

   1. Build consensus about your overall evaluation goals.

   2. Conduct a process evaluation.

   3. Conduct an outcome evaluation.

   4. Conduct an impact evaluation with a comparison group.

   5. Expand the program to multiple sites.

Satisfaction Questions

1. Overall, how satisfied are you with this presentation?

   1. Very Dissatisfied, Dissatisfied, Unsure, Satisfied, Very Satisfied

2. How much would you agree or disagree with the following statements?

(Strongly Disagree, Disagree, Neither Agree nor Disagree, Agree, Strongly Agree)

1. The information I learned in this module was useful for me.

2. The instructor(s) clearly explained the material covered in this module.

3. I felt comfortable voicing any questions I had in class.

4. The activities were helpful to my understanding of the material.

5. I understand how to apply this material to my own program.

1. After taking this module, how do you rate your understanding of the following topics?

(Very Poor, Poor, Acceptable, Good, Very Good)

1. Understand the key components of a long-term research agenda.

2. Visualize how to develop a long-term research agenda for your program.

1. Do you have any suggestions for how to improve the learning experience?

2. What did you enjoy about this learning experience? What was your favorite part?

Module 3

Presentation 4: Application to Climate Change Bundle

Satisfaction Questions

1. Overall, how satisfied are you with this presentation?

   1. Very Dissatisfied, Dissatisfied, Unsure, Satisfied, Very Satisfied

2. How much would you agree or disagree with the following statements?

   1. (Strongly Disagree, Disagree, Neither Agree nor Disagree, Agree, Strongly Agree)

   2. The information I learned in this module was useful for me.

   3. The instructor(s) clearly explained the material covered in this module.

   4. I felt comfortable voicing any questions I had in class.

   5. The activities were helpful to my understanding of the material.

   6. I understand how to apply this material to my own program.

3. After taking this module, how do you rate your understanding of the following topics?

   1. (Very Poor, Poor, Acceptable, Good, Very Good)

   2. How the climate change bundle follows evaluation processes discussed during the evaluation technical assistance (TA) sessions

   3. How the climate change bundle follows evaluation best practices discussed during the evaluation TA sessions

   4. How the evaluation TA sessions directly apply to the climate change bundle

   5. Visualize how to develop a long-term research agenda for your program.

4. Do you have any suggestions for how to improve the learning experience?

5. What did you enjoy about this learning experience? What was your favorite part?

EVALUATION CAPACITY BUILDING OBSERVATION FORM

Observer name: ______________________________

Date of observation: ___________________________

Presentation title: _____________________________

Number of participants: ________________________

1. Overall, how satisfied are you that this presentation met the learning goals/objectives?

   1. Very Dissatisfied, Dissatisfied, Unsure, Satisfied, Very Satisfied

2. How much would you agree or disagree with the following statements?

(Strongly Disagree, Disagree, Neither Agree nor Disagree, Agree, Strongly Agree; notes for each item)

1. The instructor(s) clearly explained the material covered in this module.

2. The activities were helpful to participants’ understanding of the material.

3. The answers the instructor(s) gave to participants’ questions were clear.

4. The instructor(s) provided illustrative examples.

5. The exercises in the presentation were well-facilitated.

6. The participants were engaged in the module.

7. The participants learned or improved upon the objectives set in the module.

3. How does the course help develop content knowledge, skills, or thinking practices related to these learning objectives? _______________________________________________________________________

4. What are the strengths of this learning experience? _______________________________________________________________________

5. Do you have any suggestions for how to improve this learning experience? _______________________________________________________________________