Form PRBE Bundled Evaluation - Puerto Rico

OMB Control Number:

Expiration Date:

AMERICORPS/VISTA ALUMNI FOCUS GROUP
(60 minutes)

(Research question addressed is in parentheses)

AmeriCorps alumni Focus Group Consent

Participation

Thank you again for taking the time to participate in today’s focus group. This focus group is part of a study being conducted by ICF for AmeriCorps. The reason for the study is to help AmeriCorps better understand how their programs work in Puerto Rico. More specifically, we are interested in your experiences as AmeriCorps/Vista members when working with [organization name] and the effect on the community you served. We have prepared a number of discussion questions, but please feel free to offer any additional thoughts you have about your experience. The focus group will take up to 60 minutes.

Risks

We do not anticipate any risks in being a part of this focus group. Participation in today’s discussion is completely up to you. You can decide you don’t want to answer any question we ask you, and you can stop participating in the discussion at any time. You can also decide not to participate at all and leave the focus group at any point.

Anything you share with ICF will be kept confidential. By speaking with us, you will not affect your current or future relationship with [program name]. A report summarizing feedback across different organizations and AmeriCorps Members will be shared with AmeriCorps. Only ICF staff will see the data collected from this focus group, and the information that we use from our discussions will not be identified with any one individual. However, since we are only talking to a small number of members at each organization, there is a chance that AmeriCorps or [program name] personnel will be able to guess which individual shared certain information.

We also request that you do not discuss what is disclosed in this focus group once the discussion ends. Do you have any questions about this study or this focus group?

To fully capture the discussion, we would like to record the session. Again, only ICF evaluation team members will have access to the recording, which will not be shared with anybody in your program/organization or AmeriCorps. We will not include your name(s) in these notes. Any information that can be used to identify an individual will be removed prior to sharing any findings. If you do not want to participate, you may leave at this time.

Questions

Should you have any questions about the focus group, you may contact study representative Dr. Susana Mazuelas Quirce at susana.mazuelasquirce@icf.com or Dr. Samantha Spinney at samantha.spinney@icf.com or ICF’s Institutional Review Board at IRB@icf.com.

START RECORDER NOW

Background

Let’s begin with some brief introductions. Please tell us your name, a little bit about yourself, and how long ago you were an AmeriCorps member/VISTA.

1. Were you a member of the community you served or from a different community/region?

2. Why did you become an AmeriCorps member/VISTA? How did you become aware of the program? (RQ13)

1. What encouraged you to sign up to be a member? What was it about it that interested you? (RQ15)

2. What were your goals and your expectations?

3. Were they fulfilled? Why or why not?

Program Context and Activities

3. In which organization/program did you serve?

4. What was your role in [program/organization name]? (RQ9)

   1. What services did you provide? What activities did you engage in during your service? (RQ 8b)

5. Which kind of training did you receive after joining the organization as an AmeriCorps National Service/VISTA member? (RQ14, RQ19)

   1. Did you receive a certification or other credential for your training

   2. How, if at all, did the training you received prepare you for future jobs?

Program Outcomes

6. What types of professional skills do you believe you gained through your AmeriCorps experience? Probe for technical skills, administrative skills, communications skills, etc.

7. What are you doing right now—working, in school, volunteering, something else?

   1. For those of you who are working, are you working in a similar field to the organization or program where you served with AmeriCorps?

   2. If yes, how, if at all, did your AmeriCorps experience pave the way for you to get this job? Probe for: making connections via networking, gaining relevant knowledge or skills, boosting the qualifications on your resume, etc.

   3. If no, what field are you working in now? Did your AmeriCorps experience help you—in some way—access this field? Why or why not?

8. What else could AmeriCorps or [organization name] have done to help you to take full advantage of your experience?

9. If you had to start over again as an AmeriCorps National Service/VISTA member, what would you do differently?

10. Would you recommend the program to other people? Why or why not? (RQ 15, 15a)

    1. How can AmeriCorps be more attractive to Puerto Ricans to volunteer?

11. Who would you think would be an ideal candidate to be an AmeriCorps National Service/VISTA member?

    1. What would be good recruiting strategies? (RQ 13a)

Life after AmeriCorps

12. What did you do after finishing your service time? [Probe for finding a job, going to college, or continuing to serve/volunteer in another capacity] (RQ18)

13. Did you move to another town or region, or did you stay in the same community you lived in before? (RQ18)

    1. If moved: What were the reasons for moving?

14. Why did you decide to stay in Puerto Rico? [probe for economic, educational, political, family, health, cultural, community reasons] (RQ18)

    1. Are you considering moving in the future? Why or why not?

       1. If moving: To where, why, and for how long?

1. If moving: What would encourage you to stay in Puerto Rico permanently?

15. If you keep in touch with former AmeriCorps members, do you know what they are doing, if they stayed in Puerto Rico, or what are their plans for the future? [probe for reasons to stay in Puerto Rico]

16. How has your experience with AmeriCorps influenced your decisions about staying or moving?

17. What else can AmeriCorps or [organization name] do to help you to achieve their goals after your service time?

Final Thoughts

18. Do you have any additional feedback or insights you would like to share with us regarding your experiences in the program or working with [organization name]?

Thanks for your time!

AMERICORPS ALUMNI SURVEY PROTOCOL

AmeriCorps Member Survey Consent

Participation

Thank you for taking the time to participate in this survey. This survey is part of a study being conducted by ICF to help AmeriCorps better understand how their programs are working in Puerto Rico. As a former AmeriCorps member, we are interested in how your experience serving has or has not contributed to the next steps in your professional lives. The survey will take about 20 minutes to complete.

As a small token of appreciation, ICF will raffle off 10 $25 gift cards to respondents who opt to participate in the raffle. To opt into the raffle, you will need to provide your email address, however it will be disconnected from the rest of your survey data. More information about participating in the raffle is available at the end of the survey.

Risks

We do not anticipate any risks in participating in this survey. Participation in this survey is voluntary. You can skip any question or stop answering questions at any time.

Your responses to the survey will be kept confidential. Your answers will not affect future work with AmeriCorps. A report summarizing feedback across respondents will be shared with AmeriCorps. The information that we use from this survey will not be identified with any one individual. Only ICF staff and the AmeriCorps Office of Research and Evaluation will have access to the de-identified survey data.

Questions

Should you have any questions about the focus group, you may contact study representative Dr. Susana Mazuelas Quirce at susana.mazuelasquirce@icf.com or Dr. Samantha Spinney at samantha.spinney@icf.com, or ICF’s Institutional Review Board at IRB@icf.com.

If you agree to participate in the survey, please acknowledge below by selecting, “I agree to participate.”

• I agree to participate.

• I do not agree to participate. [If selected, will use skip logic take respondent to the end of the survey]

Background

1. Are you currently serving as an AmeriCorps member?

   • Yes

   • No (skip to end of the survey)

2. How many different terms have you served in AmeriCorps?

   • 1 term

   • 2 terms

   • 3 terms

   • 4 terms

3. Please select which type of AmeriCorps member you were. (Select all that apply if you participated in multiple types of AmeriCorps programs)

	AmeriCorps State and National
	AmeriCorps VISTA
	AmeriCorps Senior
	Don’t know

4. How long did you serve as an AmeriCorps member in total (considering all different times or programs in which you have served)?

• Less than six months

• Six months – less than 1 year

• 1 year to less than 2 years

• 2 years to less than 3 years

• 3 years or more

5. Display logic – if selected “AmeriCorps State and National” in Q3: What types of services did you deliver through AmeriCorps? Select all that apply.

	Education services (e.g., delivering tutoring/mentoring/training services to students, providing services in after-school programs, developing educational curriculum or modules, helping reduce absenteeism/disciplinary issues, improving homework completion, supporting academic performance, supporting leadership development; or other initiatives for students)
	Disaster services (e.g., improving community knowledge/skills for addressing disasters, providing disaster readiness/mitigation services and emergency preparedness workshops, delivering disaster management services, assisting survivors, restoring communities affected by natural disasters)
	Health related services (e.g., delivering exercise or recreation programs/workshops, delivering services to prevent childhood obesity, supporting healthy lifestyles, providing nutrition information, serving as a companion/providing home support to the elderly, supporting individuals with an intellectual disability, providing food/nutrition to communities, providing mental health services/counseling/psychotherapy interventions, and other services to improve physical and mental well-being )
	Economic opportunity services (e.g., working with unemployed individuals to find jobs, providing financial literacy and work readiness information, supporting the visibility of small businesses, providing workshops on economic topics and business advisory services, helping individuals develop entrepreneurial ideas into small businesses or community organizations; supporting leadership development; and other services to support workforce development, business development, and affordable housing)
	Environmental services (e.g., maintaining and upkeeping planting sites, nurseries and pollinator gardens and harvesting fruits/vegetables; providing environmental education experiences and promoting environmental consciousness and practices; supporting the recovery/rehabilitation of community barren spaces and improving at-risk ecosystems; building community gardens, recycling disposal centers, and parks; holding environmental festivals; supporting communication with community leadership about environmental issues; supporting solar power and rain water storage initiatives, and other initiatives that support conservation of public lands and wildlife protection)
	Veterans and military families (e.g., providing a variety of supports to veterans and military families)
	Other focus (please specify): ________________________

6. Display logic – if selected “AmeriCorps VISTA” in Q3: What types of services did you deliver through AmeriCorps? Select all that apply.

• Identifying and leveraging funds/funding sources

• Conducting data-related tasks (e.g., developing data tracking/monitoring or data management systems, collecting or analyzing data)

• Developing/maintaining technology platforms

• Managing/training volunteers

• Developing/managing projects/programs

• Conducting communications/outreach/promotion services

• Developing designs, graphics, and/or maps

• Mentoring and/or accompanying others

• Conducting research and/or evaluation

• Developing community partnerships/alliances

• Systematizing programmatic services or administrative processes

• Developing, coordinating, or conducting workshops/trainings

• Developing informational materials (e.g., reports, profiles, educational/promotional materials, manuals)

• Community planning

• Other (please specify): ________

7. How would you describe the community you served?

• City/urban

• Country/rural

8. Did you complete your AmeriCorps service term?

• Yes

• No

9. Display logic – if selected ‘No’ in Q8: What was the primary reason why you didn’t complete your AmeriCorps service term?

• Financial issues

• Family issues

• Health/wellbeing issues

• Workplace issues

• Other (please specify): __________

10. When did you finish serving as an AmeriCorps member?

• Within the last 6 months (complete Q11 then skip to question Q14)

• Between 6 months and 1 year ago

• Between 1 and 2 years ago

• Between 2 and 3 years ago

• Between 3 and 4 years ago

• Between 4 and 5 years ago

• Between 5 and 7 years ago

• Between 7 and 10 years ago

• More than 10 years ago

11. What were you doing in the immediate six months before you first began your AmeriCorps service? Select all that apply.

• Working in the private sector

• Working in the nonprofit or social service sector

• Working in the public/government sector

• Attending high school

• Attending a vocational/technical training program

• Attending college

• Attending graduate school

• Engaged another service experience such as the Peace Corps

• Working in my own small business

• Working at a nonprofit or social entrepreneurship venture that I created

• Serving in the military

• Unemployed

• Other (please specify):___________

12. Display logic – show only if selected anything besides ‘within last 6 months’ in Q10: What did you do in the immediate six months after you completed your AmeriCorps service? Select all that apply.

• Got a job in the private sector

• Got a job in the nonprofit or social service sector

• Got a job in the public/government sector

• Enrolled in a vocational/technical training program

• Went to college

• Went to graduate school

• Enrolled in the same AmeriCorps program

• Enrolled in another AmeriCorps program

• Pursued another service experience such as the Peace Corps

• Started a small business

• Started a nonprofit or social entrepreneurship venture

• Served in the military

• Unemployed

• Other (describe what you did immediately after your AmeriCorps service): ________

13. Display logic – show only if selected anything besides ‘within last 6 months’ in Q10: If you got a job in the six months after your AmeriCorps service, was it a position in, or directly connected to, the organization or agency in which you served?

• Yes (please explain):_____________________

• No

14. What are you doing now? Select all that apply.

• Working in the nonprofit sector

• Working in the private sector

• Working in government or in the public sector

• Serving in the military

• Self-employed

• Looking for work

• Staying at home to take care of a family or household member or for other reasons

• Working as a supervisor or in a management capacity

• Serving in another AmeriCorps service program (another AmeriCorps program, Senior Corps, NCCC, or VISTA project)

• Serving with the Peace Corps

• Engaging in other volunteer service

• Making plans to attend school

• Attending college or graduate school

• Enrolled in a certificate, technical, or vocational program

• Other (please specify): ____________

15. Did you find or decide to pursue this opportunity based on a connection in your professional network you made during your time in serving with AmeriCorps?

• Yes (please explain):______________

• No

16. If you are working now, does your current job involve any of the organizations or AmeriCorps programs that you worked with during your AmeriCorps service?

• Yes (please explain:__________

• No

Influence of AmeriCorps Service on Career Path

17. Please rate to what extent you agree with the following statements.

	Strongly disagree	Disagree	Agree	Strongly agree	Don’t know or not applicable
Participating in AmeriCorps was a worthwhile experience in terms of furthering my personal goals and future personal endeavors.
Participating in AmeriCorps was a worthwhile experience in terms of furthering my educational goals and future educational endeavors.
Participating in AmeriCorps was a worthwhile experience in terms of furthering my professional goals and future professional endeavors.
Participating in AmeriCorps was a defining professional experience? (A defining experience is one that confirmed your professional goals or one that resulted in a change or shift in your professional goals.)
Training I received as an AmeriCorps member was particularly useful to me personally.
Training I received as an AmeriCorps member was particularly useful to me professionally.

18. How did your AmeriCorps service fit into how your career path has unfolded? Select the option that most closely describes your career path.

• It aligned with the career path I was already pursuing.

• My career took a different path into work in a service-related field after my AmeriCorps service.

• My career took a different path into non-service work after my AmeriCorps service.

• My career took a different path into non-service work after my AmeriCorps service, but then I returned to service work.

19. The following items pertain to the impact of your AmeriCorps service on your education and career goals. Please indicate the extent to which participation in AmeriCorps affected the following.

	Not at all impactful	Somewhat impactful	Impactful	Very impactful	Don’t know or not applicable
Your general employment skills (e.g., communication, problem solving, teamwork).
Your ability to use new technology/tools.
Your ability to work with people from different racial, ethnic, social, or educational backgrounds.

20. Please describe the impact that AmeriCorps has had on your career path.

__________________________________________________________________________________________________________________________________________________________________________

21. Please describe some of the career skills you learned during your AmeriCorps service term.

__________________________________________________________________________________________________________________________________________________________________________

22. Have you recommended to other people in your network (e.g., family, friends, coworkers) that they join AmeriCorps?

    • Yes

    • No

    • I’m not sure

23. Have you maintained communication with individuals from the organization or community where you served as an AmeriCorps member?

    • Yes

    • No

    • I’m not sure

Demographics

24. Have you lived most of your life in Puerto Rico?

    • Yes (continue to Q24)

    • No (skip to Q25)

    • Prefer not to say

25. Have you lived most of your life in the community where you are serving or where your project is serving?

• Yes

• No

• Prefer not to say

26. Do you identify as Puerto Rican?

    • Yes

    • No

    • Prefer not to say

27. Where are you currently living?

• Puerto Rico

• The continental United States

• Somewhere else (please specify): ____________

28. What is your age?

• Under 18

• 18–29

• 30–39

• 40–49

• 50–59

• 60–69

• 70+

• Prefer not to say

29. How do you describe your gender? (Select all that apply.)

• Male (including transgender men)

• Female (including transgender women)

• Nonbinary/nonconforming

• Prefer to self-describe as ____________

• Prefer not to say

30. Which one of these groups would you say best represents your race?

• White

• Black or African American

• American Indian or Alaska Native

• Asian

• Native Hawaiian or Pacific Islander

• Multi-racial or multi-ethnic (2+ races/ethnicities)

• Other (please specify): ____________

• Don’t know

• Prefer not to say

31. Are you of Hispanic, Latino/a, or Spanish origin?

• Yes

• No

• Don’t know

• Prefer not to say

32. Please rate your language proficiency in the following languages

	Not at all proficient	Beginning/ intermediate proficiency	Advanced proficiency	Fluent/native speaker	Not applicable
Spanish
English
Another language (please specify): ___________

33. What is the highest grade or year of school you completed?

• Never attended school or only kindergarten

• Elementary school

• Some high school

• High school graduate or equivalent

• Some college or technical school

• College graduate

• Some graduate school

• Completed graduate school

• Prefer not to say

34. Is there anything else you would like to share about your experience serving as an AmeriCorps member? ____________________________________________

Thank you for completing this survey! To enter into the raffle, please proceed to the next page.

***

As a small token of appreciation, ICF is raffling off 10 $25 gift cards to individuals who completed the survey and opt to participate in the raffle. To opt into the raffle, please provide your email address below. Your email address is not connected to the rest of your survey responses. A staff member from ICF will let you know by email if you have won the raffle and will send an electronic gift card. Your email address will not be used for any other purpose.

Email address: ________________________

Thank you!

AMERICORPS/VISTA MEMBER FOCUS GROUP
(60 minutes)

(Research question addressed is in parentheses)

AmeriCorps Member Focus Group Consent

Participation

Thank you again for taking the time to participate in today’s focus group. This focus group is part of a study being conducted by ICF for AmeriCorps. The reason for the study is to help AmeriCorps better understand how their programs work in Puerto Rico. More specifically, we are interested in your experiences as AmeriCorps/Vista members when working with [organization name] and the effect on the community you serve. We have prepared a number of discussion questions, but please feel free to offer any additional thoughts you have about your experience. The focus group will take up to 60 minutes.

Risks

We do not anticipate any risks in being a part of this focus group. Participation in today’s discussion is completely up to you. You can decide you don’t want to answer any question we ask you, and you can stop participating in the discussion at any time. You can also decide not to participate at all and leave the focus group at any point.

Anything you share with ICF will be kept confidential. By speaking with us, you will not affect your current or future relationship with [program name]. A report summarizing feedback across different organizations and AmeriCorps Members will be shared with AmeriCorps. Only ICF staff will see the data collected from this focus group, and the information that we use from our discussions will not be identified with any one individual. However, since we are only talking to a small number of members at each organization, there is a chance that AmeriCorps or [program name] personnel will be able to guess which individual shared certain information.

We also request that you do not discuss what is disclosed in this focus group once the discussion ends. Do you have any questions about this study or this focus group/focus group?

To fully capture the discussion, we would like to record the session. Again, only ICF evaluation team members will have access to the recording, which will not be shared with anybody in your program/organization or AmeriCorps. We will not include your name(s) in these notes. Any information that can be used to identify an individual will be removed prior to sharing any findings. If you do not want to participate, you may leave at this time.

Questions

Should you have any questions about the focus group, you may contact study representative Dr. Susana Mazuelas Quirce at susana.mazuelasquirce@icf.com or Dr. Samantha Spinney at samantha.spinney@icf.com or ICF’s Institutional Review Board at IRB@icf.com.

START RECORDER NOW

Background

Let’s begin with some brief introductions. Please tell us your name, where you are from, and whether you are serving as an AmeriCorps State and National member or a VISTA.

1. Why did you become an AmeriCorps State and National member/VISTA? (RQ1)

   1. How did you become aware of the program? (RQ13)

      1. What did you learn about first—AmeriCorps or the program where you are completing your service? (RQ13)

      2. What, if anything, did you know about AmeriCorps before applying? (RQ1)

      3. Does your social network—family, friends, colleagues—know what AmeriCorps is? If yes, what is their impression of AmeriCorps?

      4. How were you recruited to this program? (RQ13)

   2. What motivated you to apply to be an AmeriCorps service member? What was it about the program that most interested you? (RQ15)

   3. What were your goals and expectations for your AmeriCorps service? (RQ13)

2. How do you believe AmeriCorps service is perceived in Puerto Rico? [Probe for: Do people think of this as a job? Do they think of it as community service? Do they think of it as “national service”? Do they think of it as a “gap year” between college and work?]

3. Are you a member of the community that you are serving or are you from a different community/region?

Program Context and Activities

4. What is your role at [program/organization name]? (RQ9, RQ11)

   1. What services do you provide? What activities do you engage in as part of your service? Probe for differences between ASN members and VISTAs as applicable (RQ9, RQ11)

   2. How have your role and responsibilities evolved, if at all, from what you were initially told they would be when you were recruited? Have new tasks been added or removed from your overall responsibilities?

   3. What are your thoughts about the number of AmeriCorps members assigned to accomplish the project goals—are there too many, too few, or is the number correct? (RQ10)

   4. How effective is the organization’s structure—in terms of how the work is organized and led—for helping you to fulfill your duties? (RQ10)

   5. What factors help you to successfully complete your service assignment? What factors hinder successful completion of your service assignment?

5. How would you describe the communities who are served through your project? (RQ4, RQ9)

   1. What are the biggest needs of the communities? (RQ4)

   2. What are the biggest strengths or assets of the communities? (RQ4)

   3. How is the program perceived by the communities you serve? (RQ1)

6. Have you served as an AmeriCorps member/VISTA during a disaster (hurricane, earthquake, COVID, other)? (RQ5)

   1. What was that disaster and when did it occur?

   2. How, if at all, did the disaster impact your family and/or community?

   3. How, if at all, did your role at [program/organization name] change during the disaster?

   4. How, if at all, did you reconcile your role as an AmeriCorps member/VISTA serving during the disaster but also as an individual experiencing the disaster?

   5. To what degree did you feel supported by AmeriCorps? By your program/organization?

Training and Supports

7. What was the onboarding process like when you first began as an AmeriCorps member/VISTA at [program/organization] name? [RQ13, RQ14]

   1. Did you feel that this onboarding process was sufficient to orienting you to your roles and responsibilities? Why or why not? How could it be improved?

8. What types of training have you received to support your role as an AmeriCorps member/VISTA at [program/organization name]? (RQ14)

   1. Who provided this training? How long was the training?

   2. After the training, how prepared were you to perform your tasks?

   3. Is there any other training you need to perform your tasks?

   4. Did you receive a certification for your training?

   5. Are you planning to receive any type of certification or other credential by the end of your service? (RQ14)

9. What types of job skills have you developed through your AmeriCorps service at [organization/program name]? Probe for technical skills, administrative skills, communication skills, etc. (RQ14)

10. Through your service as an AmeriCorps member/VISTA, you have a living allowance/stipend, health insurance, and an education award. What are your perceptions of these member benefits? Probe for each member benefit. (RQ15)

    1. Is the living allowance/stipend enough to meet your living needs? Why or why not? (RQ15)

    2. What is the most important of these benefits? Least important? Why? (RQ15)

    3. How are you all planning to use the education award? (Probe for repaying loans, continuing education) (RQ15)

    4. What other benefits would be nice to receive? (RQ15)

11. Based on your own experience, who do you think would be an ideal candidate to be an AmeriCorps member/VISTA? (RQ13)

    1. What would be ideal recruiting strategies to reach these candidates? (RQ13)

Program Outcomes

12. From your perspective, in what ways has [program name] helped the community address their needs? (RQ16)

    1. Probe for the specifics of the program—only use probes that are applicable to the organization type and customize prior to interview:

       1. Disaster Services – preparing for, responding to, and recovering from natural and man-made disasters

       2. Environmental Stewardship – conserving natural habitats, improving energy efficiency, protecting clean air and water, and helping to mitigate the effects of climate change

       3. Economic Opportunity – building economic opportunity for national service members and populations served

       4. Healthy Futures – improving physical and mental well-being

       5. Education – supporting students in schools and out of school time programs

13. For VISTAs only: To what degree, if at all, do you believe that your service has helped to build the organizational capacity of [organization/program name]. What drives this opinion?

    1. What do you think are your main contributions to [organization/program name]? (RQ17)

    2. Do you believe that [organization/program name] is currently able to sustain their capacity without having VISTAs?

14. If you were in charge of making decisions about how to improve the program and the organization, what would you do? (RQ10)

15. Overall, how satisfied are you with your decision to serve with organization/program name? Why? Would you recommend this opportunity to others? [RQ13]

Plans after AmeriCorps

16. What are your plans once you finish your service term? (RQ18)

    1. Do you want to find a job, continue your education, find another service opportunity? (RQ18)

    2. What are your long-term career plans? In what ways, if any, has your AmeriCorps service influenced those plans? (RQ19)

       1. How, if at all, do you believe your AmeriCorps service has provided a pathway to accomplish these career plans?

    3. Do you want to stay in Puerto Rico or in the community where you live in or serve? Why or why not? (RQ 18) [probe for economic, educational, political, family, health, cultural, community reasons]

       1. If staying: Do you plan to move in the future? If so, where, why, and for how long?

       2. If moving: What would encourage you to stay in Puerto Rico permanently?

17. How has your experience with AmeriCorps influenced your decisions about staying or moving? (RQ18)

18. What else can AmeriCorps or [organization name] do to help you to achieve your goals after your service time? (RQ18, RQ19)

Final Thoughts

19. Do you have any suggestions to improve [program name]? [Probe for training/onboarding deficiencies, member-level challenges, etc.]

20. Do you have any additional feedback or insights you would like to share with us regarding your experiences in the program or working with [organization name]?

Thanks for your time!

AMERICORPS MEMBER/VISTA SURVEY

(Research question addressed is in parentheses)

AmeriCorps Member Survey Consent

Participation

Thank you for taking the time to participate in this survey. This survey is part of a study being conducted by ICF to help AmeriCorps and [organization/program name] better understand how their programs are working. More specifically, we are interested in your program’s activities and effect on the community you serve. The survey will take about 20 minutes to complete.

Risks

We do not anticipate any risks in participating in this survey. Participation in this survey is voluntary. You can skip any question or stop answering questions at any time.

Your responses to the survey will be kept confidential. Your answers will not affect your current or future work with AmeriCorps or with [organization/program name]. A report summarizing feedback across different organizations and AmeriCorps members will be shared with AmeriCorps and with grantees. The information that we use from this survey will not be identified with any one individual. Only ICF staff and the AmeriCorps Office of Research and Evaluation will have access to the de-identified survey data.

Questions

Should you have any questions about the focus group, you may contact study representative Dr. Susana Mazuelas Quirce at susana.mazuelasquirce@icf.com, Dr. Samantha Spinney at samantha.spinney@icf.com, or ICF’s Institutional Review Board at IRB@icf.com.

If you agree to participate in the survey, please acknowledge below by selecting, “I agree to participate.”

• I agree to participate.

• I do not agree to participate. [If selected, will use skip logic take respondent to the end of the survey]

Background

1. Please select the name of the organization where you serve.

• Alianza Municipal de Servicios Integrados Inc. (Skip Question 2, Display Questions Tagged ASN)

• Asociacion Pro Juventud y Com Bo Palmas (Skip Question 2, Display Questions Tagged ASN)

• Caras of the Americas (Skip Question 2, Display Questions Tagged ASN)

• Compania para el Desarrollo Integral de la Peninsula de Cantera (Peninsula of Cantera Project/ Casa Educativa urban garden) (Proceed to Question 2)

• Corporacion del Proyecto ENLACE del Cano Martin Pena (Proceed to Question 2)

• Fundacion Sila M. Calderon (Skip Question 2, Display Questions Tagged ASN)

• Instituto Psicopedagogico De Puerto Rico (Let's Move! Health Promotion Program for Adults with ID) (Skip Question 2, Display Questions Tagged ASN)

• Mujeres de Islas, Inc (Proceed to Question 2)

• Municipio De San Juan (Skip Question 2, Display Questions Tagged ASN)

• Oficina para la Promocion y el Desarrollo Humano (Skip Question 2, Display Questions Tagged ASN)

• Plenitud Iniciativas Ecoeducativas, Inc (Las Marias Community Center & Resiliency Hub/Plenitud agroecology farm) (Proceed to Question 2)

• Puerto Rico Youth At Risk Inc (Proceed to Question 2)

• Sistema Universitario Ana G. Mendez, Incorporado (Proceed to Question 2)

• I don’t know (Proceed to Question 2)

• Other:___________ (Proceed to Question 2)

1. Please select which type of AmeriCorps member you are.

	AmeriCorps State and National Member (Display Questions Tagged ASN)
	AmeriCorps VISTA (Display Questions Tagged VISTA)
	Don’t know (Display Questions Tagged ASN and VISTA)

3. Please indicate how important each of the following reasons were for you to apply to the AmeriCorps program. (RQ13)

	Not at all important	Slightly important	Important	Very important	Don’t know or not applicable
It provides an opportunity to use your knowledge, skills, and experience.
It is something that will help your own neighborhood or community.
It is a way to make a difference for a cause or issue that is important to you OR in which you are interested.
It is a way to meet people or gather with people.
Because you feel a personal responsibility to help others when they need it.
It makes your own life more satisfying.
It lets you learn things through direct, hands-on experience.
It helps you to explore possible career options.
It provides you with important work experience to help achieve career goals.
It provides an opportunity to earn a salary.

Source: These items are adapted from the Civic Engagement Volunteer Experience Survey and the Volunteer Motivation Scale.

Current Role and Services Provided

4. How long have you been with [grantee name] as an AmeriCorps member this term?

• Less than six months

• Six months – less than 1 year

• 1–2 years

• 3–5 years

• 5+ years

5. Do you serve part-time or full-time as an AmeriCorps member?

• Part-time

• Full-time

6. How did you learn about the opportunity to serve as an AmeriCorps member? Select all that apply.

• Through the organization where I am currently serving

• Through another organization in my community

• From friends, family, or colleagues.

• Information provided at my high school or college/university

• Social media

• Other:______________

7. Display logic - for ASN members only: What types of services have you been delivering through AmeriCorps this term? Select all that apply.

	Education services (e.g., delivering tutoring/mentoring/training services to students, providing services in after-school programs, developing educational curriculum or modules, helping reduce absenteeism/disciplinary issues, improving homework completion, supporting academic performance, supporting leadership development; or other initiatives for students)
	Disaster services (e.g., improving community knowledge/skills for addressing disasters, providing disaster readiness/mitigation services and emergency preparedness workshops, delivering disaster management services, assisting survivors, restoring communities affected by natural disasters)
	Health related services (e.g., delivering exercise or recreation programs/workshops, delivering services to prevent childhood obesity, supporting healthy lifestyles, providing nutrition information, serving as a companion/providing home support to the elderly, supporting individuals with an intellectual disability, providing food/nutrition to communities, providing mental health services/counseling/psychotherapy interventions, and other services to improve physical and mental well-being )
	Economic opportunity services (e.g., working with unemployed individuals to find jobs, providing financial literacy and work readiness information, supporting the visibility of small businesses, providing workshops on economic topics and business advisory services, helping individuals develop entrepreneurial ideas into small businesses or community organizations; supporting leadership development; and other services to support workforce development, business development, and affordable housing)
	Environmental services (e.g., maintaining and upkeeping planting sites, nurseries and pollinator gardens and harvesting fruits/vegetables; providing environmental education experiences and promoting environmental consciousness and practices; supporting the recovery/rehabilitation of community barren spaces and improving at-risk ecosystems; building community gardens, recycling disposal centers, and parks; holding environmental festivals; supporting communication with community leadership about environmental issues; supporting solar power and rain water storage initiatives, and other initiatives that support conservation of public lands and wildlife protection)
	Other (please specify): ________________________

8. Display logic - for VISTAs only: What are the types of services have you been delivering through AmeriCorps this term? Select all that apply.

• Identifying and leveraging funds/funding sources

• Conducting data-related tasks (e.g., developing data tracking/monitoring or data management systems, collecting or analyzing data)

• Developing/maintaining technology platforms

• Managing/training volunteers

• Developing/managing projects/programs

• Conducting communications/outreach/promotion services

• Developing designs, graphics, and/or maps

• Mentoring and/or accompanying others

• Conducting research and/or evaluation

• Developing community partnerships/alliances

• Systematizing programmatic services or administrative processes

• Developing, coordinating, or conducting workshops/trainings

• Developing informational materials (e.g., reports, profiles, educational/promotional materials, manuals)

• Community planning

• Other (please specify): ________

9. Were your roles and responsibilities as an AmeriCorps member clearly communicated to you before you started your service this term?

   • Yes

   • No

10. Have your roles and responsibilities as an AmeriCorps member changed since you began your service this term?

    • Yes (continue to Q11)

    • No (skip to Q12)

11. How much have your roles and responsibilities changed as an AmeriCorps member since you began your service this term?

    • They have changed very much.

    • They have changed.

    • They have somewhat changed.

    • They have not changed at all.

    • Don’t know or not applicable

12. To what extent do you agree with the following statements?

	Strongly disagree	Disagree	Agree	Strongly agree	Don’t know or not applicable
The organization where I serve has an adequate number of staff to effectively deliver services/ accomplish the mission.
The organization where I serve has an adequate distribution of tasks to effectively deliver services/ accomplish the mission.

13. How would you describe the community you serve?

• City/urban

• Country/rural

Training and Other Benefits

14. Have you received training to fulfill your role as an AmeriCorps member/VISTA?

    • Yes

    • No – Skip to Q19

    • Don’t know or not applicable – Skip to Q19

15. Was the amount of training (or number of training sessions) that you received adequate to complete fulfill your role as an AmeriCorp member/VISTA?

    • Yes

    • No

16. What was the content of the training? Select all that apply.

• Disaster response skills

• Program or project-specific training (i.e., a training on how to complete the task at the organization where I am serving)

• Technical skills (e.g., using or creating specialized tools, technology platforms, or systems; data skills)

• Interpersonal communication skills (e.g., verbal communication, active listening)

• Problem solving skills (e.g., identifying a problem, exploring alternative solutions)

• Teamwork skills (e.g., team building, conflict resolution)

• Project management skills (e.g., developing a work plan, financial management)

• Volunteer management skills (e.g., recruiting, managing, and retaining volunteers)

• Communications/outreach skills (e.g., social media, marketing, writing promotional materials)

• Career preparation skills (e.g., resume development, networking)

• Other (please specify): ________________________

17. How helpful was the content of the training for your role in the AmeriCorps program/project?

• Very helpful

• Helpful

• Somewhat helpful

• Not at all helpful

• Don’t know or not applicable

18. How helpful was the content of the training for your professional goals after you finish your service as an AmeriCorps member?

• Very helpful

• Helpful

• Somewhat helpful

• Not at all helpful

• Don’t know or not applicable

19. How satisfied are you with the following elements of your AmeriCorps organization?

	Not at all satisfied	Somewhat satisfied	Satisfied	Very satisfied	Don’t know or not applicable
Establishment of goals and priorities
Relationship with your manager/coordinator
Feedback received on your performance
Resources provided to efficiently develop your tasks
Relationship with the organization staff
Organization capacity to address community needs
Organization capacity to involve the community in the services provided

20. How satisfied are you with the following member benefits? Keep in mind that not all benefits are offered in all regions. If a benefit doesn’t apply to your region or wasn’t offered, please select “Not applicable.”

	Not at all satisfied	Somewhat satisfied	Satisfied	Very satisfied	Don’t know or not applicable
Living allowance
Professional development
Segal Education award
Health insurance
Childcare assistance
Loan deferment and interest forbearance
Alumni network
Other (please specify): ____________

21. Are you planning to use the Segal Education award or the end-of-service stipend (i.e., cash award)?

• Education award

• End-of-service stipend

22. Display logic – if selected ‘education award’ in Q21: How are you planning to use the Segal Education award? Select all that apply.

    • To pay student loans

    • To cover future education expenses at a college/university or a technical school

    • Other (please specify): ___________

23. Display logic – if selected ‘end-of-service stipend’ in Q21: How are you planning to use the end-of-service stipend (i.e., cash award)? Select all that apply.

    • To pay back loans or debt

    • To support my family

    • Travel

    • To start an organization or a small business

    • Other (please specify):______________

24. What can your AmeriCorps organization do to help you do your tasks better? __________________________________________________________________________________

Program Outcomes

25. How many terms have you served as an AmeriCorps member?

    • This is my first term

    • This is my second term

    • This is my third term

    • This is my fourth term

26. If you could serve more than four terms with AmeriCorps, would you do so?

    • Yes

    • No

    • I don’t know

27. The following items pertain to the impact of the AmeriCorps program on your future education and career goals. Please indicate the extent to which participation in the program impacted the following.

	Not at all impactful	Somewhat impactful	Impactful	Very impactful	Not applicable
Your general employment skills (e.g., communication, problem solving, teamwork).
Your ability to use new technology/tools.
Your ability to work with people from different racial, ethnic, social, or educational backgrounds.
The career/occupation you will pursue in the future.
Your ability to obtain a job in the career field you want to pursue
The degree/major you will pursue in the future

28. What are your future career plans? Select all that apply.

• Work in the public sector

• Work in the private sector

• Work in the non-profit sector

• I want to start my own business or organization

• Other (please specify):___________

29. Looking to the future, would you like a job working with the same or similar organization to the one where you are serving as an AmeriCorps member?

• Yes

• No

• I don’t know

30. Looking to the future, if you have a job opportunity in Puerto Rico, do you plan to stay in Puerto Rico?

• Yes

• No (please explain why):____________

• I don’t know

31. Display logic – if selected ‘Yes’ in Q30: If you are unable to find a job in Puerto Rico within 1 year of searching, would you move to another place outside of Puerto Rico?

• Yes

• No

• I don’t know

Demographics

32. Have you lived most of your life in Puerto Rico?

• Yes (continue to Q33)

• No (skip to Q34)

• Prefer not to say

33. Have you lived most of your life in the community where you are serving or where your project is serving?

• Yes

• No

• Prefer not to say

34. Do you identify as Puerto Rican?

• Yes

• No

• Prefer not to say

35. What is your age?

• Under 18

• 18–29

• 30–39

• 40–49

• 50–59

• 60–69

• 70+

• Prefer not to say

36. How do you describe your gender? Select all that apply.

• Male (including transgender men)

• Female (including transgender women)

• Nonbinary/nonconforming

• Prefer to self-describe as ____________

• Prefer not to say

37. Which one of these groups would you say best represents your race?

• White

• Black or African American

• American Indian or Alaska Native

• Asian

• Native Hawaiian or Pacific Islander

• Multi-racial or multi-ethnic (2+ races/ethnicities)

• Other (please specify): ____________

• Don’t know

• Prefer not to say

38. Are you of Hispanic, Latino/a, or Spanish origin?

• Yes

• No

• Don’t know

• Prefer not to say

39. Please rate your language proficiency in the following languages.

	I do not know the language	Beginning/ intermediate proficiency	Advanced proficiency	Fluent/native speaker	Not applicable
Spanish
English
Another language (please specify): ___________

40. What is the highest grade or year of school you completed?

• Never attended school or only kindergarten

• Elementary school

• Some high school

• High school graduate or equivalent

• Some college or technical school

• College graduate

• Some graduate school

• Completed graduate school

• Prefer not to say

41. Is there anything else you would like to share about your experience serving as an AmeriCorps member? ____________________________________________

PUERTO RICO COMMISSION STAFF INTERVIEW (1 hour)

(Research question addressed is in parentheses)

Puerto Rico Commission Interview Consent

Participation

Thank you again for taking the time to participate in today’s interview. This interview is part of a study being conducted by ICF for AmeriCorps. The reason for the study is to help AmeriCorps better understand how their programs work in Puerto Rico. More specifically, we are interested in knowing your role in supporting the different organizations. We have prepared a number of discussion questions, but please feel free to offer any additional thoughts you have about your experience. The interview will take up to one hour.

Risks

We do not anticipate any risks in being a part of this interview. Participation in today’s discussion is completely up to you. You can decide you don’t want to answer any question we ask you, and you can stop participating in the discussion at any time. You can also decide not to participate at all and leave the interview at any point.

Anything you share with ICF will be kept confidential. By speaking with us, you will not affect your current or future relationship with AmeriCorps. A report summarizing feedback across different organizations and AmeriCorps members will be shared with AmeriCorps. Only ICF staff will see the data collected from this interview, and the information that we use from our discussions will not be identified with any one individual. However, since we are only talking to a small number of members at each organization, there is a chance that AmeriCorps, other staff at the Comisión de Voluntariado y Servicio Comunitario, or grantee/sponsor organization personnel will be able to guess which individual shared certain information.

We also request that you do not discuss what is disclosed in this interview once the discussion ends. Do you have any questions about this study or this interview?

To fully capture the discussion, we would like to record the session. Again, only ICF evaluation team members will have access to the recording, which will not be shared with anybody in your organization or AmeriCorps. We will not include your name(s) in these notes. Any information that can be used to identify an individual will be removed prior to sharing any findings. If you do not want to participate, you may leave at this time.

Questions

Should you have any questions about the interview, you may contact study representative Dr. Susana Mazuelas Quirce at susana.mazuelasquirce@icf.com or Dr. Samantha Spinney at samantha.spinney@icf.com or ICF’s Institutional Review Board at IRB@icf.com.

START RECORDER NOW

Background

1. Let’s begin with some brief introductions. Please tell me a little bit about yourself.

   1. What is your role at the Comisión de Voluntariado y Servicio Comunitario?

   2. How long have you been in this position?

   3. What are your specific responsibilities?

Operations and Context

2. Please tell me more about the work of the commission.

   1. What is the overall mission of the commission? How, if at all, has this mission evolved over time?

   2. What are the major ways that the commission supports AmeriCorps State and National programs in Puerto Rico?

3. How are national service and AmeriCorps perceived in Puerto Rico? (RQ1)

   1. How do AmeriCorps grantee organizations perceive AmeriCorps and national service? (Probe for if the organizations see AmeriCorps as a service entity, an employment/workforce entity, or something else)

   2. How do you believe that organizations without AmeriCorps projects perceive AmeriCorps, if at all?

   3. How do you believe that community members perceive AmeriCorps, if at all?

   4. In your opinion, what are the benefits of having AmeriCorps State and National projects in Puerto Rico? (RQ 1)

      1. How does national service help local organizations reach their goals? (RQ 1)

4. What are the biggest community and environmental contextual factors for AmeriCorps projects in Puerto Rico? (RQ4)

   1. What are the biggest needs of communities served through AmeriCorps projects?

   2. What are the biggest assets of communities served through AmeriCorps projects?

5. What have been the impacts of disasters (e.g., hurricanes, earthquakes, COVID-19 pandemic) on communities served by AmeriCorps projects? (RQ5)

   1. How have disasters impacted the work of the commission?

   2. How has the commission supported AmeriCorps projects in Puerto Rico during major recent disasters?

   3. How has AmeriCorps headquarters supported Puerto Rico-based grantee organizations during recent disasters? Was additional support needed?

6. In what ways does the commission work with AmeriCorps headquarters to support national service in Puerto Rico? How easy or difficult is it for the commission to work with AmeriCorps headquarters? (RQ3)

   1. What are the biggest successes that have come from the commission’s interactions with AmeriCorps headquarters?

   2. What are the biggest challenges associated with interactions with AmeriCorps headquarters?

   3. Which language(s) is used to communicate with AmeriCorps headquarters? How, if at all, does language pose a challenge in interactions between the commission and AmeriCorps headquarters? (RQ8)

Accessing, Maintaining, and Expanding National Service in Puerto Rico

7. What, in your opinion, are grantee organizations’ primary motivations to apply for an AmeriCorps State and National grant? (RQ7)

   1. How do organizations typically become aware of AmeriCorps funding opportunities?

   2. What is the degree of reception and enthusiasm by the local organizations to apply for AmeriCorps State and National grants and host AmeriCorps members? (RQ1, RQ7)

8. What are the biggest challenges or barriers that organizations face in accessing AmeriCorps projects in Puerto Rico? (Probe for the match in funding requirement, administrative/reporting requirements, language/translation requirements) (RQ2, RQ3, RQ8)

9. For those organizations that are funded, what are the biggest barriers that they face in maintaining their AmeriCorps projects? (RQ2)

10. What are the biggest barriers that grantee organizations face in expanding their AmeriCorps projects—via larger grants and additional AmeriCorps members? (RQ3)

11. How does the commission work with organizations to support access to and expansion of AmeriCorps projects/national service Puerto Rico? (RQ3)

    1. What types of support are offered to grantee organizations in applying for new grants?

    2. What types of support are offered to grantee organizations in maintaining their grants?

    3. What types of support are offered to non-funded/non-supported organizations in accessing AmeriCorps grants for the first time?

    4. What other supports do organizations receive in successfully accessing and maintaining their AmeriCorps grants?

    5. What supports are still needed by organizations to access, maintain, or expand AmeriCorps projects/national service in Puerto Rico?

AmeriCorps State and Local Grant Implementation

12. What is the overall profile of the portfolio of AmeriCorps State and National grant projects overseen by the commission? (RQ9)

    1. What are the commonalities across projects in terms of the types of services that are delivered?

    2. What are the commonalities across projects in how services are delivered?

    3. Are their focus and goals what the communities in Puerto Rico need?

    4. Which other areas would need more attention and volunteering programs?

13. Which grantee organizations serve as intermediaries for grant funds (i.e., the grantee organization subgrants out funds and provides AmeriCorps members to additional partner institutions)? (RQ6)

    1. What are the benefits of this arrangement?

    2. What are the drawbacks of this arrangement?

14. How satisfied is the Commission with the implementation of AmeriCorps projects in Puerto Rico? (RQ10)

    1. What are the main successes and challenges faced by organizations when implementing the programs? (RQ10)

15. What are the grantee organizations’ experiences fulfilling their AmeriCorps performance measures and the evaluation requirements? What are some challenges that local organizations have encountered with these requirements? (RQ12)

16. What are the grantee organizations’ experiences in translating data, documents, and other materials from Spanish to English? (RQ8)

    1. Are there any other challenges experienced by grantee organizations related to the need to work in two languages? (RQ8)

17. To what degree, if any, is the commission involved in providing training to AmeriCorps members? (RQ 14)

    1. Are you aware of any specific training needs that are currently unmet?

Final thoughts

18. Do you have any additional feedback or insights you would like to share with us regarding the work of the commission to support national service in Puerto Rico?

Thanks for your time!

COMMUNITY MEMBER FOCUS GROUP
(60 minutes)

(Research question addressed is in parentheses)

Community Member Focus Group Consent

Participation

Thank you again for taking the time to participate in today’s focus group. This focus group is part of a study being conducted by ICF for AmeriCorps. The reason for the study is to help AmeriCorps better understand how their programs work in Puerto Rico. More specifically, we are interested in your perception of the programs/activities that [name of organization], an AmeriCorps-supported organization, provides in your community. We have prepared a number of discussion questions, but please feel free to offer any additional thoughts you have about your experience. The focus group will take up to 60 minutes.

Risks

We do not anticipate any risks in being a part of this focus group. Participation in today’s discussion is completely up to you. You can decide you don’t want to answer any question we ask you, and you can stop participating in the discussion at any time. You can also decide not to participate at all and leave the focus group at any point.

Anything you share with ICF will be kept confidential. By speaking with us, you will not affect your current or future relationship with [program name]. A report summarizing feedback across different organizations and AmeriCorps members will be shared with AmeriCorps. Only ICF staff will see the data collected from this focus group, and the information that we use from our discussions will not be identified with any one individual. However, since we are only talking to a small number of members from each organization, there is a chance that AmeriCorps or [program name] personnel will be able to guess which individual shared certain information.

We also request that you do not discuss what is disclosed in this focus group once the discussion ends. Do you have any questions about this study or this focus group?

To fully capture the discussion, we would like to record the session. Again, only ICF evaluation team members will have access to the recording, which will not be shared with anybody in your program/organization or AmeriCorps. We will not include your name(s) in these notes. Any information that can be used to identify an individual will be removed prior to sharing any findings. If you do not want to participate, you may leave at this time.

Questions

Should you have any questions about the focus group, you may contact study representative Dr. Susana Mazuelas Quirce at susana.mazuelasquirce@icf.com or Dr. Samantha Spinney at samantha.spinney@icf.com or ICF’s Institutional Review Board at IRB@icf.com.

START RECORDER NOW

Background

1. Let’s begin with some brief introductions. Please tell us your name and a little bit about yourself, including how long you have lived in your community.

Community Context and Needs

2. Can you tell us about your community? What is it like? (RQ4)

   1. How would you describe the individuals/families who live here? (RQ4)

   2. What are the biggest needs in your community? (RQ4) Probe for

      1. Preparing for, responding to, and recovering from natural and man-made disasters

      2. Conserving natural habitats, improving energy efficiency, protecting clean air and water, and helping to mitigate the effects of climate change

      3. Economic opportunity

      4. Physical and mental health and well-being

      5. Education

3. What are the biggest strengths or assets of your community? (RQ4)

4. How, if at all, has your community changed in the past 5 years? (RQ4)

Program Implementation

5. What can you tell us about the work of [organization/program name] in your community? (RQ9)

   1. What services/activities do they provide in your community? (RQ9)

   2. How does the community feel about their services/activities? (RQ9)

   3. Do you provide feedback on the program to [organization name]? How, if at all, do you contribute to the development, content, or activities of the program?

6. Before today, had you ever heard of AmeriCorps? (RQ1)

   1. If yes, what did you know about AmeriCorps? (RQ1)

   2. Were you aware that [organization name] receives support from AmeriCorps? (RQ1)

   3. Have you had any interaction with AmeriCorps members? If yes, what have those interactions been? What did you think about their role in your community? (RQ1)

Program Outcomes

7. In what ways has [program name] affected you or your family? (RQ16) Probe for the following topics as applicable to the organization:

1. Preparing for, responding to, and recovering from natural and man-made disasters

2. Conserving natural habitats, improving energy efficiency, protecting clean air and water, and helping to mitigate the effects of climate change

3. Economic opportunity

4. Physical and mental health and well-being

5. Education

8. In what ways has [program name] affected the community in general? (RQ16)

9. If [organization name] can’t provide these services anymore, what would you or the community do?

10. Do you have any final thoughts about the work of [program name] in your community?

Thanks for your time!

PARTNER ORGANIZATION INTERVIEW/FOCUS GROUP
(1 hour)

(Research question addressed is in parentheses)

Partner Organization Interview/focus group Consent Statement

Participation

Thank you again for taking the time to participate in today’s interview/focus group. This interview/focus group is part of a study being conducted by ICF for AmeriCorps. The reason for the study is to help AmeriCorps better understand how their programs work in Puerto Rico. More specifically, we are interested in your organization’s activities, partnership with [program name], and effect on the community you serve. We have prepared a number of discussion questions, but please feel free to offer any additional thoughts you have about your experience. The interview/focus group will take up to one hour.

Risks 

We do not anticipate any risks in being a part of this interview/focus group. Participation in today’s discussion is completely up to you. You can decide you don’t want to answer any question we ask you, and you can stop participating in the discussion at any time. You can also decide not to participate at all and leave the interview/focus group at any point. 

Anything you share with ICF will be kept confidential. By speaking with us, you will not affect your current or future relationship with [program name], the Comisión de Voluntariado y Servicio Comunitario, or AmeriCorps. A report summarizing feedback across different organizations and AmeriCorps members will be shared with AmeriCorps. Only ICF staff will see the data collected from this interview/focus group, and the information that we use from our discussions will not be identified with any one individual. However, since we are only talking to a small number of members at each organization, there is a chance that AmeriCorps, the Comisión de Voluntariado y Servicio Comunitario, or [program name] personnel will be able to guess which individual shared certain information.  

We also request that you do not discuss what is disclosed in this interview/focus group once the discussion ends. Do you have any questions about this study or this interview/focus group? 

To fully capture the discussion, we would like to record the session. Again, only ICF evaluation team members will have access to the recording, which will not be shared with anybody in your program/organization, the Comisión de Voluntariado y Servicio Comunitario, or AmeriCorps. We will not include your name(s) in these notes. Any information that can be used to identify an individual will be removed prior to sharing any findings. If you do not want to participate, you may leave at this time. 

Questions 

Should you have any questions about the interview/focus group, you may contact study representative Dr. Susana Mazuelas Quirce at Susana.mazuelasquirce@icf.co or Dr. Samantha Spinney at Samantha.spinney@icf.com or ICF’s Institutional Review Board at IRB@icf.com.

START RECORDER NOW

Background

1. Let’s begin with some brief introductions. Please tell me a little bit about yourself.

1. What is your title and role at [partner organization name]?

2. How long have you been in this position?

3. What are your specific responsibilities?

Program Operations and Context

2. First, I’d like some background of your organization. (RQ6)

   1. What is the mission of your organization?

   2. How large is your organization? About how many staff work at your organization?

3. In what ways does your organization partner with [grantee/program name]? (RQ6)

   1. How, if at all, does your organization use AmeriCorps members or VISTAs to accomplish your organization’s mission and goals? (RQ9)

      1. What are the specific projects or programs that AmeriCorps members/VISTAs support?

      2. What are their roles and responsibilities?

      3. What types of activities do they engage in?

   2. How is this partnership mutually beneficial?

   3. When did your organization begin partnering with [grantee/program name]?

   4. How has this relationship been maintained over time?

   5. How would you describe the relationship today?

4. What are the characteristics of the communities and individuals served through your AmeriCorps-supported program/program? (RQ9)

1. What are the main needs of these communities? (RQ4)

2. What are the biggest assets of the communities? (RQ4)

3. How are community members involved in program activities? Are they involved in program planning or implementation of activities?

4. What have been the impacts of disasters (e.g., hurricanes, earthquakes, COVID-19) on your programs? (RQ5)

Program Implementation

5. What are the benefits and challenges for your organization of working with [grantee name] on an AmeriCorps program? (RQ1, RQ10)

1. What have been your experiences in working with AmeriCorps members/VISTAs? What has worked well? What has been challenging?

6. What type of training, if any, does your organization need to provide to AmeriCorps members/VISTAs so that they can fulfill their role at your organization?

   1. What certifications, if any, do AmeriCorps members/VISTAs receive during their service at your organization? (RQ14)

   2. Is there any other training you think your AmeriCorps/VISTA members need to fulfill their tasks, but your organization is unable to provide?

7. Is your AmeriCorps-supported project or program being implemented as intended? In what ways, if any, has implementation evolved? (RQ10)

   1. What aspects of the program do you think are most effective? What do you think could be improved in how the program was structured and implemented?

Program Outcomes

8. Let’s focus on your program outcomes. In what ways has your partnership with (grantee/project name) helped address identified community needs? (RQ16) Use the probes below based on the specific program

   1. Disaster Services: On preparing for, responding to, and recovery from natural disasters or a health crisis, such as the COVID-19 pandemic? [probe for number of individuals served – toolkits delivered – workshops -- structures protected or restored] (RQ16)

   2. Environmental Stewardship: On conserving natural habitats, improving energy efficiency, protecting clean air and water, and helping in general to mitigate the effects of climate change? [probe for number of trainings/workshops -- acres of public parks or other public lands that are treated/improved – water supplies -- housing units or public structures weatherized or retrofitted to improve energy efficiency] (RQ16a)

   3. Economic Opportunity: On building economic opportunity for populations served? [probe for number of individuals who get training/assistant to build resumes/find a job/apply for assistance --new or renovated business] (RQ16b)

   4. Healthy Futures: On improving physical and mental well-being? [probe for number of individuals/schools served -- hours of workshops received, interventions, referrals] (RQ16c)

   5. Education: On supporting students in schools and out of school time programs? [probe for number of students/school served – workshops, activities – hours of tutoring – pass/fail rates – graduation rates] (RQ16d)

9. How do you believe that AmeriCorps service at your organization is impacting members’/VISTAs’ career pathways? (RQ18, RQ19)

   1. How, if at all, does AmeriCorps service at your organization improve members’/VISTAs’ skillsets and employability?

      1. How, if at all, does service at your organization provide a pathway for members/VISTAs to remain in Puerto Rico for work after their service term?

Interest in AmeriCorps Projects

10. Have you ever applied or considered applying for your own AmeriCorps project—either for an AmeriCorps grant or a VISTA project?

    1. If yes: Tell me more about this—have you applied before or just looked into applying?

       1. If already applied:

          1. What type of AmeriCorps project did you apply to? (Probe for AmeriCorps State and National or VISTA project)

          2. What was the result of that application?

          3. What were your primary motivations for applying for this project?

          4. What was the vision for this project and how your organization would use AmeriCorps support?

          5. What supports, if any, did you receive in preparing your application?

          6. Does your organization intend to apply again? Why or why not?

       2. If just looked into applying:

          1. What type of AmeriCorps project are you most interested in? (Probe for AmeriCorps State and National or VISTA project)

          2. What are your primary motivations for applying for this project?

          3. What is your vision for this project and how your organization would use AmeriCorps support?

          4. What supports, if any, have you received in thinking about this application? (RQ3)

          5. What barriers, if any, does your organization face in submitting the application? (Probe for match in funding requirement) (RQ3)

          6. Is your organization currently planning to submit an application? Why or why not? (RQ3)

    2. If no: Why not?

       1. Are the specific constraints or challenges that influenced your decision? (Probe for match in funding requirement, administrative/fiscal/data requirements, English language requirements) (RQ3)

11. Do you have any additional feedback or insights you would like to share with us regarding [program name] or working with [grantee name]?

Thank you for your time!

GRANTEE/ORGANIZATION DIRECTOR/STAFF INTERVIEWS (45 minutes + 60 minutes)

(Research question addressed is in parentheses; (Questions for Interview #1 are in black. Questions for interview #2 are in blue)

Grantee/Sponsor Project Director Interview Consent

Participation

Thank you again for taking the time to participate in today’s interview. This interview is part of a study being conducted by ICF for AmeriCorps. The reason for the study is to help AmeriCorps better understand how their programs work in Puerto Rico. More specifically, we are interested in your program/project’s activities and effect on the community you serve. We have prepared a number of discussion questions, but please feel free to offer any additional thoughts you have about your experience. The interview will take up to 45 minutes/60 minutes.

Risks

We do not anticipate any risks in being a part of this interview. Participation in today’s discussion is completely up to you. You can decide you don’t want to answer any question we ask you, and you can stop participating in the discussion at any time. You can also decide not to participate at all and leave the interview at any point.

Anything you share with ICF will be kept confidential. By speaking with us, you will not affect your current or future relationship with [program name], the Comisión de Voluntariado y Servicio Comunitario, or AmeriCorps. A report summarizing feedback across different organizations and AmeriCorps members will be shared with AmeriCorps. Only ICF staff will see the data collected from this interview, and the information that we use from our discussions will not be identified with any one individual. However, since we are only talking to a small number of members at each organization, there is a chance that AmeriCorps, the Comisión de Voluntariado y Servicio Comunitario, or [program name] personnel will be able to guess which individual shared certain information.

We also request that you do not discuss what is disclosed in this interview once the discussion ends. Do you have any questions about this study or this interview?

To fully capture the discussion, we would like to record the session. Again, only ICF evaluation team members will have access to the recording, which will not be shared with anybody in your program/organization, the Comisión de Voluntariado y Servicio Comunitario, or AmeriCorps. We will not include your name(s) in these notes. Any information that can be used to identify an individual will be removed prior to sharing any findings. If you do not want to participate, you may leave at this time.

Questions

Should you have any questions about the interview, you may contact study representative Dr. Susana Mazuelas Quirce at susana.mazuelasquirce@icf.com or Dr. Samantha Spinney at samantha.spinney@icf.com or ICF’s Institutional Review Board at IRB@icf.com.

START RECORDER NOW

Background (Interview #1, Repeated in Interview #2 if Needed)

1. Let’s begin with some brief introductions. Please tell me a little bit about yourself.

   1. What is your title and role at [organization name]?

   2. How long have you been in this position?

   3. What are your specific responsibilities?

   4. How long has your organization been working with AmeriCorps members?

Program Operations and Context (Interview #1)

I’d like to ask you a few questions about your programs and the community that you serve.

2. First, I’d like some background of your organization.

   1. What is the mission of your organization?

   2. When was your organization founded?

   3. How large is your organization? How many offices does your organization have? About how many staff work at your organization? Are any of the current staff former AmeriCorps/VISTA members? (RQ6)

   4. What is the leadership structure of your organization? [see if they can provide an org chart or at least describe it]

   5. How are decisions made at your organization? What is the governance structure? (RQ6)

   6. Does your organization engage volunteers that are not AmeriCorps members? If so, how many?

   7. What other funding sources support your organization besides AmeriCorps? [Probe for other grants, the commonwealth, private donations, etc.]

3. Before your organization applied for an AmeriCorps project, what, if anything, did you know about AmeriCorps? (RQ1)

4. What motivated your organization to apply for an AmeriCorps State and National grant and/or to be a VISTA sponsor? (RQ7) [Probe for differences in motivations]

   1. What was the process to apply to be an AmeriCorps grantee/VISTA sponsor? (RQ7)

   2. What challenges, if any, did you experience during the application process? (RQ2?)

   3. Were you required to provide matching funds to receive AmeriCorps funding? How, if at all, did the match in funding requirement serve as a barrier to applying? (RQ3)

   4. What individuals or resources helped to help you to gather documentation, complete forms, and in general, submit the application? How, if at all, did the Puerto Rico service commission provide guidance or support? What about AmeriCorps headquarters? (RQ7)

   5. For organizations that are both an AmeriCorps State and National grantee and a VISTA sponsor: Why did you decide to apply for both? Did you apply to both at the same time or at different times? [If different times] What prompted your organization to apply for this second program? (RQ7)

   6. Has your organization considered applying for a larger AmeriCorps grant?

   7. If yes: What factors have influenced your organization’s decision about whether to pursue a larger AmeriCorps project? What are the barriers to expanding the size of your organization’s AmeriCorps grant project? (RQ3)

5. FOR ASN PROJECTS: What types of programs and services does your organization provide through its AmeriCorps State and National grant? (RQ6, RQ9)

   1. What are the major goals and priorities of the project? (RQ9)

   2. Do any aspects for the project follow a specific program model? Which ones? Why are you following this model? (RQ9)

   3. Which specific activities does your organization engage in? (RQ9)

6. FOR ASN PROJECTS: Do you partner with other organizations to deliver services/activities for your AmeriCorps State and National grant? What are the number and types of partners that your organization is working with? (RQ6)

   1. How would you describe your relationship with these organizations?

   2. Does your organization function almost like an intermediary in dispersing AmeriCorps grant funds and national service members to smaller organizations? If yes, please describe how this works.

7. FOR ASN PROJECTS: What are the characteristics of the communities and individuals served through your AmeriCorps State and National grant? (RQ9).

1. How are community members involved in program activities? Are they involved in program planning or implementation of activities?

8. For VISTA Projects: How is your organization’s VISTA project supporting capacity building at your organization? (RQ11)

   1. What types of capacity building services are delivered through project?

   2. How, if at all, are partners involved in the VISTA project?

   3. How, if at all, do the capacity building services support or complement the services provided through the AmeriCorps State and National grant?

9. Overall, what are the major community and environmental contextual factors for your organization’s AmeriCorps projects? (RQ4)

   1. What are the biggest assets in the community or environment that support project implementation? (RQ4)

   2. What are the biggest challenges in the community or environment that hinder program implementation? (RQ4)

   3. What have been the impacts of disasters (e.g., hurricanes, earthquakes, COVID-19) on your programs? (RQ5)

      1. How, if at all, did AmeriCorps provide support to your organization during those times? For example, more flexibility with documentation and deadlines, a revision of the goals, outcomes, or performance measures, etc. (RQ5)

Program Implementation (Interview #2)

10. How easy or difficult is it for your organization to maintain your AmeriCorps grant and, if applicable, VISTA project? What are the biggest challenges your organization faces in maintaining your project(s)? (RQ10) [Probe for differences between ASN and VISTA projects]

    1. What materials and data need to be translated between Spanish and English for the AmeriCorps project? What is your organization’s experience with this translation? In what way does translation serve as a challenge to your organization? (RQ13, RQ8)

    2. Are there other particular administrative or fiscal aspects of the project that serve as barriers? (RQ10)

    3. How, if at all, does the commission support your organization in maintaining its grant? (RQ10)

    4. How if at all, does AmeriCorps headquarters support your organization in maintaining its grant? (RQ10)

    5. What else can AmeriCorps and the Commission do to help your organization and other similar organizations to fulfill your goals? (RQ10)

11. Which kind of data does your organization collect and use to support organizational operations? (RQ12)

    1. What data do you collect to support AmeriCorps progress reporting requirements? (RQ12)

    2. What data do you collect, if any, to measure other aspects of your programs’ outcomes and impact? (RQ12)

    3. What tools do you use to collect data? (probe for surveys, forms, sign up sheets)? Are they sufficient to capture the data your organization needs? (RQ12)

    4. How does your organization store or manage data? (Probe for spreadsheets, Salesforce, etc.) (RQ12)

    5. How, if at all, does your organization use data to improve your organization’s programs and services? (RQ12)

    6. Which other data would be beneficial for you to have? (RQ12)

12. Thinking about how this program/service was structured and implemented, is the program operating as intended? If not, in what ways is it different than intended? Why were those changes made? (RQ10)

    1. What aspects of the program do you think are most effective and why? (RQ10)

    2. What facilitators have assisted the program in meeting intended goals? (RQ10) [probe for organizational infrastructure capabilities]

    3. What barriers has the program faced? (RQ10) [probe for organizational infrastructure capabilities]

AmeriCorps/VISTA Members

13. Let’s now shift our attention to the AmeriCorps/VISTA members. First, what do AmeriCorps/VISTA members bring to your organization? (RQ1)

    1. How many AmeriCorps/VISTA members are in your organization? Is that a sufficient number to accomplish the program goals? Why or why not?

    2. What is the role of AmeriCorps members in each of these programs? (RQ1)

    3. Specifically for VISTA members, how do they support your organization? (RQ11)

14. What types of outreach and strategies do you use for recruitment of AmeriCorps members/VISTAs? (RQ13)

    1. What are the most effective? And the least effective outreach and recruitment strategies? (RQ13)

    2. Do you recruit AmeriCorps members/VISTAs within the communities you serve? Why or why not? (RQ13)

    3. How, if at all, do recruitment strategies differ between AmeriCorps members and VISTAs?

    4. What recruitment materials/resources developed by AmeriCorps headquarters—in English or Spanish—does your organization use, if any? How do you use those materials/resources? How can those materials be improved, if at all?

15. What is the profile of the ideal applicant when recruiting prospective AmeriCorps members/VISTAs for your organization? (RQ13)

    1. How easy or difficult is it to find these applicants? (RQ13)

    2. Generally, what is the profile of an individual interested in serving as an AmeriCorps member/VISTA in your organization?

    3. Why do you believe that individuals are interested in serving as AmeriCorps members/VISTAs in your organization? (RQ13)

16. What strategies does your organization use to manage/supervise AmeriCorps members and VISTAs at your organization? (RQ13)

    1. How, if at all, does these strategies support retention of AmeriCorps members at your organization during their service term?

17. Which types of training do AmeriCorps members/VISTAs have through their service at your organization? (RQ14, RQ19)

    1. Who provided the training(s)? How were they delivered?

    2. What certifications, if any, do AmeriCorps members/VISTAs receive during their service?

    3. Is there any other training you think your AmeriCorps/VISTA members need to fulfill their tasks, but your organization is unable to provide?

18. How do you believe that AmeriCorps service at your organization is impacting members’/VISTAs’ career pathways? (RQ18, RQ19)

    1. Have you kept in touch with former members/VISTAs? If yes:

       1. What types of work are they doing now?

       2. About what proportion have stayed in Puerto Rico after their service term? For those who have stayed in Puerto Rico, have they remained in the communities in which they have served or moved elsewhere on the island?

    2. How, if at all, does AmeriCorps service at your organization improve members’/VISTAs’ skillsets and employability?

       1. How, if at all, does service at your organization provide a pathway for members/VISTAs to remain in Puerto Rico for work after their service term?

Program Outcomes

19. Let’s focus on your program outcomes. In what ways have your AmeriCorps projects helped address identified community needs? (RQ16) Use the probes below based on the specific program

    1. Disaster Services: On preparing for, responding to, and recovery from natural disasters or a health crisis, such as the COVID-19 pandemic? [probe for number of individuals served – toolkits delivered – workshops -- structures protected or restored] (RQ16)

    2. Environmental Stewardship: On conserving natural habitats, improving energy efficiency, protecting clean air and water, and helping in general to mitigate the effects of climate change? [probe for number of trainings/workshops -- acres of public parks or other public lands that are treated/improved – water supplies -- housing units or public structures weatherized or retrofitted to improve energy efficiency] (RQ16a)

    3. Economic Opportunity: On building economic opportunity for populations served? [probe for number of individuals who get training/assistant to build resumes/find a job/apply for assistance --new or renovated business] (RQ16b)

    4. Healthy Futures: On improving physical and mental well-being? [probe for number of individuals/schools served -- hours of workshops received, interventions, referrals] (RQ16c)

    5. Education: On supporting students in schools and out of school time programs? [probe for number of students/school served – workshops, activities – hours of tutoring – pass/fail rates – graduation rates] (RQ16d)

20. For VISTA projects: What are the outcomes of the VISTA capacity building services on your organization? (RQ17)

    1. How, if at all, has your VISTA project supported your organization’s sustainability?

21. Do you have any additional feedback or insights you would like to share with us regarding your programs or the AmeriCorps/Vista members?

Thank you for your time!

NON-SUPPORTED ORGANIZATION FOCUS GROUP (45 minutes)

(Research question addressed is in parentheses)

Non-Supported Organization Focus Group Consent

Participation

Thank you again for taking the time to participate in today’s focus group. This focus group is part of a study being conducted by ICF for AmeriCorps. The reason for the study is to help AmeriCorps better understand how their programs work in Puerto Rico. More specifically, we are interested in knowing your experience either applying for AmeriCorps State and National grants or VISTA projects and any perceptions about the process. We have prepared a number of discussion questions, but please feel free to offer any additional thoughts you have about your experience. The focus group will take up to 45 minutes.

Risks

We do not anticipate any risks in being a part of this focus group. Participation in today’s discussion is completely up to you. You can decide you don’t want to answer any question we ask you, and you can stop participating in the discussion at any time. You can also decide not to participate at all and leave the focus group at any point.

Anything you share with ICF will be kept confidential. By speaking with us, you will not affect your current or future relationship with AmeriCorps. A report summarizing feedback across different organizations and AmeriCorps members will be shared with AmeriCorps. Only ICF staff will see the data collected from this focus group, and the information that we use from our discussions will not be identified with any one individual. However, since we are only talking to a small number of members at each organization, there is a chance that AmeriCorps or Comisión de Voluntariado y Servicio Comunitario personnel will be able to guess which individual shared certain information.

We also request that you do not discuss what is disclosed in this focus group once the discussion ends.

Do you have any questions about this study or this focus group?

To fully capture the discussion, we would like to record the session. Again, only ICF evaluation team members will have access to the recording, which will not be shared with anybody in your program/organization, the Comisión de Voluntariado y Servicio Comunitario, or AmeriCorps. We will not include your name(s) in these notes. Any information that can be used to identify an individual will be removed prior to sharing any findings. If you do not want to participate, you may leave at this time.

Questions

Should you have any questions about the focus group, you may contact study representative Dr. Susana Mazuelas Quirce at susana.mazuelasquirce@icf.com or Dr. Samantha Spinney at samantha.spinney@icf.com or ICF’s Institutional Review Board at IRB@icf.com.

START RECORDER NOW

Background

1. Let’s begin with some brief introductions. Please tell me a little bit about yourself.

   1. What is your title and role at [organization name]?

   2. How long have you been in this position?

   3. What are your specific responsibilities?

Program Operations, Context, and Outcomes

I’d like to ask you a few questions about your programs and the community that you serve.

2. What is the mission of your organization? What types of programs and services does your organization provide?

3. What are the characteristics of the communities and individuals served through the programs?

4. In what ways have your programs helped the community address their needs?

5. How, if at all, do you currently use volunteers to accomplish your organization’s mission?

6. What are the greatest needs of your organization to strengthen capacity?

7. Does your organization currently receive grants to support your work? What types of grants?

8. How, if at all, does your organization collect and manage program data?

Experiences and perceptions of AmeriCorps

9. You all are from organizations that previously applied for an AmeriCorps State and National grant or VISTA project. What motivated your organization to apply for an AmeriCorps State and National grant and/or to be a VISTA sponsor? (RQ7)

   1. How did you become aware of the AmeriCorps opportunity(ies)? 

10. Describe the program/services you hoped to support through the AmeriCorps project.

11. What was the process like to apply for an AmeriCorps State and National grant or VISTA project? (RQ7) 

    1. What challenges, if any, did you experience during the application process? (Probe for the match in funding requirement, submission of an application in English, the evaluation requirement) (RQ3)

    2. Which resources or guidance, if any, did the Commission offer to help you complete and submit your application? (RQ7) How useful were these resources or guidance?

    3. What resources or guidance, if any, did AmeriCorps headquarters offer (e.g., webinar, materials on the website, individual staff who answered questions) to help you complete and submit your application? How useful were these resources or guidance?

       1. Did you find any Spanish language resources to help you? If yes, to what degree did these resources provide you with helpful information?

12. Do you plan to keep applying for future AmeriCorps projects?

    1. If so, what do you plan to do differently for your next application?

    2. What resources or guidance do you need to help put together a successful next application?

13. Do you have any additional feedback or insights you would like to share with us regarding your experience with the AmeriCorps State and National grant/VISTA sponsorship application process?

Thanks for your time!

PRE-/POST-EVALUATION TECHNICAL ASSISTANCE PROGRAM ASSESSMENT

Instructions: On behalf of AmeriCorps (formerly known as the Corporation for National and Community Service), ICF is providing evaluation capacity building workshops throughout one year to grantees and sites that are participating in the [bundle name] evaluation. We are inviting you to complete this 15-minute survey at the beginning and end of your participation.

Participation in this survey is voluntary, but we hope you will participate because your experience and perspective are extremely valuable to assess the success of these workshops.

1. Awareness/Attitudes/Motivation:

To what extent do you agree with the following statements (1 – Strongly Disagree, 2 – Disagree, 3 – Neither Agree nor Disagree, 4 – Agree, 5 – Strongly Agree):

A. I think that an evaluation…

1. Will help me understand my program.

2. Will inform the decisions I make about my program.

3. Will justify funding for my program.

4. Will help to convince managers that changes are needed in my program.

5. Is absolutely necessary to improve my program.

6. Should involve program participants in the evaluation process.

7. Will help improve services to target populations.

8. Evaluation takes away resources more than can be used to provide services.

9. Evaluation is incorporated in my daily work.

10. I review evaluation reports regularly.

11. Evaluation findings affect the decisions I make in my work.

B. I am motivated to…

1. Learn about evaluation.

2. Start evaluating my program.

3. Support other staff to evaluate their program.

4. Encourage others to buy into evaluating our program

C. To what extent are the following factors important in your motivation to engage in program evaluation? (1 – Not at all, 2 – To a small extent, 3 – To a moderate extent, 4 – To a large extent, 5 – Always)

1. A need to meet accountability requirements

2. A desire to enlighten and support government policy making and planning

3. Changes in the organization (e.g., reorganization, new leadership, or vision)

4. A mandate from leadership to increase the learning function of evaluation

5. A perceived lack of internal evaluation knowledge and skills

6. A desire to seek new or increased funding

7. A desire to use evaluation to make program improvements

8. other factors (specified)

2. Skills/Knowledge/Behaviors:

A. To what extent do you agree with the following statements? (1 – Not at all, 2 – To a small extent, 3 – To a moderate extent, 4 – To a large extent, 5 – Always)

I know how to…

1. Recognize key concepts in evaluation (evidence, data, performance measurement, evaluation).

2. Recognize the components of an evaluation plan.

3. Describe what a “theory of change” is

4. Recognize how theory of change connects to a logic model.

5. Recognize the attributes of a good evaluation question.

6. Identify strategies to collect information from participants.

7. Recognize features of process and outcome indicators of a program.

8. Recognize who should collect data.

9. Recognize when data should be collected.

10. Identify common data analysis terms.

11. Recognize how quantitative and qualitative analysis is performed.

12. Recognize how to use the evaluation findings to improve a program.

13. Read an evaluation report and recognize its basic components.

B. To what extent do you agree with the following statements? (1 – Not at all, 2 – To a small extent, 3 – To a moderate extent, 4 – To a large extent, 5 – Always)

In the past year, I helped my organization to

1. Provide funding to carry out an evaluation or hire an external evaluator (as part of a grant or through other means)

2. Provide training or technical assistance to conduct evaluation (by your staff, consultants, or other means)

3. Conduct evaluations of programs funded by your organization

4. Use evaluation findings to improve programs funded by your organization

5. Use evaluation findings to demonstrate and communicate effectiveness of programs funded by your organization

C. Does your organization have any of the following? (1 – Not at all, 2 – To a small extent, 3 – To a moderate extent, 4 – To a large extent, 5 – Always)

1. Staff position(s) or group within your organization dedicated to evaluation

2. External evaluation partner(s) -- consultant(s) or organization(s) that provide your organization with evaluation services

3. Part of the organizations' budget dedicated to evaluation

4. Evaluation processes that are integrated into ongoing organizational practices

5. Appropriate capacity, systems, and expertise in place internally to conduct high-quality evaluation.

6. Sufficient staff time to conduct evaluation activities (e.g., identifying or developing a survey, collecting information from participants).

7. Identified indicators that appropriately measure the impact of our work.

3. Barriers

To what extent have you experienced the following barriers to conducting a program evaluation?

1. Not enough time

2. Not enough money

3. Not knowing how to conduct a program evaluation

4. Not having assistance with data collection

5. Not getting enough people to respond to surveys

6. Not knowing how to analyze data

7. Not having people to turn to for consultation and assistance

8. Not knowing what questions to ask

9. Not knowing how to write up results

10. Other factors (please specify):

4. To what extent would your organization be interested in training opportunities on the following topics (1 – Strongly Disagree, 2 – Disagree, 3 – Neither Agree nor Disagree, 4 – Agree, 5 – Strongly Agree):

1. Designing evaluations

2. Developing evaluation questions

3. Conducting process evaluations

4. Conducting outcome evaluations

5. Data collection methods

6. Sampling methods

7. Data analysis methods

8. Using evaluation data for program improvement

9. Building program and evaluation partnerships

5. About the Respondent

1. [Demographics] What race/ethnic group do you identify with? (check all that apply)

• Asian/Asian-American

• Black/African-American

• Hispanic/Latino/Latina

• White

• Native American/American Indian/Alaskan Native/Native Hawaiian/Pacific Islander

• Multi-racial or multi-ethnic (2+ races/ethnicities)

• Prefer not to respond

2. [Demographics] What is your gender?

   • Male

   • Female

   • Non-conforming

   • Prefer not to respond

   • Prefer to self-describe (please specify): ______

3. How long have you worked with your organization?

   • Less than 1 year

   • 2-5 years

   • 5-10 years

   • 10-20 years

   • 20+ years

4. What is your position in the organization?

   • Leader (CEO, Executive Director)

   • Other Executive

   • Manager/Supervisor

   • Staff

   • Board member

   • Other (please specify)

5. How long has your organization worked with CNCS?

   • Less than 1 year

   • 2-5 years

   • 5-10 years

   • 10-20 years

   • 20+ years

EVALUATION CAPACITY BUILDING POST-SESSION SURVEY

1. Overall, how satisfied are you with this session/workshop?

Very Dissatisfied, Dissatisfied, Unsure, Satisfied, Very Satisfied

2. How much would you agree or disagree with the following statements?

(Strongly Disagree, Disagree, Neither Agree nor Disagree, Agree, Strongly Agree)

1. The information I learned in this session/workshop was useful for me.

2. The instructor(s) clearly explained the material covered in this session/workshop.

3. I felt comfortable voicing any questions I had in the session/workshop.

4. The activities were helpful to my understanding of the material.

5. I understand how to apply this material to my own program.

1. After taking this module, how do you rate your understanding of the evaluation topics discussed in the session/workshop?

(Very Poor, Poor, Acceptable, Good, Very Good)

4. Do you have any suggestions for how to improve the learning experience?

5. What did you enjoy about this learning experience? What was your favorite part?

6. What resources would be the most helpful in supporting your understanding of the topics covered in this presentation? 

• Examples from AmeriCorps Grantees/Sponsors with similar interventions

• Frequently Asked Questions and Answers

• Grantee/Sponsor Discussion Guides

• Templates, Rubrics, Checklists

• How-To Videos

• Other (please specify):__________

EVALUATION CAPACITY BUILDING OBSERVATION FORM

Observer name: ______________________________

Date of observation: ___________________________

Presentation title: _____________________________

Number of participants: ________________________

1. Overall, how satisfied are you that this session/workshop met the learning goals/objectives?

   1. Very Dissatisfied, Dissatisfied, Unsure, Satisfied, Very Satisfied

2. How much would you agree or disagree with the following statements?

(Strongly Disagree, Disagree, Neither Agree nor Disagree, Agree, Strongly Agree; notes for each item)

1. The instructor(s) clearly explained the material covered in this module.

2. The activities were helpful to participants’ understanding of the material.

3. The answers the instructor(s) gave to participants’ questions were clear.

4. The instructor(s) provided illustrative examples.

5. The exercises in the session/workshop were well-facilitated.

6. The participants were engaged in the session/workshop.

7. The participants learned or improved upon the objectives set in the session/workshop.

3. How does the course help develop content knowledge, skills, or thinking practices related to these learning objectives? _______________________________________________________________________

4. What are the strengths of this learning experience? _______________________________________________________________________

5. Do you have any suggestions for how to improve this learning experience? _______________________________________________________________________

55