Compassion Capital Fund Impact Evaluation Process Study - Intermediary Protocol

Interview Protocols

Intermediary Respondents

CCF Impact Evaluation Process Study

Interview Protocols

Respondents: Intermediary Project Director & Key Staff

The following is to be used as a guide. Interviewers should attempt to cover the topics with all respondents to the extent that they are knowledgeable about the topics given their role and function within the organization. The discussion may vary somewhat from the specifics or details of the interview questions listed below based on the unique circumstances of the organization or the respondent’s position and role.

Interviewers shall inform respondents that the information they provide will be kept private and/or confidential, to the best of our ability using several procedural and control measures to protect the data from unauthorized use, and will be used by researchers only for the purpose of this study.

Interviewers shall inform respondents of the desire to audio-record the interview to enhance the accuracy and quality of the data collection. The interview shall not be recorded if the respondent objects.

Skip pattern: Specific questions that have been fully answered by one respondent should not be asked of subsequent respondents (e.g., how long organization has been in operation). However, multiple perspectives on more general topics are of interest and should be covered with each respondent, as appropriate given their role and function.

Respondent Background

Position

Years in Current Position

Prior experience and relevant qualifications

Intermediary Background

This information will supplement Organizational Features and Prior Experience questions on the intermediary survey that ask about type of organization; faith-based or secular; geographic area; annual operating budget; and prior experience providing TA, Training, sub-awards and receipt of federal funding.

1. What is your organization’s overall mission?

2. What is your organization’s main area of expertise? (What is it best known for?)

3. How does your work as an intermediary providing organizational capacity building assistance to faith- and community-based organizations fit with your overall mission?

4. How long has the organization provided training and assistance in the area of organizational capacity building? (Had the organization provided this type of assistance prior to the receipt of the CCF grant?)

5. Did the experience include providing capacity building assistance to the kinds of organizations you served through the CCF grant or to different kinds of organizations? (probe related to size, faith-based, programmatic service areas, etc)

6. In addition to training and services related to organizational capacity building, does your organization provide other types of services? Does it provide direct service to individuals/families? Please describe.

7. Please provide an overview of your organization’s overall structure and operations:

   • How many people (full- or part-time) work for the organization (total number)?

   • How many staff members work on the CCF project? How many consultants?

   • How is the organization operationally structured? Get a copy of project’s org chart if there is one.

8. Describe the structure and organization specifically related to the operation of your CCF Demonstration grant project.

   • How are staff who worked on the grant organized? [For example, staff who do one-on-one technical assistance vs. staff who do group training; or by expertise or type of capacity building – fundraising specialists vs. board development experts]

   • What are the roles and responsibilities of the staff members who work directly with FBCOs on the CCF grant project? What are their backgrounds?

   • Did you hire new/additional staff specifically to work on the CCF grant project?

   • Are there other organizations or individuals that play key roles? Describe role of partners and/or consultants (if applicable).

   • How do you make decisions about whether to use staff or consultants for specific functions/roles?

   • How are staff and consultants assigned? For example, is there a designated liaison to each FBCO receiving intensive services?

   • What are the expected types and extent of communication between your staff (or designated consultants) and the FBCOs you serve through the CCF grant? How is communication initiated and supported between staff and the FBCOs you serve?

   • Which staff do FBCOs work with and communicate with as they move through the program?

Overall Approach under CCF grant

1. How was the decision made about how many organizations you would award sub-awards and serve with intensive technical assistance? (Note: use this question as opportunity to follow-up on response to question on intermediary survey that asks whether their CCF program tries to serve as many organizations as possible or serve limited number with as much resources as possible.)

2. Do you use a formal organizational assessment tool or other formal process to determine training/technical assistance needs of FBCOs or is the training/technical assistance more typically guided by their requests? If tool used, is this a tool that you developed or one that is available to others? (obtain name of tool and copy if available in public domain)

3. Do your staff/consultants use a particular capacity building curriculum? If yes,

what is the name of the curriculum? If not, describe the approach you use for capacity building. What do you see as the benefits of your approach?

4. How did you decide on the curriculum or approach that you use? Is it fairly standard or is there variation across FBCOs?

Recruitment and Description of Organizations Served

1. Did you plan to target a certain type(s) of organization(s) for intensive services under the CCF grant (e.g., those of a certain size operating budget, or working in specific services fields such as at-risk youth)? Did those plans hold or did your approach change over time? On the spectrum of nonprofit organizations that runs from newly emerging, start-up organizations to mature experienced organizations – how would you describe your target organizations?

2. What recruitment methods were most successful for identifying eligible organizations to receive sub-awards and technical assistance under the CCF supported program? Did you modify methods over time? Did you meet your target number and type of organizations for both sub-awards and technical assistance? If not, what do you think was the primary reason for that (e.g., did something interfere with your outreach or organizations’ decisions to participate)? What would you do differently with regard to outreach and recruitment for sub-awards and intensive technical assistance?

3. Are there ways that the group of organizations that participated in your recent CCF supported program are different from your initial expectations? (Either the types of organizations, their organizational capacity needs, etc.)

Assessment

1. You said you (did/did not) use a formal assessment tool. Now I’d like to understand more about the details of how staff/consultants learn about the structure and function of specific organizations and their capacity building needs (Probe: do they conduct executive interviews, site visit, assessment tool)? When does this occur? How long does it take? (Get copy of documents/protocol.)

2. How are these findings/assessments used? (e.g., pre-post measures; to tailor services)

3. What is critical to learn about organizations’ baseline level of capacity?

4. In your experience, can organizations usually define their own needs well? In your view, to what extent does the fact finding or assessment process bring to light new areas of need that they may not have been aware of? (Probe specifically with regard to organizational needs related to: leadership development; organizational development; program development; and community engagement.)

5. When their thoughts about their needs and your assessment of their needs differ, how is this resolved?

6. Given how you identify organizations to work with through sub-awards and technical assistance, do you find their capacity building needs are fairly similar or not? What are some of the primary differences that you see? How do you tailor services to meet diverse needs?

Provision of Services

1. Is there a standard set of services from which staff/organizations pick based on their priorities or is most of the service provision customized – or a combination?

2. How are decisions made regarding the particular services to be provided/received and the source, timing and duration of services?

3. What approach(es) to technical assistance methods do your staff/consultants use? (Probe: one-on-one, small group, large group, mentoring) Are there particular drawbacks or benefits associated with employing specific techniques?

• Does each organization generally get the same amount of TA and training?

4. Describe (in detail) the services you provide under the CCF grant:

• Sub-award funding

• One-on-one TA

• Group training/training workshops

For each describe the process, purpose, topics covered, number of hours over what period, etc. For sub-awards, describe the announcement, review and selection processes, as well as the number of awards and amounts. Any restrictions on what FBCOs could use sub-awards for?

(Follow up on any issues identified in the intermediary survey about #s served, # of hours of services received, and amount of sub-awards.)

5. Which areas of organizational capacity are most commonly addressed through one-on-one TA (e.g. leadership development, organizational development, program development, revenue development, community engagement)? What areas are most commonly addressed through training? Through sub-awards?

6. Which area(s) of capacity are FBCOs most interested in addressing?

7. Overall, is there a primary area of capacity that you have found to be the most important or critical to work on in terms of improving grassroot organizations’ overall capacity?

8. In your experience, have you observed characteristics of organizations that receive capacity building assistance that are associated with an ability to take advantage of that assistance and institute change more readily than others?

Successes and Challenges

1. From among all of the types of support and assistance you provide, what do you consider your strongest area? What kinds of needs are you best equipped to address?

2. Which services would you say lead to the most positive change within the faith-based and community organizations you assist? Please describe any special practices or ways in which you deliver services that you feel contributes to your and their success.

3. Are there needs within the community organizations that you are more challenged to meet? What do you do in those circumstances?

4. As you consider the objectives of the CCF grant, which ones do you believe you have met most satisfactorily?

5. How does your organization assess whether you are helping to build the capacity of the organizations you serve? (Probe: interviews, pre-post assessments) What outcomes do you measure? For instance, do you track organizations and their success in fundraising? Do you know/track if they access other funding sources (e.g., federal, state/local, foundation funding)?

6. What do you define as the key indicators of success for your organization as an intermediary under the CCF program?

7. What challenges have you encountered in operating under the CCF grant program? What factors and/or conditions hinder your work? How have you tried to address these?

8. Provide an example of one of your greatest successes under the CCF grant. (Provide an example where your organization played an important role, not just a case where an FBCO has done well.)

5