VISTA Assessment of Project Sustainability

DRAFT 7-30-08

START TIME ________________

2008 vista PROGRAM Evaluation

Telephone interview with staff member from the sponsoring organization

VISTA Project:

Sponsoring Organization:

Contact from sampling form:

Hello, may I speak with (NAME OF RESPONDENT)

RESPONDENT ON PHONE: My name is ______________________________ calling on behalf of the Corporation for National and Community Service, which is the federal agency that runs the VISTA program. I am calling regarding the (NAME) VISTA project (BRIEF DESCRIPTION OF PROJECT) which according to our records began in (DATE) and may or may not be active today. We are conducting a study of VISTA projects nation-wide called the 2008 VISTA Program Evaluation and (VISTA PROJECT) was selected to be part of this study.

We have been told that you are the most knowledgeable person in your organization regarding (VISTA PROJECT). Will you be able to answer questions about this VISTA project?

IF NOT THE MOST KNOWLEDGEABLE PERSON: Who would be the most knowledgeable person in your organization we could talk to?

Name:

E-mail:

Telephone number:

In general, what is the best time to reach her/him? Best times: ___________ a.m. ___________ p.m.

IF RESPONDENT IS MOST KNOWLEDGEABLE:

I recently sent you an e-mail letter regarding this study together with a profile of (YOUR VISTA) project and a summary of the questions we will be asking. Did you receive the e-mail, and the project profile? (IF NOT, VERIFY ADDRESS AND RESEND).

Your organization has been chosen through a random selection method for this telephone interview. While participation in the survey is voluntary, your responses are critical to provide a complete picture of the VISTA program nationwide. You may terminate this interview at any time. Reports of this survey will not contain your name or the name of your organization. Any confidential information you provide will not be disclosed without your permission to anyone other than the researchers conducting this study except as otherwise required by law. The interview will take about 30 minutes.

First I would like to ask you a few background questions about you and your involvement with the (VISTA PROJECT).

Interviewee Contact information

Is the contact information for you on the profile correct?

YES 1

NO 2

IF NO, please tell me what to correct.

Interviewee Name:

Name of Organization: _____________________________________________________________________

Address:

City: ST: Zip:

Phone:

E-mail:

What is your position within (ORGANIZATION)?

1. Were you working with (NAME OF ORGANIZATION):

1a. When the VISTA project was planned?

YES 1

NO 2

1b. When the VISTA project was implemented?

YES 1

NO 2

1c. When the last VISTA left the project?

YES 1

NO 2

NOTE: IF NO TO Q1 (a,b, and c), SKIP Q2 and GO to Q3

2. Which of the following activities did you participate in with regard to (VISTA PROJECT)?
(CIRCLE ALL THAT APPLY)

YES NO

a. Development of a concept paper (previously known as Application Part A) 1 2

b. Development of the VISTA application 1 2

c. Development of project goals 1 2

d. Establishing ways to measure achievement of goals 1 2

e. Maintaining contact with the VISTA state office staff 1 2

f. Recruiting VISTA members 1 2

g. Training or orienting VISTA members 1 2

h. Supervising VISTA members 1 2

i. Conducting project activities 1 2

j. Preparing and/or submitting project reports 1 2

k. Evaluating project work or progress 1 2

12. Is there any other activity you participated in that I

haven’t mentioned? ………………………………………… 1 2

IF YES (SPECIFY)

We reviewed some of (VISTA PROJECT)’s project documents submitted to VISTA such as the project application and progress reports in order to learn about the project and your organization. The profile you received along with the email I sent to you is a summary of the information we collected from these documents. We would like to verify some of the information from these reports, and then fill in the gaps.

3. Have you had time to review the project profile to determine whether the information presented in the profile is correct?

YES 1 (GO TO Q4)

NO 2

IF NO, Please take a few minutes to review the project profile before we continue with the interview.

4. Is the information listed in the project profile accurate? (in section 1,2,3…of the project profile)

YES 1 (GO TO Q5)

NO 2

4a. IF NO, What changes should be made?

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

NOTE: CHECK THE PROJECT PROFILE TO DECIDE WHICH QUESTION TO ASK NEXT. THERE ARE TWO CATEGORIES OF PROJECTS:

A. PROJECTS WITH ONLY THE ORIGINAL GOALS LISTED IN THE PROJECT PROFILE AND NO INFORMATION ON WHETHER THERE WAS A MAJOR CHANGE IN THE GOALS. [GO TO Q8]

B. PROJECTS WITH THE ORIGINAL GOALS LISTED AND MAJOR CHANGES IN THE GOALS (i.e. AT LEAST TWO SETS OF GOALS) [GO TO Q5]

GOALS

The project profile lists each project goal we identified from the application and project reports. Now I would like to use this list to talk about the major goals of (VISTA PROJECT).

GO OVER THE LIST OF GOALS

5. Information from project reports indicates that there was a change in the original project goals. Is that correct? (GO OVER THE REVISED GOALS – THE LAST SET OF GOALS IN THE PROFILE)

YES 1 (GO TO Q6)

NO 2 (GO TO Q8)

6. What were the primary reasons for the change in project goals?

7. How was the Corporation State Office (CSO) involved in the process of changing goals?

IF 3 OR FEWER GOALS ARE LISTED IN THE PROFILE GO TO Q9

ASK ONLY IF MORE THAN 3 GOALS ARE LISTED IN THE PROFILE:

8. Identify up to three most important goals Of the VISTA Project.

Goal 1: ________________________________________________________________

Goal 2: ___________________________________________________ _____________

Goal 3: ________________________________________________________________

9. Looking at your most important/(three) goal(s)), in general, how central were these goals to your organization’s mission? (For each goal) would you say the goal was not central to your organization’s mission, somewhat central, or very central?

GOAL	Not central	Somewhat Central	Very Central	Not applicable
Goal 1	1	2	3	4
Goal 2	1	2	3	4
Goal 3	1	2	3	4

10. (For each goal) would you say the goal was not achieved, partly achieved or completely achieved?

GOAL	Not achieved	Partly achieved	Completely achieved
Goal 1	1	2	3
Goal 2	1	2	3
Goal 3	1	2	3

11. Were VISTA(s) assigned to the project not helpful, somewhat helpful, helpful or critical in achieving project goals?

GOAL	Not helpful	Somewhat helpful	Helpful	Critical
Goal 1	1	2	3	4
Goal 2	1	2	3	4
Goal 3	1	2	3	4

12. How many VISTA positions were you awarded for this project?
______________________
Number of VISTA positions


12a. Overall, how difficult was it to fill these VISTA positions? Was it very difficult, somewhat difficult
or not at all difficult?

VERY DIFFICULT 1

SOMEWHAT DIFFICULT 2

NOT AT ALL DIFFICULT 3

IF GOALS WERE COMPLETELY OR PARTLY ACHIEVED ASK:

13. Think of the goals that your project completely or partly achieved, how much influence did the following factors have on your project’s ability to achieve these goals? Would you say they had no influence, some influence, or a lot of influence?

NONE SOME A LOT

a. Good program design 1 2 3

b. Individual efforts of a VISTA 1 2 3

c. Support from your organization 1 2 3

d. Community support 1 2 3

e. Monetary or in-kind support 1 2 3

f. Involvement of volunteers other than VISTAs 1 2 3

g. Political support 1 2 3

h. Collaboration with other agencies 1 2 3

i. Training of the VISTAs by the Corporation…………………... 1 2 3
j. Training of the VISTAs by your organization 1 2 3

k. Are there any other factors that influenced your project’s ability to achieve its goals?

YES 1 2 3

If YES (SPECIFY) ______________________________________________________________

IF GOALS WERE NOT ACHIEVED ASK:

14. Now think of the goals that your project did not achieve. What were the most important factors that hindered the achievement of each of these goals?

14a Goal 1

14b Goal 2

14c Goal 3

ACTIVITIES

Next I am going to ask about project activities.

15. Did any of the project activities evolve over time to be significantly different from the activities initially planned as part of the VISTA application?

YES 1 (GO TO Q15a)

NO 2 (GO TO Q16)

15a. Which activities were changed and how?

15b. What were the reasons for the changes in activities?

16. What were the main challenges (VISTA PROJECT) faced in implementing the project’s activities? Did challenges include . . .

YES NO

a. Not enough community support 1 2

b. Too few volunteers in the community 1 2

c. Not enough agency-wide support 1 2

d. Lack of or decrease in political support 1 2

e. Insufficient monetary/in-kind support 1 2

f. Not enough collaboration with external agencies 1 2

g. Change in community needs… 1 2

h. Inability to recruit qualified candidates to serve as VISTAS 1 2

i. Poorly trained or unprepared VISTAs 1 2

j. Were there any other important challenges that (VISTA PROJECT) faced in implementing its project activities? 1 2

IF YES, (SPECIFY)

17. Were these challenges mostly overcome?

MOSTLY YES 1 (GO TO Q17a)

MOSTLY NO 2 (GO TO Q17b)

DK 8 (GO TO Q19)

17a. IF MOSTLY YES, how were they overcome?

17b. IF MOSTLY NO, What do you think would have enabled the project to overcome these challenges?

18. How helpful was the Corporation State Office in addressing the challenges or supporting the activities?
    Were they:

Not very helpful 1

Helpful 2

Very helpful 3

19. How much did the community support this project, where 1 is “Almost no support” and 5 is “Very extensive support”?

1

2

3

4

5

Almost no Very extensive

support support

20. Please describe the community support for this project.

PROJECT CONTINUATION/ SUSTAINABILITY

The next few questions ask what happened to (VISTA PROJECT) after the last VISTA left.

21. Has the VISTA project continued to exist in some form after the last VISTA left your organization

YES 1 (GO TO Q22)

NO 2 (GO TO Q23)

22. IF YES, In what ways have the project activities continued? (PROVIDE EXAMPLES)

23. Why wasn’t the VISTA project or its activities continued? Was it because:

YES NO

a. The project met all of its goals and objectives 1 2

b. Lack of resources 1 2

c. Lack of community support 1 2

d. Something else 1 2

(SPECIFY) __________________________________________________________

SUPPORT FOR PROJECT, TRAINING, AND TECHNICAL

We next ask about the technical assistance and training provided by VISTA Headquarters and the Corporation State Office to your VISTA project.

24. Did the (VISTA PROJECT) receive any of the following types of training or technical assistance from the Corporation?

VISTA Sources	YES	NO	DON’T KNOW	IF YES, How frequently did you use this source?	IF YES, how helpful was this source?
a. Technical assistance provided by the Corporation’s program officers	1	2	8	About once every 6 months or less 1 1 Once a month 2 More than once a month 3	Not at all 1 Somewhat 2 Very 3
b. Assistance delivered by the Corporation’s national training providers	1	2	8	About once every 6 months or less 1 1 Once a month 2 More than once a month 3	Not at all 1 Somewhat 2 Very 3
c. Conferences, meetings, or trainings hosted by the Corporation	1	2	8	About once every 6 months or less 1 1 Once a month 2 More than once a month…………3	Not at all 1 Somewhat 2 Very 3
d. Online materials such as toolkits, manuals, tip sheets, or sample forms	1	2	8	About once every 6 months or less 1 1 Once a month 2 More than once a month ………..3	Not at all 1 Somewhat 2 Very 3
e. e-courses or webinars	1	2	8	About once every 6 months or less 1 1 Once a month 2 More than once a month…………3	Not at all 1 Somewhat 2 Very 3

25. Did your organization ever request training or technical assistance that it did not receive?

YES 1 (GO TO Q25a)

NO 2 (GO TO Q26)

DK 8 (GO TO Q26)

25a. If yes, please explain.

SPONSORING ORGANIZATION BACKGROUND INFORMATION

Now we would like to learn a little more about your organization

26. For about how many years has your organization been in existence?

________________
YEARS

27. Did your entire organization sponsor the VISTA project, or did only a unit of the organization sponsor it?

ENTIRE ORGANIZATION 1

UNIT 2

SPECIFY NAME OF UNIT

OTHER 3

(SPECIFY)

FOR Q28-Q37 REFER TO RESPONSE IN Q27:

28. About how many employees does your (ORGANIZATION/UNIT) have today?

__________

NUMBER OF EMPLOYEES

29. About how many employees from your [ORGANIZATION/UNIT] were actively involved with the (VISTA PROJECT)?

__________

NUMBER OF EMPLOYEES

30. How many VISTA supervisor positions were assigned to the VISTA project?

___________

NUMBER OF VISTA SUPERVISOR POSITIONS

31. Did the VISTA supervisor receive training from the Corporation?

YES 1

NO 2

32. For each VISTA supervisor position, approximately what percentage of time was dedicated to supervising VISTAs?

Supervisor	0-25%	26-50%	51-75%	76-100%
1	1	2	3	4
2	1	2	3	4
3	1	2	3	4

33. Given the constraints on the Supervisors' time, were the supervisors able to meet the needs of the VISTA(s) for supervision all of the time, most of the time, or some of the time?

SOME OF THE TIME 1

MOST OF THE TIME 2

ALL OF THE TIME ……………………….. 3

34. Was the VISTA Project Director also a VISTA supervisor?

YES 1

NO 2

34. Before this project, how many other VISTA projects did your (ORGANIZATION/UNIT) sponsor?

No other projects before this one 1

One other 2

Two or more ………………………………. 3

DK 8

34. Briefly describe your organization’s mission.

34. In working toward its mission, doesyour organization primarily:

Provide direct services 1

Manage other organizations 2

Both provide direct services and manage other organizations 3

Other (SPECIFY) 4

34. How much experience would you say your (ORGANIZATION/UNIT) had with the key activities of (VISTA PROJECT) prior to the start of the VISTA project, where 1 is “No Experience” and 5 is “Very extensive experience”?

1

2

3

4

5

No experience Very extensive

experience

34. How would you rate the support given to the (VISTA PROJECT) by your (ORGANIZATION/UNIT), where 1 is “No support” and 5 is “Very extensive support”?

1

2

3

4

5

No support Very extensive

support

40. Would your organization have taken on this project without the VISTA(s)?

YES 1

NO 2

DK 8

41. Overall, how much supervision did the VISTAs need, where 1 is “No supervision” and 5 is”Very extensive supervision”?

1

2

3

4

5

No supervision Very extensive

supervision

42. Do you have any other comments about the VISTAs who served your project?

________________________________________________________________________________________________________________________________________­­­­­_____________________________________________________

END

Thank you for answering these questions.

END TIME _____________________________

Call Record for telephone interview with sponsoring organization staff

Result codes: Complete (CO) No answer (NA)

Left message (LM) Refusal (R)

Call back (CB) Problem (PB)

New contact (NC)

Date/Time	Contact	Result

Best Person to speak to for VISTA interview:

Phone:

Best time to call:

5