9 AC Member Process

Process Evaluation of the Minnesota Reading Corps Site Visit Protocol: AmeriCorps Member

Process Evaluation of the Minnesota Reading Corps

AmeriCorps Member

School Name:

Site Visitor Name:

Date:

Location:

Interviewee Name(s) and Title(s):

Introduction

Good morning/afternoon. My name is [INSERT NAME] and I am with [NORC at the University of Chicago/TIES]. We are working on behalf of the Corporation for National and Community Service to conduct a process evaluation of the Minnesota Reading Corps program to provide a thorough understanding of the operations and activities of the MRC as it is implemented in each PreK site or elementary school. It will seek to determine if the actual activities and service are true to the intended model of the program, and to assess whether the actual process is likely to produce the intended outcomes and reach the intended target population.

This interview will address these topics and others as we seek to understand your project’s processes and strategies and any aspects of your program that can be replicated in other AmeriCorps programs. The interview should take approximately 45 minutes. Your open and honest opinions are appreciated, but participation is voluntary and you may choose to skip any questions or end the interview at any time. Please also be advised that we will be providing an evaluation report for each individual project to CNCS. While we will not use your name or any others in this report, and we will attempt to minimize the use of identifiable information, it may be possible for CNCS to identify you through your position or through other details that you share in your interview.

If you have questions about the study after this interview, please contact the Project Director, Carrie Markovitz, at 301-634-9388. If you have questions about your rights as a participant in this research project, please call the NORC Institutional Review Board Administrator at 866-309-0542.

Do you consent to participate in this discussion?

[IF YES, then proceed. IF NO, then terminate interview.]

I would like to record this interview in case my notes are not clear and comprehensive, and to make sure that we accurately report your responses. The recording will be shared only with the small team of researchers working on this study and will be deleted at the end of the project. Do you agree to have this interview recorded for note-taking purposes only?

[IF YES, then proceed. IF NO, then: “That’s fine. Please be patient as I take notes.”]

BACKGROUND

First, we would like to discuss your general experience serving as an AmeriCorps Member at [name of site/school].

1. Is this your first year serving as an AmeriCorps Member with MRC? YES NO

1. [IF NO] How many years have you served as an MRC tutor? _________ years

1. Have you always served at the same school? YES NO

2. Have you always served in the same position? YES NO

3. At which other schools have you served?

2. Are you full-time (40 hours) or part-time (20 hours) member? FT PT

2. Why did you decide to apply to be an AmeriCorps Member for MRC?

4. Please describe the process of applying and interviewing for the position.

1. Once you applied, who did you interview with? (Principal, Internal Coach, Teacher, others)

2. Did you already have any background in this area (new or retired teacher, previous tutor, etc.)?

5. What responsibilities do you have as an AmeriCorps Member at [name of site/school]?

[PROBE: Provide tutoring only; Recruit and /or train volunteers to support school-based literacy efforts (e.g., read with students during after-school time); Develop literacy resources for families; Coordinate literacy events for families; Other: _____________________________)]

5. Besides providing Reading Corps services during the school day, have you participated in other service activities outside of the school day to fulfill the hours requirement needed to successfully complete the program. [PROBE: volunteering at the school or in the community to fulfill service hours requirement]

1. At the beginning of the school year did you have a plan for fulfilling the hours requirement needed to successfully complete your service, which may include hours outside of the school day? YES NO

2. [IF YES] Did you share or discuss your plan with your Internal Coach or Program Coordinator?

TRAINING AND ORIENTATION

Please tell us about your training and orientation to [name of site/school].

5. Please describe the type of training you received to become an MRC tutor for [PreK;K-3] once you were accepted to the program.

5. Did you receive any additional training once placed in your current site/school? YES NO

1. [IF YES] What kind of training did you receive? Was it from MRC, the school/site, or another source?

9. Is there additional training you wish you would have had before you started as a tutor?

9. How did the other school staff (director/principal, teachers, counselors) react to your addition to the site/school when you first began serving?

MRC PROGRAM IMPLEMENTATION

We would like to learn more about how the interventions and tutoring actually occurs at [name of site/school] with [Pre-K; K-3] students.

9. Ask PreK only:

   1. What types of interventions are provided to Tier 2 and Tier 3 students? Briefly describe.

   2. Is tutoring provided one-on-one, in small groups? to the whole class? [DELIVERY METHOD]

   3. Do you feel that you contribute to making the environment more literacy-rich? YES NO

      1. [IF YES] In what ways?

      2. [IF NO] Why not?

9. Ask K-3 only: When you started the school year were only students nearest to target performance receiving services? YES NO Don’t Know

1. [If NO] Who else was served?

2. As the school year progressed did other students performing below target receive services? Please describe. [PROBE: English Language Learners, Students with an Individualized Education Plan ( IEP), Tier 3 students]

9. We would like to know when the interventions/tutoring sessions are conducted. [FREQUENCY]

   1. PreK How many days per week do students attend? [PROBE: 5-days, 4-days, 3-days.]

      1. Are literacy interventions provided each day?

2. K-3 Are tutoring sessions provided each day? [PROBE: Daily, twice weekly, weekly, bi-weekly; day(s) of week; time of day]

9. What is the duration of the:

   1. PreK Tier 2 or Tier 3 intervention? ________________

   2. K-3 tutoring session? ________________

9. Does each of your targeted student receive the same number of:

   1. PreK Tier 2 or Tier 3 interventions? YES NO

      1. [IF YES] Is this an effective strategy?

      2. [IF NO] How are these decisions made for each student and by whom?

2. K-3 tutoring sessions for the same number of weeks? YES NO

   1. [IF YES] Is this an effective strategy?

2. [IF NO] How are these decisions made for each student and by whom?

9. Where do Tier 2 or Tier 3 interventions [PreK]/tutoring sessions [K-3] take place?

Indicate place: ___________________________________________________

1. [IF THE INTERVENTION IS PROVIDED TO THE WHOLE CLASS] Does the teacher stay in the classroom while the tutoring session is conducted?

2. [IF THE STUDENTS ARE REMOVED FROM THE CLASSROOM]

   1. Where exactly does the tutoring take place?

   2. Is this place conducive to [one-on-one tutoring or small group sessions]?

17. There are a number of literacy interventions that are used in the MRC model. Can you tell us how frequently you use the following interventions? Please refer to the list. [Give Respondent the PreK or K-3 list] [INTERVENTIONS]

17a.	PreK Literacy Rich Daily Schedule	Never	Seldom	About half the time	Usually	Always	If used Usually, Always, or About half the time, why do you use this?	If used Never or Seldom why don’t you use this?
	Arrival
	Sign-in
	Meal time
	Daily Message *
	Repeated Read Aloud * Required for Professional Corps Teacher
	Tier 1 Small Group
	Journal (weekly)
	Choice Time (active learning)
	Tier 2 or Tier 3 (interventions daily)

17b.	MRC K-3 Interventions	Never	Seldom	About half the time	Usually	Always	If used Always, Usually, or About half the time, why do you use this?	If used Never or Seldom why don’t you use this?
	Letter/Sound Correspondence
	Phoneme Blending
	Phoneme Segmenting
	Blending Words
	Repeated Reading with Comprehension
	Strategy Practice
	Repeated Reading Chart
	Newscaster Reading
	Duet Reading
	Pencil Tap
	Stop / Go
	Great Leaps

17. Is any technology used during the tutoring session?

17. Who coordinates:

1. PreK the time when students can receive targeted interventions?

2. K-3 tutoring scheduling?

17. What kinds of assessments are done for each student? [PROBE: Progress Monitoring; Benchmarking]

1. What is your role in monitoring student progress?

2. Where do you enter the data? [PROBE: OnCorps; AIMS web]

21. For students that require supplemental reading or literacy assistance:

1. Does the Internal Coach seek your input when making decisions about meeting the needs of individual children? YES NO

2. Do teachers or school counselors ask for your input in assessing the students you work with? YES NO

3. In your opinion, is your input valued?

22. Does the Internal Coach conduct observations of you conducting interventions and /or assessments? YES NO

1. How often does this occur?

2. Are there any other ways that your tutoring assistance is monitored or assessed?

22. How often do you meet with the Internal Coach for coaching sessions? ________________ [COACHING]

1. What kinds of topics are addressed? [PROBE: Engagement, delivery of content, challenges, discipline]

2. Have you received additional training?

   1. [IF YES] In what areas?

24. Were the materials or processes provided by MRC adapted in any way at [name of site/school]? YES NO

[IF YES] Please describe. [PROBE: Who was involved? Was the adaptation approved by the Master Coach? Was MRC involved? Did MRC approve the adaptation?]

24. What types of supports or coaching do other school staff provide to you? [PROBE: Advice, sharing professional development materials, etc.]

24. Are you aware of any other efforts to tutor similar students at your school? YES NO

1. [IF YES] Are you familiar with their tutoring program? Do you think the other program is similar to the MRC program? How or how not?[PROBE: similar in tutoring materials, training, instruction, assessments?]

24. Have you ever shared your tutoring materials or information you learned in your MRC training with other school staff or other non-MRC reading tutors? YES NO

1. [IF YES] Do you know if they use the information you shared with them when working with students on reading?

24. Who determines whether a student no longer requires tutoring? How is this determination made?

24. FOR K-3 only Is it common for you to tutor students who graduated from the program, but then began to struggle with their reading and require more assistance? YES NO

1. [IF YES] Do you use a different approach with repeat students than with new ones? Why or why not?

FACILITATORS AND BARRIERS TO PROGRAM IMPLEMENTATION

We would like to talk with you about what has worked well in implementing the program and where there may have been some challenges.

24. What has worked well in implementing the MRC model in this site/school?

24. What problems or challenges were encountered during implementation? How were they resolved?

24. From your perspective, do any current policies and procedures make it hard to implement the program as intended [in this site/school? In this class?] Please describe. [PROBE: School district, MRC, AmeriCorps State/National]

24. Is there sufficient organizational support for implementing the program? [PROBE: Minnesota Reading Corps? Region or school district? Site/school? Other?]

1. Could any changes be made with to better support program implementation?

   1. Minnesota Reading Corps

2. Region or school district

3. Site/school

4. Other

24. Is there anything you would change about the MRC program? YES NO

[PROBE: With the model? As implemented at this site/school? Need to adapt to the population? Being an AmeriCorps Member?]

1. [IF YES] What?

RESULTS AND LESSONS LEARNED

It is important for any evaluation that we understand your view of the relative effectiveness of the program and how you have benefited from serving with AmeriCorps. So we would like to ask you some questions about your opinions on the success of the MRC program at [name of site/school] and your experience.

24. In your opinion, is the program successful in helping students improve their literacy or reading skills? How have you determined this?

24. In your opinion, what is the most rewarding part of being an AmeriCorps Member in the MRC program?

24. In your opinion, what is the most challenging part of being an AmeriCorps Member in the MRC program?

24. What do you plan to do after you finish serving as an AmeriCorps Member for MRC?

24. Do you plan to serve in MRC next year? YES NO

24. Has participating in the MRC affected your plans for the future? If so, in what ways?[PROBE: Volunteering, teaching, etc.]

24. Do you have any advice about participating in the MRC program that might be helpful to other AmeriCorps Members? Or ServeMN?

Thank you for your time. We appreciate it very much.

9