District State Date
Kindergarten Entry Assessment Case Studies
District Version Interview Protocol
Introduction and Purpose of Interview
Thank you for your willingness to participate in this interview. We are talking with state and district administrators, principals, kindergarten teachers, preschool administrators, and professional development staff in your state about the Kindergarten Entry Assessment, including the development and implementation of the assessment and how the data are used. We’d like to learn what is working well in addition to the challenges and how people are overcoming them. We believe information about your experiences will be valuable for other states and districts interested in implementing KEAs. Your experience also can help inform federal policy and help the U.S. Department of Education build technical assistance that is useful for all states tackling this important effort. A public report will be available at the end of the study to promote sharing of lessons learned but it will not identify any individual districts, schools, or people.
This interview will take approximately 60 minutes. We might ask questions that do not apply to your state, district, or schools. If so, please just let us know that this is the case.
Consent Process
Have each respondent read and sign the consent form. Ask permission to audio-record the conversation.
______________________________________________________________________________
We understand that your state has [selected or modified XX measure or developed its own measure [local KEA name] as your kindergarten entry assessment.
What has been your role in terms of the selection and implementation of [local KEA name]?
How has your district tried to create buy-in and participation in [local KEA name]?
Do all the schools with kindergarten children in your district participate in the [local KEA name]? If not, is there a pattern or reason for which schools are participating?
I am now going to ask you some questions about the [local KEA name] itself and the nuts and bolts of administering it.
When is [local KEA name] administered (i.e., during what time frame of the school year)? How frequently is it administered? Is repeated use throughout the year an option?
Are students who begin the year late assessed (i.e., after the assessment is completed for most students)? Do you have data on how many students begin school this late in the school year?
Describe how [local KEA name] is implemented in your schools.
Probe:
What components and or measures make up your kindergarten entry assessment?
What types of evidence are collected as part of the assessment? For example, are knowledge, skills, and abilities assessed through observation, checklists, direct assessments, and/or parent reports? [Probe for each component of the KEA.]
Who administers or provides information about the student for [local KEA name]?
How are families involved in providing information?
To what degree is technology used to support the administration or scoring of the [measure]?
What did the state communicate to you, as a district administrator, about the purpose of the [local KEA name] and what it is trying to assess?
What did the district communicate to parents, teachers, and administrators about the purpose of [local KEA name] and what it is trying to assess?
What training, supports, or resources do district-level staff receive from the state on the administration of [local KEA name]? Do you receive the same training as the principals and teachers?
What training, supports, or resources do teachers and school administrators receive on the administration of [local KEA name]? Please describe the various types of training. Probe: What training, supports, or resources do they receive for using the data from [local KEA name]?
So, would you say your training and support model uses [read the following list]? (Mark all that apply.)
Workshops
Webinars
Coaching
Online modules (e.g., PPTs on websites)
Resource documents such as administration manuals
Other (specify): ________________________________________
Who provides the training, supports, or resources (the state, the district, a contractor)? If using a train-the-trainer model, who are the local trainers and who provides their training?
Do individuals need to be certified before administering the assessment?
What are the qualifications of the trainers?
Are trainers local (in-state or in-district) staff or outside contractors?
Does the state or vendor train the trainers?
What other training, supports, or resources would have been helpful, if any?
What, if any, plans are there to continue training after the initial introduction (e.g., booster training, coaching, PLC)?
When your district administers the [local KEA name], is it a standard process across all participating districts in the state? Or can your district adapt administration or implementation procedures? If so, what can vary and who can choose?
Similarly, within your district, do all schools need to administer the [measure] in a standard way? Or can schools adapt administration or implementation procedures? If so, what can vary and who can choose?
Probe: For example, can a school choose which measure(s) or components of measures (e.g., a particular domain) to administer? Can a school sample children (versus universally assessing all children in the district/school)? Can it change administration procedures (e.g., opt in or out of parent reports, home observations)?
If schools use different measures or processes to collect data, does the district collect information on the measures or processes that schools choose to use? Does the district compare results across schools using different measures or data collection processes to inform implementation?
✪Is there a document that summarizes the implementation guidelines that you can share with me?
How does the district monitor and evaluate whether the [local KEA name] is being implemented according to the state's recommended guidelines?
What roles do districts and schools play in the monitoring process?
What issues have been uncovered? How have they been addressed?
What does the district do if there are issues with implementation?
Are there ways implementation guidance from the state could be improved? If yes, how?
Can you describe how your district submits the resulting data from [local KEA name] to the state?
Probe:
Who is involved in submitting the data?
How are the initial results entered or recorded?
In what form are data submitted from the school to the district? How about from the district to the state?
How does the data eventually get to the state?
Where are the data entered (e.g., test publisher’s website, statewide longitudinal data system, student permanent record)/cumulative file)?
What is the data submission timeline?
What is the approximate amount of time spent by district staff on supporting implementation of [local KEA name]?
What other district costs are there to implement [local KEA name] (e.g., materials, substitutes to release teachers)? Please don’t include costs directly paid by the state (e.g., if the state pays for a trainer directly).
Finding one assessment that can accurately identify young children’s development status across multiple learning domains is challenging. Ensuring that the measure is also linguistically appropriate for children whose primary language is not English adds another layer of complication, as does the need to accurately assess the strengths and needs of young children with developmental delays or disabilities.
Within this context, we’d like to ask about your thoughts concerning the use of [local KEA name] with these particular populations.
How does implementation of [local KEA name] differ for students who are English leaners (e.g., children are assessed in their home language, use of nonverbal stimuli, observations conducted by bilingual staff or adults)?
To what degree do you believe the [local KEA name] accurately assesses the knowledge, skills, and abilities of English learners? To what degree do you believe the [local KEA name] and those who administer [local KEA name] can distinguish between language differences and language delays?
In what ways are those who administer the kindergarten entry assessment prepared/trained to use [local KEA name] with English learners?
In what ways are expectations or interpretations of the findings adapted for English learners (who are learning their home language while also acquiring a second language such as in dual language programs)?
How does implementation of [local KEA name] differ for students with developmental delays or disabilities?
To what degree do you believe [local KEA name] accurately assesses the knowledge, skills, and abilities of children with developmental delays or disabilities?
In what ways are those who administer the kindergarten entry assessment prepared/trained to use [local KEA name] with children with developmental delays or disabilities?
What accommodations are made and for whom? (For example, are children given augmentative or alternative communication or written systems, visual or sensory support, assistive equipment or devices, input is collected from special education staff?)
Are there children with disabilities for whom [local KEA name] is not appropriate? If yes,
What alternate assessments are used with these children?
In what ways are the results used for screening and referral or IEP purposes?
In what ways are expectations or interpretations of the findings modified or adapted for children with developmental delays or disabilities?
What additional training, supports, or resources would have been helpful in using [local KEA name] with children with developmental delays or disabilities?
What have been the most important factors in supporting the implementation of the KEA?
What have been the biggest challenges with implementing the KEA?
(Use probes below.)
Funding, If so, how?
Administrator buy-in, If so, how?
Teacher buy-in or participation, If so, how?
The [local KEA name] instrument itself, If so, how?
Lack of training, If so, how?
Not enough or unclear administration directions, If so, how?
Political challenges, If so, how?
Timing challenges, If so, how?
Lack of shared vision among stakeholders about the purpose and use of the [local KEA name] or communication about its purpose and use with districts and schools, If so, how?
Technology difficulties (e.g., website for scoring and data entry not ready or functioning correctly), If so, how?
Other (specify): ________________________________________
What solutions have you identified for addressing those challenges?
The kindergarten entry assessment may be used in conjunction with existing district and school assessments.
If this is the case in your district, could you describe how [local KEA name] and the existing assessments work together (probe if there is a separate district K entry assessment)?
What are the main reasons for using the other kindergarten assessments in addition to [local KEA name]?
Now we’d like to switch gears and ask you a set of questions about how kindergarten entry assessment results are shared and used to inform policy and practice. Let’s begin with the results of the [local KEA name] and how people are informed of them.
How many children in your district were assessed using [local KEA name measure] in fall 2014? What percentage of your district’s kindergarten children does this represent? Probe: If you do not have this information now, could you email it to me?
What information do you receive on [local KEA name] results?
When and how often do you receive information?
In what format is information shared (e.g., access to database, printed or online reports, parent conferences)?
How is the information broken out (e.g., individual students or aggregated by student demographic characteristics, districts, schools, classrooms, or by preschool program)?
What does the information tell you for each of the groups?
What supporting information do you receive to interpret results and consider implications for learning supports at school and at home?
Who else receives the results of [local KEA name]? (Check all that apply) Probe for any groups not mentioned. Don’t give probe for respondent’s affiliation.
State Education Agency administration
State agencies that oversee preschool programs and services (e.g., DHHS and Head Start)
District administration
Principals
Kindergarten teachers
First through third grade teachers
Preschool educators
Parents (If so, are results translated into families’ languages?)
Policymakers/Legislators
Other (specify): ________________________________________
What information do others receive about the results? Let’s take it one at a time and I will ask you some detailed questions about what the different roles receive.
What is the schedule for sharing [local KEA name] results? In other words, when and how often does [each recipient checked above] receive information?
In what format is information shared (e.g., access to database, printed or online reports, parent conferences)?
How is the information broken out (e.g., individual students or aggregated by student demographic characteristics, districts, schools, classrooms, or by preschool program)?
What supporting information does [each recipient checked above] receive to interpret results and consider implications for learning supports at school and at home?
What challenges are there with sharing information with [each recipient checked above]?
✪ Is there a report with aggregate results from 2014 available that you can share with me?
✪ Do you also have sample reports that are tailored for district staff, principals, teachers, or parents you could share with me?
Now let’s move to how [local KEA name] results are used to inform policy and practice.
Please describe the training, supports, or resources you, as a district administrator, received to help you use [local KEA name] results? Who provided the training, supports, or resources?
Which training, supports, or resources are most useful?
What other training, supports, or resources would have been useful?
How do you, as a district administrator, use [local KEA name] results to inform decisions at the district level?
After getting a description, probe: Do you use results to make decisions to:
Identify students in need of extra services or further assessment? If so, how?
Identify specific skills where students need improvement to inform teaching of individual students or groups of students or curriculum planning? If so, how?
Engage families in knowing children’s strengths and needs and ways to support learning at home? If so, how?
Promote dialogue between early childhood and elementary educators about ways to increase children’s readiness (e.g., align expectations and/or instruction)? If so, how?
Help teachers understand where children should be on a comprehensive set of domains and how to assess children's progress? If so, how?
Benchmark learning to track progress in kindergarten throughout the year, If so, how?
Follow cohorts and subgroups of students over time to inform progress on closing achievement gaps? If so, how?
Make decisions about [at district, school, or program levels]:
Budget allocations or targeting of investments? If so, how?
Curriculum? If so, how?
Professional development for PreK or K-12 teachers? If so, how?
Early childhood policies and practices? (e.g., Identify changes needed across PreK programs throughout the district), If so, how?
K-12 policies and practices? (e.g., District-wide planning for Kindergarten) If so, how?
Other (specify): ________________________________________
After thinking about how you use the data, what information from the [local KEA name] do you think is most useful?
What other information would have been useful?
How do you expect principals to use [local KEA name] results to inform decisions at the school level?
For example, do you expect them to use results to [read list]? (Mark all that apply)
Identify students in need of extra services or further assessment? If so, how?
Identify specific skills where students need improvement to inform teaching of individual students or groups of students or curriculum planning? If so, how?
Engage families in knowing children’s strengths and needs and ways to support learning at home? If so, how?
Promote dialogue between early childhood and elementary educators about ways to increase children’s readiness (e.g., align expectations and/or instruction)? If so, how?
Help teachers understand where children should be on a comprehensive set of domains and how to assess children's progress? If so, how?
Benchmark learning to track progress in kindergarten throughout the year, If so, how?
Follow cohorts and subgroups of students over time to inform progress on closing achievement gaps? If so, how?
Make decisions about [school level]:
Budget allocations or targeting of investments? If so, how?
Curriculum? If so, how?
Professional development for PreK or K-12 teachers? If so, how?
Early childhood policies and practices? If so, how?
K-12 policies and practices? (e.g., School-wide planning for Kindergarten) If so, how?
Other (specify): ________________________________________
Please describe the training, supports, or resources principals receive to help them use [local KEA name] results.
How do you expect kindergarten teachers to use KEA [local KEA name] results?
For example, do you expect them to use results to [read list]? (Mark all that apply)
Identify students in need of extra services or further assessment? If so, how?
Identify specific skills where students need improvement to inform teaching of individual students or groups of students or curriculum planning? If so, how?
Engage families in knowing children’s strengths and needs and ways to support learning at home? If so, how?
Promote dialogue between early childhood and elementary educators about ways to increase children’s readiness (e.g., align expectations and/or instruction)? If so, how?
Help teachers understand where children should be on a comprehensive set of domains and how to assess children's progress? If so, how?
Benchmark learning to track progress in kindergarten throughout the year, If so, how?
Make decisions about:
Curriculum? If so, how?
K-12 policies and practices? (e.g., School-wide planning for Kindergarten) If so, how?
Other (specify): ________________________________________
Please describe the training, supports, or resources kindergarten teachers receive to help them use [local KEA name] results.
Use only if the preschool is part of the district. Do preschool directors and/or teachers receive the reports? If yes,
How do you expect preschool directors and teachers to use [local KEA name] results?
For example, do you expect them to [read list]? (Mark all that apply)
Identify specific skills where students need improvement to inform teaching of individual students or groups of students or curriculum planning? If so, how?
Engage families in knowing children’s strengths and needs and ways to support learning at home? If so, how?
Promote dialogue between early childhood and elementary educators about ways to increase children’s readiness (e.g., align expectations and/or instruction)? If so, how?
Help teachers understand where children should be on a comprehensive set of domains and how to assess children's progress? If so, how?
Make decisions about [district-wide or program level]:
Budget allocations or targeting of investments? If so, how?
Curriculum? If so, how?
Professional development for PreK or K-12 teachers? If so, how?
Early childhood policies and practices? If so, how?
K-12 policies and practices? (e.g., District-wide planning for Kindergarten) If so, how?
Other (specify): ________________________________________
Please describe the training, supports, or resources preschool directors and/or teachers receive to help them use [local KEA name] results, if any.
Describe any policies regarding how [local KEA name] results are not to be used, if there are any.
Listen for but don’t say the uses below, let the respondent say them. (Check all that apply):
Determine promotion or prevent kindergarten entry in any way
Principal evaluation, teacher evaluation
Evaluation of preschool programs
Screenings of children
Other (specify): ________________________________________
How are these policies communicated to districts, schools, and teachers?
How are these policies enforced? Please give me a specific example.
What have been the most important factors in supporting the use of [local KEA name] results to improve students’ educational experience?
What have been the biggest challenges with trying to use the [local KEA name] results?
(Use probes below.)
Lack of training, If so, how?
Reports hard to interpret, If so, how?
Lack of alignment with standards, If so, how?
Timing of when results are shared, If so, how?
Lack of perceived reliability, If so, how?
Lack of buy-in, If so how?
Other (specify): ________________________________________
What solutions have you identified for addressing those challenges?
This is our final set of questions! In closing we’d like you to reflect on how your state, district, and schools administered and used kindergarten entry assessment data and try to offer lessons learned.
Looking back over the [local KEA name] implementation, how well do you think it has gone? Has the [local KEA name] served the purposes it was intended to serve? [Probe: refer back to the general purpose they say has been communicated]
What concerns did you have going into implementation? Do these concerns still remain now that you have begun implementing the [local KEA name]? Do you have new concerns that now that you are implementing the [local KEA name]?
What parts of implementation have gone especially well?
How could implementation be improved?
Are there lessons learned or recommendations you can share with other districts about kindergarten entry assessment selection, implementation, and use of kindergarten entry assessment results to inform policy and practice?
Thank you for your time and thoughtful responses.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Tracy Huang |
| File Modified | 0000-00-00 |
| File Created | 2021-01-26 |