NAEP Wave 2 (TEL and Assessment Feedback)

NATIONAL ASSESSMENT OF
EDUCATIONAL PROGRESS
Wave 2 Submittal for 2014
VOLUME II
Part 3
Assessment Feedback Questions

Part 3 contains:
School Coordinator Debriefing Interview Questions
Post Assessment Follow-Up Survey Questions
Teacher/School Survey Feedback Question

The amount of time estimated to complete these forms:
School Coordinator Debriefing Interview Questions: 7 minutes
Post Assessment Follow Up Survey Questions: 10 minutes
Teacher/School Survey Feedback Question (No additional burden-part of Teacher/School
Questionnaires)

Page 1 of 18

School Coordinator Debriefing Interview Questions

Page 2 of 18

School Coordinator Debriefing Interview
Directions: Complete with the school coordinator before leaving the school on assessment day.

1.

Overall, how do you think the assessment(s) went in your school?
Would you say:
Read aloud and check the appropriate box.

2.



Very well



Satisfactorily



Unsatisfactorily

How well did the instruction brochure you received with the Preassessment
Packet prepare and direct you in performing the school coordinator tasks?
Would you say:
Read aloud and check the appropriate box.

3.



Very well



Satisfactorily



Unsatisfactorily

NAEP is interested in the feedback you have received from other school staff
members about this assessment. Would you say it has been:
Read aloud and check the appropriate box.



Positive



Neutral



Negative



No feedback



Record comments:
_____________________________________________________________________
_____________________________________________________________________

Page 3 of 18

4.

5.

Did anyone meet with the students to explain the purpose of the assessment?



Yes



No



Don’t know

Did the students receive any special instructions to prepare them for the
assessment?



Yes



No



Don’t know



If yes, specify type of instructions received:

_____________________________________________________________________
_____________________________________________________________________

6.

Will the students receive (or have they already received) something from the
school for participating in the assessment?



Yes



No



Don’t know



If yes, specify what (e.g., food, candy, money, party):

_____________________________________________________________________
_____________________________________________________________________

7.

Do you have any feedback about the material you received from NAEP
(publications, letters, etc.)?



Yes



No



Don’t know



Record comments:

_____________________________________________________________________
_____________________________________________________________________

Page 4 of 18

8.

High School Students Strategies Form
In order to obtain more information on high school student participation and motivation,
we would like to know which strategies were used in the school. Please answer the
following questions:
Prepare and Schedule NAEP
1.

Which of the following methods did the school use to prepare for NAEP?











Add NAEP to school calendar
Use the NAEP MySchool website to track assessment activities for NAEP
Post information about NAEP on school website
Use the NAEP Appointment Card
Send notification letter to students selected for NAEP before assessment day
NAEP announced prior to assessment day to parents, students, and/or faculty
NAEP announced on the day of the assessment
Give teachers a list of students with testing time and location
Provide NAEP staff with a master list of where students are at the beginning of
the assessment
 Flexible scheduling for students
 Other (please specify) ______________________________________________
 Did not do anything extra to prepare for NAEP
Work With Teachers
2.

Which of the following methods did the school use to inform teachers about the
importance of NAEP?

 Announce NAEP at a school faculty meeting
 Hold a meeting with teachers for the sole purpose of discussing NAEP
assessment

 Resources used at any time:
 PowerPoint presentation – Introducing NAEP to Teachers
 Video – Introducing NAEP to Teachers
 Talking points to answer teacher questions about NAEP
 Other NAEP materials (i.e., Measure Up and Teacher’s Guide)
 Other (please specify) ______________________________________________
 Did not do anything extra to inform teachers about the importance of NAEP

Page 5 of 18

8.

High School Students Strategies Form (continued)

(cont)
Motivate Students
3. Which of the following methods did the school use to motivate the selected
students and be sure they attended?
 Met with students to explain the importance of NAEP. If a meeting was held
with students, what resources were used at the meeting?
 PowerPoint presentation – Introducing NAEP to Students
 Video – Introducing NAEP to Students
 Talking points to answer student questions about NAEP
 References to the NCES website to find additional resources and past
results
 Publish an article about NAEP in school newspaper
 Senior class advisors, teachers, counselors, and/or administration urge the
students to participate
 Invite staff to attend assessment sessions
 Importance of NAEP stressed at beginning of assessment session by school
staff
 Thank students for participation in NAEP (please specify):
 Informal – (e.g., verbal thank-you from school coordinator, principal,
counselor)
 Formal – (e.g., letter, certificate, community service hours, recognition at
assembly)
 Other (please specify) ______________________________________________
 Did not do anything extra to motivate the students
Provide Incentives
4.

Which of the following incentives did the school give to the students who
participated?

 NAEP Certificate of Community Service
 Recognition at a school/awards assembly or other announcement
 Food incentive before or after the assessment (for example pizza, donuts,
cookies, etc.)

 Lottery for items (yearbook, prom tickets, entertainment, restaurant coupon,
parking spot, hall pass, etc.)

 All students given item for participating (lunch line pass, key chain, early release,
restaurant coupon, etc.)

 Other (please specify) ______________________________________________
 Did not do anything extra to provide incentives for the students

Page 6 of 18

9.

Discuss any issues documented on the Session Debriefing Form that should be
brought to the school coordinator’s attention.

10.

I have placed copies of all documents used in the assessment in the NAEP
Storage Envelope. NAEP would like you to retain the envelope until June 1 or
the end of the school year in case there should be questions about the
assessment. A postage-paid postcard is stapled to the outside as a reminder
for you to destroy the contents after the “Destroy By” date. Please drop the
card in the mail when the package has been destroyed.

11.

Thank you for everything you have done to make this a successful
assessment. I have a Certificate of Appreciation I would like to give the school
as a token of our appreciation for your participation.
Give the school coordinator the School Certificate of Appreciation.
PRIVATE SCHOOLS THAT HAVE BEEN PROMISED ATLASES. I brought NAEP
atlases your school was offered this past summer.


12.

Distribute atlases.

Offer the school coordinator any used mathematics ancillary foam shapes, rulers,
protractors, and spinner squares.

Page 7 of 18

School Post Assessment Follow Up Survey

Page 8 of 18

INSTRUCTIONS

After the interview, assign a summary rating on the front cover that best reflects the school coordinator’s responses, as
well as any unsolicited comments he/she makes during your conversation that are relevant to the team’s performance.
The rating should factor in the school coordinator’s overall evaluation of the team, as well as any favorable or unfavorable
reports about individual team members. In assigning your rating, be sure to differentiate, when possible, a negative
attitude about the NAEP program versus a negative evaluation of the assessment team’s performance.
The rating scale for the overall evaluation consists of a five-category scale, with values ranging from “1,” signifying
excellent, to “5,” meaning the performance was unacceptable. A “Not Enough Information” (“8”) option is provided, but
unless the school coordinator consistently answers your questions with “don’t know,” assign a rating based on the
opinions the school coordinator was able to express. Ratings of “4” and “5” should be extremely rare and, when assigned,
you will need to provide more details about the team’s performance. Promptly discuss with your field manager if a team
receives an overall rating of “4” or “5.”

The rating scale is explained in more detail below.
Excellent

(1)

The team was totally prepared, organized, and professional. The school
coordinator had nothing but a glowing review for each question asked of them.
Almost all questions were answered either “very well” or “very good.”

Good

(2)

The team was prepared and professional, but there were a few small missteps
that kept this from being an excellent assessment.

/

(3)

The team was adequate and got the job done, but the school coordinator
answered most of the questions as “satisfactorily” or “adequate” (as opposed to
very well/very good). Additional briefing or training may be in order.

Unsatisfactory

(4)

The team was unprepared and/or unprofessional at times.

Satisfactory
Adequate

The team’s performance definitely requires improvement. Additional briefing or
training may be in order.
Unacceptable
/ Problem

(5)

The team was totally unprepared and/or unprofessional. This performance was
unacceptable, and immediate remedial action is definitely required.

Not Enough
Information to
Rate

(8)

Due to the session scheduling, the school coordinator was unable to observe any
portion of the team’s performance or enough of the performance to rate the team.

Please consult the NAEP Assessment Team Quality Control Form Telephone Version 2013 MAIN/TEL Question-byQuestion Specifications for additional guidelines on what you should consider when assigning these ratings.

Page 9 of 18

1

THIS PAGE INTENTIONALLY LEFT BLANK

Page 10 of 18

2

INTRODUCTION TO THE SCHOOL COORDINATOR

Hello, this is ______ from the National Assessment of Educational Progress. I work with the NAEP
Supervisor/Assessment Coordinator for this area. A few days ago, our assessment team conducted (a/#
_______) session(s) at your school. NAEP would like to thank you for all of your work in making the
assessment at _________________________ (SCHOOL NAME) possible. As part of our efforts to
improve the performance of our assessment teams, we routinely call a random sample of school
coordinators to find out how the assessment and preassessment visit went. This call should only take a
few minutes.
Just as a reminder, the assessment team at your school was led by _________________________
(SV/AC NAME). When responding to these questions, please feel free to speak frankly. Your comments
will help improve the quality of the assessments.

QUESTIONS ABOUT THE PREASSESSMENT VISIT

First, I have a few questions about the preassessment visit that ______________ (SV/AC NAME)
conducted with you on _______________ (PREASSESSMENT DATE).

1.

Overall, how would you rate the preassessment visit at your school?
Would you say the visit went very well, satisfactorily, or unsatisfactorily?
VERY WELL ...................................
SATISFACTORILY .........................
UNSATISFACTORILY ....................
1a.

1 (Q2)
2 (Q2)
3 (Q1a)

IF UNSATISFACTORILY: Why do you feel the preassessment visit went unsatisfactorily?
(PROBE FOR SPECIFICS)
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________

2.

Overall, how would you rate the process of updating the demographic information on the Student
Information Reports?
Would you say the process worked very well, satisfactorily, or unsatisfactorily?
VERY WELL ...................................
SATISFACTORILY .........................
UNSATISFACTORILY ....................
2a.

1 (Q3)
2 (Q3)
3 (Q2a)

IF UNSATISFACTORILY: Why do you feel the process of updating the demographic
information worked unsatisfactorily? (PROBE FOR SPECIFICS)
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
Page 11 of 18

3

QUESTIONS ABOUT THE PREASSESSMENT VISIT (CONTINUED)

3.

Overall, how would you rate the process of determining accommodations for students with disabilities
and English language learners? Would you say the process of determining accommodations worked
very well, satisfactorily, or unsatisfactorily?

VERY WELL ...................................
SATISFACTORILY .........................
UNSATISFACTORILY ....................
3a.

1 (Q4)
2 (Q4)
3 (Q3a)

IF UNSATISFACTORILY: Why do you feel the process worked unsatisfactorily? (PROBE FOR
SPECIFICS)
__________________________________________________________________________
__________________________________________________________________________
__________________________________________________________________________
__________________________________________________________________________________________

4.

Do you have any other comments or suggestions about the preassessment visit?
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
QUESTIONS ABOUT THE ASSESSMENT

Now I have a few questions about the assessment that was conducted at your school
On ___________________________ (ASSESSMENT DATE).
5.

Overall, how would you say the NAEP team handled the assessment at your school?
Would you say very well, satisfactorily, or unsatisfactorily?
VERY WELL ...................................
SATISFACTORILY .........................
UNSATISFACTORILY ....................
5a.

1 (Q6)
2 (Q6)
3 (Q5a)

IF UNSATISFACTORILY: Why do you feel the assessment team handled things
unsatisfactorily?
(PROBE FOR SPECIFICS)
__________________________________________________________________________
__________________________________________________________________________
__________________________________________________________________________
__________________________________________________________________________

Page 12 of 18

4

QUESTIONS ABOUT THE ASSESSMENT (CONTINUED)

6.

Did the team arrive on time?
YES ................................................
NO ..................................................
DON’T KNOW..................................
6a.

1 (Q7)
2 (Q6a)
8 (Q7)

IF NO, RECORD DETAILS ABOUT THE LATE ARRIVAL.
(PROBE: Could you tell me more about that or why they were late?)
__________________________________________________________________________
__________________________________________________________________________

7.

As far as you know, did (all of) the session(s) start on time?
YES ................................................
NO (SOME OR ALL WERE DELAYED)
DON’T KNOW..................................
7a.

1 (Q8)
2 (Q7a)
8 (Q8)

IF NO, RECORD ANY DETAILS OFFERED ABOUT THE DELAY.
(PROBE: Could you tell me more about why the session(s) was/were delayed?)
__________________________________________________________________________
__________________________________________________________________________
__________________________________________________________________________

7b.

DELAY WAS CAUSED BY (CODE ONE):
TEAM ..............................................
OTHER REASON ............................
COULD NOT DETERMINE ..............

8.

1
2
8

Did you have the opportunity to observe (any of) the assessment session(s)?
YES ................................................
NO ..................................................
8a.

IF YES: How would you rate the way the assessment administrator(s) handled the session(s)
you observed? Would you say very well, adequately, or poorly?
VERY WELL ....................................
ADEQUATELY.................................
POORLY .........................................
DON’T KNOW..................................

8b.

1 (Q8a)
2 (Q9)

1
2
3
8

(Q9)
(Q8b)
(Q8b)
(Q9)

IF ADEQUATELY OR POORLY, RECORD ANY DETAILS MENTIONED.
(PROBE: Could you tell me more about that or how the assessment administrators handled the
sessions?)
__________________________________________________________________________
__________________________________________________________________________
__________________________________________________________________________
Page 13 of 18

5

QUESTIONS ABOUT THE ASSESSMENT (CONTINUED)

9.

How would you rate the overall organization of the assessment team?
Would you say it was very good, adequate, or poor?
VERY GOOD ..................................
ADEQUATE ....................................
POOR .............................................
DON’T KNOW..................................
9a.

1
2
3
8

(Q10)
(Q9a)
(Q9a)
(Q10)

IF ADEQUATE OR POOR, RECORD ANY DETAILS MENTIONED.
(PROBE: Could you tell me more about the team’s organization?)
__________________________________________________________________________
__________________________________________________________________________
__________________________________________________________________________

10.

How would you rate the overall professionalism of the assessment team?
Would you say it was very good, adequate, or poor?
VERY GOOD ..................................
ADEQUATE ....................................
POOR .............................................
DON’T KNOW..................................
10a.

1
2
3
8

(Q11)
(Q10a)
(Q10a)
(Q11)

IF ADEQUATE OR POOR, RECORD ANY DETAILS MENTIONED.
(PROBE: Could you tell me more about the team’s professionalism?)
__________________________________________________________________________
__________________________________________________________________________
__________________________________________________________________________

11.

How would you rate the way the assessment team interacted with the other school staff?
Would you say the interaction was very good, adequate, or poor?
VERY GOOD ..................................
ADEQUATE ....................................
POOR .............................................
DON’T KNOW..................................
11a.

1
2
3
8

(Q12)
(Q11a)
(Q11a)
(Q12)

IF ADEQUATE OR POOR, RECORD ANY DETAILS MENTIONED.
(PROBE: Could you tell me more about the team’s interaction with other school staff?)
__________________________________________________________________________
__________________________________________________________________________
__________________________________________________________________________________________

Page 14 of 18

6

QUESTIONS ABOUT THE ASSESSMENT (CONTINUED)

12.

How would you rate the way the assessment team interacted with the students?
Would you say the interaction was very good, adequate, or poor?
VERY GOOD ..................................
ADEQUATE ....................................
POOR .............................................
DON’T KNOW..................................
12a.

1
2
3
8

(Q13)
(Q12a)
(Q12a)
(Q13)

IF ADEQUATE OR POOR, RECORD ANY DETAILS MENTIONED.
(PROBE: Could you tell me more about the team’s interaction with students?)
__________________________________________________________________________
__________________________________________________________________________
__________________________________________________________________________

13.

At the end of the session(s), did ________________ (SV/AC name) give you the red NAEP Storage
Envelope and tell you when the materials should be destroyed? (REVIEW IF NECESSARY).
YES ................................................ 1
NO .................................................. 2
DON’T KNOW ………………………. 8

14.

Is there anything (else) you would like to say about the assessment team or any of its members?
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
SUGGESTIONS

15.

NAEP is conducted regularly and we continue to refine the materials and procedures for future
assessments. Do you have any suggestions on how we can make it easier for schools to participate in
the future?
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________

These are all of the questions I have. On behalf of the national assessment, I want to thank you again for
all of your help.

Page 15 of 18

7

FEEDBACK GIVEN TO SV/AC/AA AND THEIR RESPONSE *
School ID __________________

SV/AC/AA Name:

School Name ____________________________________ Assessment Date _______________

Feedback/Response:

________________________________________________________________________________________________________
________________________________________________________________________________________________________
________________________________________________________________________________________________________
________________________________________________________________________________________________________
________________________________________________________________________________________________________
________________________________________________________________________________________________________
________________________________________________________________________________________________________
________________________________________________________________________________________________________
________________________________________________________________________________________________________
________________________________________________________________________________________________________
________________________________________________________________________________________________________
________________________________________________________________________________________________________
________________________________________________________________________________________________________

* PLACE A COPY OF THIS SECTION IN THE SV’S/AC’S PERSONNEL FOLDER AND EACH AA’S PERSONNEL FOLDER, IF
APPROPRIATE. ALSO, INCLUDE A COPY WHEN THE SV, AC OR AA RECEIVED AN EXCEPTIONALLY POSTIVE REVIEW.

Page 16 of 18

8

Teacher/School Online Evaluation Question

(Part of Teacher and School Questionnaires)

Page 17 of 18

Evaluation Question Regarding Online Completion   

If given the choice, would you have preferred to respond to these survey questions on paper or 
online? 

A) On Paper 
B) Online 
C) No preference 

Page 18 of 18