2016 Main National Assessment of Educational Progress (NAEP) Administration

NATIONAL ASSESSMENT OF
EDUCATIONAL PROGRESS
Submittal for 2016
VOLUME II
Part 4
Assessment Feedback Questions

Part 4 contains:
School Coordinator Debriefing Interview Questions
Post Assessment Follow-Up Survey Questions

The amount of time estimated to complete these forms:
School Coordinator Debriefing Interview Questions: 7 minutes
Post Assessment Follow-Up Survey Questions: 10 minutes

Part 4 - Page 1 of 16

School Coordinator Debriefing Interview
Directions: Complete with the school coordinator before leaving the school on assessment
day.
1. Overall, how do you think the assessment(s) went in your school?
Would you say:
Read aloud and check the appropriate box.


Very well



Satisfactorily



Unsatisfactorily

2. How well did the instruction brochure you received with the
Preassessment Packet prepare and direct you in performing the school
coordinator tasks?
Would you say:
Read aloud and check the appropriate box.


Very well



Satisfactorily



Unsatisfactorily

3. NAEP is interested in the feedback you have received from other school
staff members about this assessment. Would you say it has been:
Read aloud and check the appropriate box.


Positive



Neutral



Negative



No feedback

 Record comments:
_______________________________________________________________
_______________________________________________________________

Part 4 - Page 2 of 16

4. Did anyone meet with the students to explain the purpose of the
assessment?


Yes



No



Don’t know

5. Did the students receive any special instructions to prepare them for the
assessment?


Yes



No



Don’t know

 If yes, specify type of instructions received:
_______________________________________________________________
_______________________________________________________________
6. Will the students receive (or have they already received) something from
the school for participating in the assessment?


Yes



No



Don’t know

 If yes, specify what (e.g., food, candy, money, party):
_______________________________________________________________
_______________________________________________________________

7. Do you have any feedback about the material you received from NAEP
(publications, letters, etc.)?


Yes



No



Don’t know

 Record comments:
_______________________________________________________________
_______________________________________________________________

Part 4 - Page 3 of 16

8.

High School Students Strategies Form
In order to obtain more information on high school student participation and motivation,
we would like to know which strategies were used in the school. Please answer the
following questions:
Prepare and Schedule NAEP
1.

Which of the following methods did the school use to prepare for NAEP?















Add NAEP to school calendar
Use the NAEP MyNAEP website to track assessment activities for NAEP
Post information about NAEP on school website
Use the NAEP Appointment Card
Send notification letter to students selected for NAEP before assessment day
NAEP announced prior to assessment day to parents, students, and/or faculty
 Use social media (Facebook/Twitter) to announce NAEP
 Send notification letter to students selected for NAEP before
assessment day
NAEP announced on the day of the assessment
Give teachers a list of students with testing time and location
Provide NAEP staff with a master list of where students are at the beginning of
the assessment
Flexible scheduling for students
Other (please specify) ______________________________________________
Did not do anything extra to prepare for NAEP

Encourage Teachers to Support NAEP
2.

Which of the following methods did the school use to inform teachers about the
importance of NAEP?

 Explain the importance of NAEP to teachers in a meeting, memo, announcement,

etc. If the importance of NAEP was explained to teachers, what resources, if any,
were used from the Best Practice Guide:
 PowerPoint presentation – Introducing NAEP to Teachers
 Video – Introducing NAEP to Teachers
 Talking points to answer teacher questions about NAEP
 Social media (Facebook/Twitter) posts for teachers
 Other NAEP materials (i.e., Measure Up Newsletter and Facts for Teachers)
 Other (please specify) ______________________________________________
 Did not do anything extra to inform teachers about the importance of NAEP

Part 4 - Page 4 of 16

8.
(cont)

High School Students Strategies Form (continued)
Motivate Students to Participate
3. Which of the following methods did the school use to motivate the selected
students and ensure student attendance for the NAEP assessment?
 Met with students to explain the importance of NAEP. If a meeting was held
with students, what resources were used from the Best Practices Guide?
 PowerPoint presentation – Introducing NAEP to Students
 Video – Introducing NAEP to Students
 Talking points to answer student questions about NAEP
 References to the NCES website to find additional resources and past
results
 Use social media (Facebook or Twitter) to share information about NAEP with
students
 Publish an article about NAEP in school newspaper or newsletter
 Senior class advisors, teachers, counselors, and/or administration urge the
students to participate
 Invite staff to attend assessment sessions
 Importance of NAEP stressed at beginning of assessment session by school
staff
 Recognize or thank students for participation in NAEP (please specify):
 Informal – (e.g., verbal thank-you from school coordinator, principal,
counselor)
 Formal – (e.g., letter, certificate, community service hours, recognition at
assembly, honor tags to wear at graduation)
 Other (please specify) ______________________________________________
 Did not do anything extra to motivate the students
Provide Incentives
4.

Which of the following incentives did the school give to the students who
participated?

 NAEP Certificate of Community Service
 Recognition at a school/awards assembly or other announcement
 Food incentive before or after the assessment (for example pizza, donuts,
cookies, etc.)

 Lottery for items (yearbook, prom tickets, entertainment, restaurant coupon,
parking spot, hall pass, etc.)

 All students given item for participating (lunch line pass, key chain, early release,
restaurant coupon, etc.)
 Other (please specify) ______________________________________________
 Did not do anything extra to provide incentives for the students

Part 4 - Page 5 of 16

9.

Discuss any issues documented on the Session Debriefing Form that should
be brought to the school coordinator’s attention.

10.

I have placed copies of all documents used in the assessment in the
NAEP Storage Envelope. NAEP would like you to retain the envelope
until June 1 or the end of the school year in case there should be
questions about the assessment. Please log on to MyNAEP and record
the date you destroy the envelope.

11.

Thank you for everything you have done to make this a successful
assessment. I have a Certificate of Appreciation I would like to give the
school as a token of our appreciation for your participation.
Give the school coordinator the School Certificate of Appreciation.
PRIVATE SCHOOLS THAT HAVE BEEN PROMISED ATLASES. I brought
NAEP atlases your school was offered this past summer.


Distribute atlases.

Part 4 - Page 6 of 16

DEBRIEFING NOTES
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
__________________________________________________________________________________
Part 4 - Page 7 of 16

School Post Assessment Follow Up Survey

Part 4 - Page 8 of 16

INSTRUCTIONS

After the interview, assign a summary rating on the front cover that best reflects the school
coordinator’s responses, as well as any unsolicited comments he/she makes during your
conversation that are relevant to the team’s performance. The rating should factor in the school
coordinator’s overall evaluation of the team, as well as any favorable or unfavorable reports
about individual team members. In assigning your rating, be sure to differentiate, when possible,
a negative attitude about the NAEP program versus a negative evaluation of the assessment
team’s performance.
The rating scale for the overall evaluation consists of a five-category scale, with values ranging
from “1,” signifying excellent, to “5,” meaning the performance was unacceptable. A “Not
Enough Information” (“8”) option is provided, but unless the school coordinator consistently
answers your questions with “don’t know,” assign a rating based on the opinions the school
coordinator was able to express. Ratings of “4” and “5” should be extremely rare and, when
assigned, you will need to provide more details about the team’s performance. Promptly discuss
with your field manager if a team receives an overall rating of “4” or “5.”
The rating scale is explained in more detail below.
Excellent

(1)

The team was totally prepared, organized, and professional.
The school coordinator had nothing but a glowing review for
each question asked of them. Almost all questions were
answered either “very well” or “very good.”

Good

(2)

The team was prepared and professional, but there were a
few small missteps that kept this from being an excellent
assessment.

Satisfactory /
Adequate

(3)

The team was adequate and got the job done, but the school
coordinator answered most of the questions as “satisfactorily”
or “adequate” (as opposed to very well/very good). Additional
briefing or training may be in order.

Unsatisfactory

(4)

The team was unprepared and/or unprofessional at times.
The team’s performance definitely requires improvement.
Additional briefing or training may be in order.

Unacceptable
/ Problem

(5)

The team was totally unprepared and/or unprofessional. This
performance was unacceptable, and immediate remedial
action is definitely required.

Not Enough
Information to
Rate

(8)

Due to the session scheduling, the school coordinator was
unable to observe any portion of the team's performance or enough
of the team's performance to rate the team.

Please consult the NAEP Assessment Team Quality Control Form Telephone Version 2015
MAIN/TBA Question-by-Question Specifications for additional guidelines on what you should consider
when assigning these ratings.

Part 4 - Page 9 of 16

THIS PAGE INTENTIONALLY LEFT BLANK

Part 4 - Page 10 of 16

INTRODUCTION TO THE SCHOOL COORDINATOR

INTRODUCTION TO THE SCHOOL COORDINATOR

Hello, this is ______ from the National Assessment of Educational Progress. I work with the
NAEP Supervisor/Assessment Coordinator for this area. A few days ago, our assessment team
conducted (a/# _______) session(s) at your school. NAEP would like to thank you for all of your
work in making the assessment at _________________________ (SCHOOL NAME) possible.
As part of our efforts to improve the performance of our assessment teams, we routinely call a
random sample of school coordinators to find out how the assessment and preassessment visit
went. This call should only take a few minutes.
Just as a reminder, the assessment team at your school was led by _____________________
_________________________ (SV/AC NAME). When responding to these questions, please
feel free to speak frankly. Your comments will help improve the quality of the assessments.

QUESTIONS ABOUT THE PREASSESSMENT REVIEW CALL
First, I have a few questions about the preassessment review call that ______________ (SV/AC
NAME) conducted with you on _______________ (PREASSESSMENT REVIEW CALL DATE).

1. Overall, how would you rate the NAEP representative’s handling of the preassessment
review call?
Would you say:
Read aloud and check the appropriate box.
Very good
Satisfactory
Unsatisfactory
Decline to answer
2. Overall, how would you rate the NAEP representative on leading the review and update
of student demographic information during your call?
Would you say:
Read aloud and check the appropriate box.
Very good
Satisfactory
Unsatisfactory
Decline to answer

Part 4 - Page 11 of 16

QUESTIONS ABOUT THE PREASSESSMENT REVIEW CALL (CONTINUED)
3. Overall, how would you rate the NAEP representative on leading the review of
accommodations for students with disabilities and English language learners?
Would you say:
Read aloud and check the appropriate box.
Very good
Satisfactory
Unsatisfactory
Decline to answer
4. Do you have any other comments or suggestions about the NAEP representative's
review of the preassessment activities you conducted on MyNAEP?
Yes
No
Don’t know


Record comments:
_____________________________________________________________
_____________________________________________________________
_____________________________________________________________
QUESTIONS ABOUT THE ASSESSMENT

Now I have a few questions about the assessment that was conducted at your school
on ___________________________ (ASSESSMENT DATE).

5. Overall, how would you say the NAEP team handled the assessment at your school?
Would you say:
Read aloud and check the appropriate box.
Very well
Satisfactorily
Unsatisfactorily
Decline to answer

Part 4 - Page 12 of 16

QUESTIONS ABOUT THE ASSESSMENT (CONTINUED0
6. Did the team arrive on time?
Would you say:
Read aloud and check the appropriate box.
Yes
No
Don’t know
Decline to answer
7. As far as you know, did the session(s) start on time?
Would you say:
Read aloud and check the appropriate box.
Yes
No
Don’t know
Decline to answer
8. Did you have the opportunity to observe any of the assessment session(s)?
Would you say:
Read aloud and check the appropriate box.
Yes
No
Don’t know
Decline to answer
9. How would you rate how the NAEP representatives handled the session(s) you
observed?
Would you say:
Read aloud and check the appropriate box.
Very well
Adequately
Don’t know
Decline to answer

Part 4 - Page 13 of 16

QUESTIONS ABOUT THE ASSESSMENT (CONTINUED)
10. How would you rate the overall organization of the assessment team?
Would you say:
Read aloud and check the appropriate box.
Very good
Adequate
Poor
Don’t know
Decline to answer
11. How would you rate the overall professionalism of the assessment team?
Would you say:
Read aloud and check the appropriate box.
Very good
Adequate
Poor
Don’t know
Decline to answer
12. How would you rate how the assessment team interacted with other school staff?
Would you say:
Read aloud and check the appropriate box.
Very well
Adequate
Poor
Don’t know
Decline to answer

Part 4 - Page 14 of 16

QUESTIONS ABOUT THE ASSESSMENT (CONTINUED)

13. How would you rate how the assessment team interacted with the students?
Would you say:
Read aloud and check the appropriate box.
Very well
Adequate
Poor
Don’t know
Decline to answer
14. At the end of the session(s), did the NAEP representative give you the red NAEP storage
envelope and tell you when the materials should be destroyed?
Would you say:
Read aloud and check the appropriate box.
Yes
No
Don’t know
Decline to answer
15. Is there anything else you would like to say about the assessment team or any of its
members?
Yes
No
Don’t know


Record comments:
_______________________________________________________________
_______________________________________________________________
_______________________________________________________________

Part 4 - Page 15 of 16

SUGGESTIONS
16. NAEP is conducted regularly, and we continue to refine the materials and procedures
for future assessments. Do you have any suggestions on how we can make it easier for
schools to participate in the future?
Yes
No
Don’t know


Record comments:
______________________________________________________________
______________________________________________________________
______________________________________________________________

17. Think about all of the contacts you have had with NAEP staff and the activities that took
place during the preassessment visit and on assessment day. Please give the NAEP
assessment team an overall rating?
Read answers and record the SC’s answer:
Excellent
Good
Satisfactory
Unsatisfactory
Unacceptable
Not enough information to rate
18. Would you like to speak with a NAEP representative who oversees the team that
administered your assessment to discuss any of the activities related to NAEP that took
place in your school?
Yes, please have a NAEP representative contact me.
No, I don’t need to speak with a NAEP representative.

Part 4 - Page 16 of 16