Form Appendix E Appendix E Appendix E NAEP 2021 Feedback Forms

National Center for Education Statistics

National Assessment of Educational Progress

National Assessment of Education Progress (NAEP) 2021

Appendix E

NAEP 2021 Assessment Feedback Form

_{OMB# 1850-0928 v.18}

November 2019

No changes since v.10

Appendix E provides a sample from 2019. Appendix E will be updated to include fewer questions (hence the updated time estimate of two minutes rather than ten) and submitted to OMB in Amendment # 2 by June 2020.

NAEP 2019 Telephone Quality Control Form

Introduction TO the School Coordinator

Hello, this is ______ from the National Assessment of Educational Progress. I work with the NAEP assessment coordinator for this area. A few days ago, our assessment team conducted (a/# _______) session(s) at your school. NAEP would like to thank you for all of your work in making the assessment at _________________ (SCHOOL NAME) possible. As part of our efforts to improve the performance of our assessment teams, we routinely call a random sample of school coordinators to determine how well the assessment coordinator performed tasks related to the telephone review of the preassessment tasks you completed in MyNAEP. We are also interested in the performance of the NAEP team on assessment day. This call should only take 10 minutes.

Just as a reminder, the assessment team at your school was led by _________________________ (AC NAME). When responding to these questions, please feel free to speak frankly. Your comments will help improve the quality of the assessments.

Questions about the INITIAL CALL

First, I have a few questions about the initial call that ______________ (SV/AC NAME) conducted with you on _______________ (INITIAL CALL DATE). As a reminder, during that call you may have discussed MyNAEP registration, been introduced to brochures about using the site, walked through logging in, seen the Prepare for Assessment screens, and scheduled a preassessment review call.

1. Overall, how would you rate the NAEP representative’s handling of this initial call?

Would you say it was very good, satisfactory, or unsatisfactory?

VERY GOOD 1 (Q2)

SATISFACTORY 2 (Q2)

UNSATISFACTORY 3 (Q1a)

1a. IF UNSATISFACTORY: Why do you feel the NAEP representative’s handling of the initial call was unsatisfactory? (PROBE FOR SPECIFICS)

Questions about the Preassessment REVIEW CALL

Now I have a few questions about the preassessment review call that ______________ (SV/AC NAME) conducted with you on _______________ (PREASSESSMENT REVIEW CALL DATE). As a reminder, during this call you may have reviewed the list of students selected for NAEP, completed the SD or ELL student information, reviewed the procedures to notify parents, reviewed procedures for distributing and monitoring questionnaires, identified any newly enrolled students, planned assessment day logistics, and covered participation and other support activities.

2. Overall, how would you rate the NAEP representative’s handling of the preassessment review call?

Would you say it was very good, satisfactory, or unsatisfactory?

VERY GOOD 1 (Q3)

SATISFACTORY 2 (Q3)

UNSATISFACTORY 3 (Q2a)

2a. IF UNSATISFACTORY: Why do you feel the NAEP representative’s handling of the preassessment review call was unsatisfactory? (PROBE FOR SPECIFICS)

3. Overall, how would you rate the NAEP representative on leading the review and update of student demographic information on MyNAEP during your call?

Would you say the review process was very good, satisfactory, or unsatisfactory?

VERY GOOD 1 (Q4)

SATISFACTORY 2 (Q4)

UNSATISFACTORY 3 (Q3a)

3a. IF UNSATISFACTORY: Why do you feel the process of reviewing and updating the demographic information was unsatisfactory? (PROBE FOR SPECIFICS)

Questions about the Preassessment REVIEW CALL (CONTINUED)

4. Overall, how would you rate the NAEP representative on leading the review of accommodations for students with disabilities and English language learners? Would you say the process of reviewing accommodations was very good, satisfactory, or unsatisfactory?

VERY GOOD 1 (Q5)

SATISFACTORY 2 (Q5)

UNSATISFACTORY 3 (Q4a)

4a. IF UNSATISFACTORY: Why do you feel the review was unsatisfactory? (PROBE FOR SPECIFICS)

 

5. Do you have any other comments or suggestions about the NAEP representative’s review of the preassessment activities you conducted on MyNAEP?

Questions about the Assessment

Now I have a few questions about the assessment that was conducted at your school

on ___________________________ (ASSESSMENT DATE).

6. Did the team arrive on time?

YES 1 (Q7)

NO 2 (Q6a)

DON’T KNOW 8 (Q7)

6a. IF NO, RECORD DETAILS ABOUT THE LATE ARRIVAL.

(PROBE: Could you tell me more about that or why they were late?)

Questions about the Assessment (CONTINUED)

7. As far as you know, did (all of) the session(s) start on time?

YES 1 (Q8)

NO (SOME OR ALL WERE DELAYED) 2 (Q7a and Q7b)

DON’T KNOW 8 (Q8)

7a. IF NO, RECORD ANY DETAILS OFFERED ABOUT THE DELAY.

(PROBE: Could you tell me more about why the session(s) was/were delayed?)

7b. DELAY WAS CAUSED BY (CODE ONE):

TEAM 1

OTHER REASON 2

COULD NOT DETERMINE 8

8. Did you have the opportunity to observe (any of) the assessment session(s)?

YES 1 (Q8a)

NO 2 (Q9)

8a. IF YES: How would you rate the way the NAEP representative(s) handled the session(s) you observed? Would you say very well, adequately, or poorly?

VERY WELL 1 (Q9)

ADEQUATELY 2 (Q8b)

POORLY 3 (Q8b)

DON’T KNOW 8 (Q9)

8b. IF ADEQUATELY OR POORLY, RECORD ANY DETAILS MENTIONED.

(PROBE: Could you tell me more about that or how the NAEP representative handled the sessions?)

Questions about the Assessment (CONTINUED)

9. How would you rate the overall organization of the assessment team?

Would you say it was very good, adequate, or poor?

VERY GOOD 1 (Q10)

ADEQUATE 2 (Q9a)

POOR 3 (Q9a)

DON’T KNOW 8 (Q10)

9a. IF ADEQUATE OR POOR, RECORD ANY DETAILS MENTIONED.

(PROBE: Could you tell me more about the team’s organization?)

10. How would you rate the overall professionalism of the assessment team?

Would you say it was very good, adequate, or poor?

VERY GOOD 1 (Q11)

ADEQUATE 2 (Q10a)

POOR 3 (Q10a)

DON’T KNOW 8 (Q11)

10a. IF ADEQUATE OR POOR, RECORD ANY DETAILS MENTIONED.

(PROBE: Could you tell me more about the team’s professionalism?)

11. How would you rate the way the assessment team interacted with the other school staff?

Would you say the interaction was very good, adequate, or poor?

VERY GOOD 1 (Q12)

ADEQUATE 2 (Q11a)

POOR 3 (Q11a)

DON’T KNOW 8 (Q12)

11a. IF ADEQUATE OR POOR, RECORD ANY DETAILS MENTIONED.

(PROBE: Could you tell me more about the team’s interaction with other school staff?)

Questions about the Assessment (CONTINUED)

12. How would you rate the way the assessment team interacted with the students?

Would you say the interaction was very good, adequate, or poor?

VERY GOOD 1 (Q13)

ADEQUATE 2 (Q12a)

POOR 3 (Q12a)

DON’T KNOW 8 (Q13)

12a. IF ADEQUATE OR POOR, RECORD ANY DETAILS MENTIONED.

(PROBE: Could you tell me more about the team’s interaction with students?)

13. At the end of the session(s), did ________________ (AC NAME) give you the red NAEP Storage Envelope and tell you when the materials should be destroyed? (REVIEW IF NECESSARY).

YES 1

NO 2

DON’T KNOW…………………………………….. 8

14. Think about all of the contacts you have had with NAEP staff and the activities that took place during the preassessment review call/visit and on assessment day. Overall, how would you rate the NAEP team’s handling of the assessment at your school?

Would you say it was excellent, good, satisfactory, unsatisfactory, unacceptable, not enough information to rate, or don’t know?

EXCELLENT 1 (Q15)

GOOD 2 (Q15)

SATISFACTORY 3 (Q15)

UNSATISFACTORY 4 (Q14a)

UNACCEPTABLE 5 (Q14a)

NOT ENOUGH INFORMATION TO RATE 8 (Q15)

DON’T KNOW 8 (Q15)

14a. IF UNSATISFACTORY OR UNACCEPTABLE: Why do you feel the assessment team’s handling of the assessment was unsatisfactory?

(PROBE FOR SPECIFICS)

Questions about the Assessment (CONTINUED)

15. Is there anything (else) you would like to say about the assessment team or any of its members?

16. NAEP is conducted regularly, and we continue to refine the materials and procedures for future assessments. Do you have any suggestions as to how we can make it easier for schools to participate in the future?

These are all of the questions I have. On behalf of NAEP, I want to thank you again for all of your help.

20