NAEP 2011 School Reports Focus Group Studies

My name is Maria Ivancin and I work for the Market Research Bureau, a research company, working on behalf of the National Center for Education Statistics (NCES) within the Department of Education. We do research to help clients make decisions about many different types of issues.

This particular method that we’re using today is what we call a focus group discussion. It allows us to talk with people in a bit more depth than other methods such as surveys. We are looking to get your honest opinions. I will be asking questions, but those questions are meant to generate discussion. There are no right or wrong answers to anything that we will be discussing. I encourage you all to express your opinion even if it might be different from everyone else’s. We want all types of viewpoints – positive, negative, and everything in between. We do not have to come to any consensus on what we’ll be talking about. Everyone’s opinion is important so I would like to hear from everyone.

You’ve been shown some information and reports. I just want to point out that these are not materials that I have developed, so you will not flatter me or insult me by anything that you say. My job is to ensure that we cover everything that we need to cover and that I keep the discussion going. I have no stake in the outcome of the discussion.

We will be audio recording the session so that I have a record of the discussion to write my report. Once the report is written we will destroy the recording.

Are you all comfortable with using the WebEx technology? ANSWER ANY QUESTIONS ABOUT WEBEX

I’d like everyone to introduce themselves. Let me start (MODERATOR INTRO). Now, tell me about yourself. When you do, please be sure to tell me your name (first name is okay), a little bit about your school (size, grades, type of school), how long you’ve been in education and anything else you’d like to tell us about yourself.

REACTION TO REPORT

You’ve all had an opportunity to review the reports we made available.

What is your overall reaction to these reports?

What do you like about this kind of report?

What don’t you like about it?

Is it easy to understand? IF NOT: What is confusing or difficult to understand?

Is the information in here interesting?

Does it present information that is relevant to your school?

Is the information relevant directly to you? For whom might it be more relevant?

Do you find the information credible? Why?/why not?

Are there any limitations to this information? What limitations do you see?

Is there anything that might be missing? Is there anything else that you would like to see in this kind of report?

Is there anything in here that shouldn’t be here? Is there anything that’s inappropriate or offensive in any way?

How does this compare to the kind of information that you receive on other types of tests?

How would you use this information? In what way would it help you? PROBE FOR:

General planning
Curriculum development
To discuss with teachers
To discuss with parents

Would you want to share this report with anyone? With whom?

What role would your teachers play in using this information?

What role would your subject or curriculum specialists play in using this information?

Would anyone else in your school play a role or use this information? Who? In what way would they use it?

Do you think that having this kind of information would make your school more likely to participate in NAEP? Why?/Why not?

What limitations do you see with the data or report? Are there specific ways in which the data or report should not be used?

What about the visual/graphic appearance/layout of the report? How do you feel about the way the report looks?

Is it easy to read?

Is it engaging/inviting?

How does the layout/appearance compare to the kind of reports that you receive on other types of tests?

How would you expect to get this report? PROBE: Hard copy, email, online/website

SOURCES OF INFORMATION/INVOLVEMENT

Where do you get information about what’s happening in education? About what’s happening in your school system?

What sources do you trust the most in getting information about education?

Do you read any publications/newspapers that deal specifically with education or your school?

Do you pay attention to education stories in the general media – on the television news or articles in the newspaper or magazines?

Do you check any web sites? Which ones?

How does your school communicate with you?

How does your school communicate with parents?

Where would you go to get additional information about NAEP in general?

What kind of information would you like to get about NAEP?

CONCLUSIONS

Do you have any additional comments about these reports, NAEP, or education in general?

Thank you.

Moderator’s Guide: Teachers/Subject Specialists

INTRODUCTION

My name is Maria Ivancin and I work for the Market Research Bureau, a research company. We do research to help clients make decisions about many different types of issues.

This particular method that we’re using today is what we call a focus group discussion. It allows us to talk with people in a bit more depth than other methods such as surveys. We are looking to get your honest opinions. I will be asking questions, but those questions are meant to generate discussion. There are no right or wrong answers to anything that we will be discussing. I encourage you all to express your opinion even if it might be different from everyone else’s. We want all types of viewpoints – positive, negative, and everything in between. We do not have to come to any consensus on what we’ll be talking about. Everyone’s opinion is important so I would like to hear from everyone.

I will be presenting some materials a little bit later in the discussion. These are not materials that I have developed, so you will not flatter me or insult me by anything that you say. My job is to ensure that we cover everything that we need to cover and that I keep the discussion going. I have no stake in the outcome of the discussion.

We will be audio recording the session so that I have a record of the discussion to write my report. Once the report is written we will destroy the recording.

Behind me you will see a two-way mirror. There are some people who are interested in listening to our discussion. We put them in a separate room because we do not want them to influence you in any way. Their presence in the same room, might affect how you would respond to some of my questions. We do not want that. We want to eliminate any possible bias.

I’d like everyone to introduce themselves. Let me start (MODERATOR INTRO). Now, tell me about yourself. When you do, please be sure to tell me your name (first name is okay), the grades/subject areas you teach, how long you’ve been teaching and anything else you’d like to tell us about yourself.

Today we’re going to be talking about testing in schools.

TESTING

Tell me a little about the different types of tests that are given in your school. What kinds of tests are given in your school? How often? When?

What role does testing play in education today?

What role does it play in your school?

What’s good about testing?

Does it help your students? In what way?

Does it help you in any way? How?

Does it improve education in general? How?

What’s not good about testing?

What kinds of state standardized tests are given in your school?

Are there any other tests that you are aware of that are administered as a way of improving education in general?

Are there any national tests that you are aware of?

Are you aware of any national assessments of education?

Does the Federal government do any kind of testing to assess the quality of education?

NAEP – AWARENESS, KNOWLEDGE AND ATTITUDES

IF NOT MENTIONED: Have you ever heard of the National Assessment of Educational Progress or NAEP? Have you ever heard of the Nation’s Report Card?

What do you know about it?

How did you first hear about it?

Has your school ever been involved in a NAEP test?

Have you ever been involved in a NAEP test?

Are you aware of releases of NAEP reports?

Do you know when they are released?

Have you ever seen a NAEP report?

What does it include? What does it show?

Do you receive NAEP reports when they are released?

Do you know if your school receives reports when they are released?

Do you/your school use NAEP data? How?

Does it provide you with valuable/useful information?

IF NOT AWARE OF NAEP – READ DESCRIPTION

What is your overall reaction to NAEP/The Nation’s Report Card based on this description?

Is this something that you were aware of prior to today?

What is good about NAEP? What does it do to help improve education?

What kinds of problems or concerns, if any, would you have with it? What disadvantages do you think there might be?

REACTION TO REPORT

What kinds of information do you think NAEP should provide? PROBE FOR:

To the general public
To policymakers
To the education community
To the schools who take part in NAEP

SHOW REPORTS TO RESPONDENTS. ALLOW A FEW MINUTES FOR THEM TO REVIEW.

What is your overall reaction to this?

What do you like about this kind of report?

What don’t you like about it?

Is it easy to understand? IF NOT: What is confusing or difficult to understand?

Is the information in here interesting?

Does it present information that is relevant to you? For whom might it be more relevant?

Would you find the information credible? Why?/why not?

Are there any limitations to this information? What limitations do you see?

Is there anything that might be missing? Is there anything else that you would like to see?

Is there anything in here that shouldn’t be here? Is there anything that’s inappropriate or offensive in any way?

How does it compare to the kind of information that you receive on other types of tests?

How would you use this information? In what way would it help you? PROBE FOR:

General planning
Curriculum development
To discuss with parents

Would you want to share this report with anyone? With whom?

What role would your principal play in using this information?

Would anyone else in your school play a role or use this information? Who? In what way would they use it?

Do you think that having this kind of information would make your school more likely to participate in NAEP? Why?/Why not?

EXPLAIN LIMITATIONS OF DATA. Do these limitations affect how you view this data/report? In what way?

What about the visual/graphic appearance/layout of the report? How do you feel about the way the report looks?

Is it easy to read?

Is it engaging/inviting?

How does the layout/appearance compare to the kind of reports that you receive on other types of tests?

How would you expect to get this report? PROBE: Hard copy, email, online/website

SOURCES OF INFORMATION/INVOLVEMENT

Where do you get information about what’s happening in education? About what’s happening in your school system?

What sources do you trust the most in getting information about education?

Do you read any publications/newspapers that deal specifically with education or your school?

Do you pay attention to education stories in the general media – on the television news or articles in the newspaper or magazines?

Do you check any web sites? Which ones?

How does your school communicate with you?

Where would you go to get additional information about NAEP in general?

What kind of information would you like to get about NAEP?

CONCLUSIONS

Do you have any additional comments about these reports, NAEP, or education in general?

Thank you.

NAEP 2011 School Reports Focus Group Studies – Volume II 18

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	Teachers Working Group
Author	Dresher, Amy
File Modified	0000-00-00
File Created	2021-02-04