MEMORANDUM
DATE:
TO: Shelly Martinez
Office of Information and Regulatory Affairs, Office of Management and Budget
FROM: Stephen Provasnik
National Center for Education Statistics
THROUGH: Kashka Kubzdela
National Center for Education Statistics
SUBJECT: Differences between the TIMSS 2011 and PIRLS 2011 questionnaires used in the field trial (FT) and those submitted for full scale (FS) study (OMB # 1850-0645)
To aid OMB’s review, this memo outlines differences between Trends in International Mathematics and Science Study (TIMSS) and the Progress in International Reading Literacy Study (PIRLS) materials approved under field test (FT) 2010 emergency clearance on November 3, 2009 (OMB# 1850-0645 v.5) and those included in the current regular submission package for the full-scale (FS) study 2011 data collection.
Differences between the field test (FT) and full-scale (FS) or main study are listed below for each of the questionnaires, in the following order:
TIMSS Grade 8 Questionnaires
TIMSS Grade 4 Questionnaires
TIMSS-PIRLS Grade 4 School Questionnaire
PIRLS Questionnaires.
Note that instead of a separate TIMSS grade 4 school questionnaire and a separate PIRLS school questionnaire, we will use a combined grade 4 school questionnaire.
Except for few changes to the questions added as U.S. national options, all changes to questions were made by the international organizers.
Rewording of existing question
In the TIMSS school questionnaire, the following two response questions were reworded from the field test version:
Question 12 B (“How often does your school ask parents to do the following? 1. Never, 2. Once a year, 3. 2-3 times a year, 4. More than 3 times a year”):
“Ask parents to volunteer for school projects, programs, and trips” is now worded as “Volunteer for school projects, programs, and trips”
“Ask parents to serve on school committees” is now worded as “Serve on school committees”“
Deleted questions
Several questions/items were deleted from the field-tested school questionnaire. The following questions or items no longer appear in the FS school TIMSS questionnaire:
Do eighth-grade teachers in your school have an opportunity to participate in a mentoring program?
Two items were deleted from the question “How much is your school’s capacity to provide instruction affected by a shortage or inadequacy of the following?”
Second language teachers
Special equipment for physically disabled students
One item was deleted from the question “How often does your school ask parents to do the following?”
Ask parents to raise funds for the school
One item was deleted from the question “During the past year, approximately how much time have you spent on the following school leadership activities in your role as a school principal?”
Teaching some classes to understand your students and their problems
Rewording of existing questions
Question 9A: “Do you use computers in your teaching?” is now worded as “Do you use computers in your teaching in any of the following ways? a) For preparation, b) For administration, c) In your classroom instruction.”
Additions
Question 5 (“During your college or university education, what was your major or main area(s) of study?”) has added Biology, Physics, Chemistry, and Earth Science as response options for grade 8 math teachers.
Question 23 C (“How often do you have the students do the following computer activities during mathematics lessons?”) now includes the response option “Process and analyze data.”
Rewording of existing questions
Question 9A (formerly “Do you use computers in your teaching?”) is now worded as “Do you use computers in your teaching in any of the following ways? a) For preparation, b) For administration, c) In your classroom instruction.”
Rewording of existing questions
In response to some problems in the field test, the U.S. race/ethnicity question (1B) has reverted to the version of the ‘ethnicity’ question that has been used in prior administrations. Question 1B now reads: “Are you Hispanic or Latino? 1. Yes, I am Hispanic or Latino 2. No, I am not Hispanic or Latino”
The following response options were used in the FT but have been deleted from the FS version:
Yes, I am Mexican, Mexican American, or Chicano
Yes, I am Puerto Rican or Puerto Rican American
Yes, I am Cuban or Cuban American
Yes, I am from some other Hispanic or Latino background
In addition, the definition of “Hispanic or Latino” has become part of the script that U.S. test administrators read (e.g., “Hispanic or Latino includes anyone whose parents or ancestors are from Central or South America--places like Mexico, Puerto Rico, Cuba, El Salvador, Argentina, Chile, and other similar countries”).
In prior administrations of TIMSS and PIRLS, a series of questions sought information on the nationality (or birth place) of the parents or guardians of the student, as well as the student. In the FT, these questions were broken out so that they asked first “Was your mother born in the United States? 1. Yes 2. No” and then “Was your stepmother born in the United States? 1. I don’t have a stepmother. 2. Yes. 3. No.” The same pattern was followed for the father.
For the FS main study questionnaire, questions related to parents’ national origin have reverted to combine biological parents with step-parents and guardians. Specifically, question 4 was reworded as follows:
“4A: Was your mother (or stepmother or female legal guardian) born in the United States? (“United States” includes the 50 states, its territories, the District of Columbia, and U.S. military bases abroad.)
1. Yes
2. No
4 B: Was your father (or stepfather or male legal guardian) born in the United States?
1. Yes
2. No”
Deleted items
One response option was deleted from question 9 (“Do you have any of these things at your home?”):
Three vehicles (such as a car, small truck, or sport utility vehicle)
Five items were deleted from question 13 (“How often do the following things happen at home? 1. Every day or almost every day 2. Once or twice a week 3. Once or twice a month 4. Never or almost never”):
My parents help me with my schoolwork
My parents help me practice my math skills
My parents help me practice my science skills
I talk with my parents about what I am reading
My parents like to know where I am after school
One item was deleted from question 17 (“How much do you agree with these statements about your mathematics lessons? 1. Agree a lot 2. Agree a little 3. Disagree a little 4. Disagree a lot”):
I have difficulty paying attention
One item was deleted from question 18 (“How much do you agree with these statements about mathematics? 1. Agree a lot 2. Agree a little 3. Disagree a little 4. Disagree a lot”):
I keep working on mathematics problems until I am finished
One item was deleted from question 19 (“How much do you agree with these statements about learning science? 1. Agree a lot 2. Agree a little 3. Disagree a little 4. Disagree a lot”):
Learning science takes hard work
One item was deleted from question 20 (“How much do you agree with these statements about your science lessons? 1. Agree a lot 2. Agree a little 3. Disagree a little 4. Disagree a lot”):
I have difficulty paying attention
One item was deleted from question 21 (“How much do you agree with these statements about science? 1. Agree a lot 2. Agree a little 3. Disagree a little 4. Disagree a lot”):
I keep working on science problems until I am finished
Deleted questions
Based on problems encountered in FT, this U.S. question (below) has been eliminated from FS main study questionnaires:
“The next question asks about people you live with. If you live in more than one place, answer the question about the place that is your home most of the time.”
Does your mother live at home with you? Y/N
Does your stepmother or female legal guardian live at home with you? Y/N
Does your father live at home with you? Y/N
Does your stepfather or male legal guardian live at home with you? Y/N
Does your grandmother or grandfather live at home with you? Y/N
How many brothers or sisters live with you? ____
How many OTHER family members live at home with you? ____
How many people live at home with you who are not part of your family? ____
In addition, the following three questions were deleted from the student questionnaire.
Do you ever use a computer? 1. Yes, 2. No
What do you think about the teachers and students in your school? Tell how much you agree with these statements. 1. Agree a lot 2. Agree a little 3. Disagree a little 4. Disagree a lot
I think that students in my school try to do their best
I think that teachers in my school want students to do their best
I think that students in my school are not interested in learning
I think that teachers in my school listen to what students have to say
I think that students in my school work hard on their schoolwork
I think that teachers in my school understand that some students have difficulty in learning
I think that students in my school pay attention in class
I think that teachers in my school make an extra effort to help students
I think that students in my school feel that they can ask teachers for extra help
I think that teachers in my school care about all students
How often do you use a computer to do these things? 1. Every day or almost every day 2. Once or twice a week 3. Once or twice a month 4. Never or almost never
Look up information for school
Look up information on a topic you like (e.g., your favorite athlete, animals you like, or places you have traveled to)
Exchange information about schoolwork online (e.g., getting homework, asking for help, discussion boards)
Rewording of existing questions
Question 9A (“Do you use computers in your teaching? 1. Yes, 2. No”) is now worded as “Do you use computers in your teaching in any of the following ways? a) For preparation, b) For administration, c) In your classroom instruction.”
Question 34 (“If Yes, How often do you have the students do the following computer activities during science lessons? 1. Every or almost every day, 2. Once or twice a week, 3. Once or twice a month, 4. Never or almost never”) had one item reworded:
“Use instructional software to develop and practice skills and procedures” now worded as “Practice skills and procedures”
Added items
Question 34 (“If Yes, How often do you have the students do the following computer activities during science lessons? 1. Every or almost every day, 2. Once or twice a week, 3. Once or twice a month, 4. Never or almost never”) had two items added:
Do scientific procedures or experiments
Study natural phenomena through simulations
Question 23 (“If Yes, How often do you have the students do the following computer activities during mathematics lessons? 1. Every or almost every day 2. Once or twice a week 3. Once or twice a month 4. Never or almost never”) added the following item:
Explore mathematics principles and concepts
Deleted questions
The following two questions were deleted from the TIMSS grade 4 teacher questionnaire:
“Are you the students’ general teacher for reading, mathematics and science? 1. Yes 2. No”
“Is science taught mainly as a separate subject (i.e., not integrated with other subjects) to the students in this class? 1. Yes 2. No”
Rewording of existing questions
As in the TIMSS grade 8 student questionnaire, the race/ethnicity question (1B) and national origin questions (4A and B) were reworded. We have reverted to the wording used in prior administrations. See comments under TIMSS grade 8 student questionnaire in regard to both questions.
Question 9D about activities outside of school was reworded to include “Boys and Girls Club.” For FS main study questionnaire it now reads “Do you belong to a club outside of school (like Girl Scouts, 4-H, or Boys and Girls Club)?”
Deleted questions
“If you were not born in the United States, how old were you when you came to the United States?
Older than 5 years old
1 to 5 years old
Younger than 1 year old”
“The next question asks about people you live with. If you live in more than one place, answer the question about the place that is your home most of the time.
Does your mother live at home with you? Y/N
Does your stepmother or female legal guardian live at home with you? Y/N
Does your father live at home with you? Y/N
Does your stepfather or male legal guardian live at home with you? Y/N
Does your grandmother or grandfather live at home with you? Y/N
How many brothers or sisters live with you? ____
How many OTHER family members live at home with you? ____
How many people live at home with you who are not part of your family? ____
In addition, as with the grade 8 student questionnaire, the TIMSS grade 4 questionnaire has deleted the U.S. question on family composition for the FS.
A single school questionnaire will be utilized during the FS. This questionnaire has the following changes from the field study.
Rewording of existing questions
Five items within question 19 were reworded from the FT to the FS for clarification.
“Identifying the main ideas of what they have read” is now worded as “Identifying the main idea of a text”
“Explaining or supporting their understanding of what they have read” is now worded as “Explaining or supporting students' understanding of a text”
“Comparing what they have read with experiences they have had” is now worded as “Comparing a text with personal experience”
“Making predictions about what will happen next in the text they are reading” is now worded as “Making predictions about what will happen next in a text”
“Making generalizations and drawing inferences based on what they have read” is now worded as “Making generalizations and drawing inferences based on a text”
Deleted questions
Two questions were completely removed from the school questionnaire.
“Do fourth-grade teachers in your school have an opportunity to participate in a mentoring program? 1. Yes 2. No”
“Do the fourth-grade students in your school have separate teachers for any of the core subjects? 1. Yes 2. No.
English
Mathematics
Science”
Deleted items
Two items were deleted from question 12 (“How much is your school's capacity to provide instruction affected by a shortage or inadequacy of the following? 1. Not at all 2. A little 3. Some 4. A lot”):
Second language teachers
Special equipment for physically disabled students
One item was deleted from question 17 (“During the past year, approximately how much time have you spent on the following school leadership activities in your role as a school principal? 1. No time 2. Some time 3. A lot of time”):
Teaching some classes to understand your students and their problems
Rewording of existing questions
Question 9 A (formerly “Do you use computers in your teaching? 1. Yes, 2. No”) is now worded as “Do you use computers in your teaching in any of the following ways? a) For preparation, b) For administration, c) In your classroom instruction.”
Deleted questions
“Are you the students’ general teacher for reading, mathematics and science? 1. Yes 2. No”
“How many students receive additional or more challenging reading instruction because they are advanced readers? ____ fourth-grade students in this class”
Rewording of existing questions
As in the TIMSS grade 8 student questionnaire, the race/ethnicity question (1B) and national origin questions (4A and B) were reworded. We have reverted to the wording used in prior administrations. See comments under TIMSS grade 8 student questionnaire in regard to both questions.
Question 9D about activities outside of school was reworded to include “Boys and Girls Club.” For FS main study questionnaire it now reads “Do you belong to a club outside of school (like Girl Scouts, 4-H, or Boys and Girls Club)?”
The item stem for question 17 (formerly ““Think about your lessons about the reading you do for school. Tell how much you agree with each of these statements.”) was changed to “Think about the reading you do for school. How much do you agree with these statements about your reading lessons?”
Deleted questions
“Do you ever use a computer? 1. Yes 2. No”
“How often do you use a computer to do these things? 1. Everyday or almost everyday 2. Once or twice a week 3. Once or twice a month 4. Never or almost never”
Look up information for school
Look up information on a topic you like (for example, your favorite athlete, animals you like, or places you have traveled to)
Exchange information about schoolwork online (for example, getting homework, asking for help, discussion boards)
“If you were not born in the United States, how old were you when you came to the United States?
Older than 5 years old
1 to 5 years old
Younger than 1 year old”
In addition, as with the grade 8 student questionnaire, the PIRLS grade 4 questionnaire has deleted the U.S. question on family composition for the FS.
Deleted items
One response option was deleted from question 9 (“Do you have any of these things at your home?”):
Three vehicles (such as a car, small truck, or sport utility vehicle)
Deleted items from question 8 (“How often do the following things happen at home? 1. Everyday or almost everyday 2. Once or twice a week 3. Once or twice a month 4. Never or almost never””):
My parents help me with my schoolwork
My parents help me practice my reading
My parents help me practice my math skills
I talk with my parents about what I am reading
My parents like to know where I am after school
Deleted items from question 17 (“Think about the reading you do for school. How much do you agree with these statements about your reading lessons? 1. Agree a lot 2. Agree a little 3. Disagree a little 4. Disagree a lot”):
The things my teacher asks me to read are difficult
I have difficulty paying attention
Deleted items from question 21 (“Do you read for any of the following reasons? Tell how much you agree with each of these statements. 1. Agree a lot 2. Agree a little 3. Disagree a little 4. Disagree a lot”):
I like to get compliments for my reading
I read to do better in school
I like reading stories that are entertaining (for example, funny, scary, or with exciting adventures)
Being a good reader helps me do better in school
I read to please my teacher
Being a good reader helps me in my daily life
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | MEMORANDUM |
| Author | Stephen Provasnik |
| File Modified | 0000-00-00 |
| File Created | 2021-02-01 |