Study Changes

DATE: 09 January 2015

TO: Shelly Martinez, OMB

THROUGH: Kashka Kubzdela, NCES

FROM: Patrick Gonzales, NCES

RE: Program for International Student Assessments (PISA) 2015 Main Study (OMB# 1850-0755 v.16)

OMB granted clearance for the PISA 2015 Field Test and Main Study Recruitment on September 19, 2013 (1850-0755 v.14). The purpose of this memo is to summarize changes between the activities and materials approved in the field test package and those that will be used in the PISA 2015 Main Study as described in Parts A and B and appendixes in this submission (the attached final U.S. versions of the 2015 main study questionnaires have been approved by the PISA Governing Board and PISA international consortium).

Appendix D – Nonresponse Bias Analysis Plan

The following phrase was added to the NRBA plan qualifying that per NCES standards, the NRBA analyses would be done only if the response rates fall below NCES standards: “If school response rates are below NCES statistical standards”.

Part A - Procedural

In the PISA 2015 Field Test and Main Study Recruitment request approved on September 19, 2013, we indicated that we planned to subsample 8 students to return for a second assessment session to take the financial literacy assessment on computer. We will now subsample 11 students to ensure that we have the same number of students assessed as expected. This change is a result of a review of data from those countries that administered financial literacy in the field test and is being implemented across all countries administering financial literacy in the main study. We have updated Part A and Part B to reflect this procedural change. This change does not affect the burden estimate.

Part A - Burden

We have updated the burden estimates (see Section 12 of Part A) to reflect the possible participation in PISA 2015 of two states that desire state-level estimates (in addition to the participation of Puerto Rico which was included in 1850-0755 v.14). The revised overall estimate is based on a national sample, two state samples, and a Puerto Rico sample.

Appendix C - Questionnaire Items

Changes between the field test and main study questionnaire items, including revisions to the field test items, new items, and eliminated field test items (more items were field tested than were planned to be used in the main study), are listed below in tables. In the main study, one student questionnaire form will be used, whereas the field test used 3 forms with items rotated across students. As discussed in the briefing NCES provided to OMB in June 2014, the field trial evaluated whether the items and scales functioned properly to measure the target constructs: (a) that they were intended to measure (validity), (b) in a cross-nationally valid manner, and (c) efficiently. In some cases, the field trial was used to compare alternative measurement approaches for measuring constructs. Several new, innovative measurement methods were introduced in the PISA 2015 field trial. These appeared throughout the questionnaires, but primarily in the student and teacher questionnaires. Statistical analyses of the field test data conducted by the International Consortium focused on evaluating the psychometric functioning of (a) items within scales and (b) the scales themselves. Some of the new methods were deemed successful and the PISA Governing Board approved their use in the main study questionnaires. The selection of items for the main study also took into consideration research risk, policy implications, country priorities, implementation issues, and per-item and per-construct timing information.

Additionally, Appendix C presents the student locator items that will be asked of students to support a possible future methodological study to validate PISA by relating student performance on PISA 2015 to other cognitive, education, and employment outcomes. In addition to the items that were cleared for use in PISA 2012 (1850-0755 v.13), we will also ask students for email addresses of a relative or another contact to facilitate follow up.

Summary of Changes to Field Test School and Student Questionnaire Items

1. School Questionnaire Items

1. Modified Items

2. Deleted items

2. Student Questionnaire Items

Note: There is only 1 form of the student questionnaire for the Main Study. Each student will complete the questionnaire estimated at 30 minutes. Also, there are no new items in the Student questionnaire for the Main Study (i.e., added post field trial).

1. Modified Items

2. Deleted items

3. General Teacher Questionnaire Items

1. Modified Items

2. Deleted Items

4. Science Teacher Questionnaire Items

1. Modified Items

2. Deleted Items

1. School Questionnaire Items

1. Modified Items

2015 Main Study
Item #	Item
General Direction	Note: The estimated time to complete has been reduced from 60 minutes to 45 minutes
SC002	Note: Stem has been updated to school year of interest As of February 1, 2015, what was the total school enrolment (number of students)?
SC018	Note: Responses Q03NA and Q04NA have been deleted. How many of the following teachers are on the staff of your school? Q03TA) Teachers with less than an associate's degree Q04TA) Teachers with an associate's degree

SC037	Note: Responses Q11NA, Q12NA, Q13NA have been deleted. Do the following arrangements aimed at quality assurance and improvements exist in your school and where do they come from? Q11TA) Policies on negative consequences for teachers performing poorly Q12TA) Policies on rewards for teachers performing exceptionally well Q13TA) Policies on the continuing professional development of particular poorly performing teachers
SC040	Note: Responses Q01NA, 04NA, 06NA, 07NA, 08NA, 09NA, 10NA, 13NA, and 14NA have been deleted. Based on your last internal school evaluation results, did your school implement any measures in the following areas? Q01NA) School's educational resources (e.g., textbooks, computer, library materials, laboratory materials) Q04NA) Extra-curricular activities (such as participation, content) Q06NA) Assessment practices Q07NA) Social climate in school Q08NA) Teacher cooperation Q09NA) Use of ICT for teaching and learning Q10NA) School management (leadership, counseling, cooperation, school program) Q13NA) Diversity management Q14NA) Cooperation with external partners
SC041	Note: Responses Q02NA and Q07NA have been deleted. Thinking about the last external evaluation in your school. Do the following statements apply? Q02NA) We used the results of the external evaluation to draw up clearly defined measures. Q07NA) The effects triggered by the external evaluation "disappeared" very quickly at our school.
SC063	Note: Responses Q01NA, Q07NA, and Q08NA have been deleted. Do the following statements about parental involvement apply to your school? Q01NA) Our school invites parents to participate in school-wide activities on a regular basis. Q07NA) Our school identifies and integrates resources and services from the community to strengthen school programs, family practices, and student learning and development. Q08NA) Our school keeps parents informed about their child's progress.
SC064	Note: Responses Q05NA01 and Q06TA01 have been deleted. During 2012-2013 academic year, what proportion of students’ parents participated in the following school-related activities? Q05NA01) Volunteered to support school activities (e.g., volunteered in the school library, media center, or cafeteria, assisted a teacher, appeared as a guest speaker) _____ Q06TA01) Assisted in fundraising for the school. _____

2. Deleted Items

Item #	Stem
SC005	Do the following statements about ICT use apply to your school?
SC006	How many instructional days per calendar week does your school provide to students in 10th grade?
SC007	How many instructional days are intended in this school year for students in 10th grade?
SC008	On approximately how many days in the last 12 months was your school closed or deviated from the regular curriculum because of the following reasons?
SC011	Which of the following statements best describes the schooling available to students in your location?
SC015	About what percentage of your total funding for a typical school year comes from the following sources?
SC020	How many teachers in your school are teaching any of the following science subjects?
SC021	Which of the following procedures for recruiting teachers is more common at your school?
SC022	Is the completion of a teacher education or training program required to teach at your school?
SC023	In your school the completion of a teacher education or training program is required to teach. How strict is this rule?
SC024	In the last academic year, did you fill all vacant 10th grade science teaching positions at your school?
SC026	For how many days within a school year is a teacher in your school required to take part in professional development activities?
SC028	Does your school have a formal policy concerning professional development activities?
SC029	Does your formal school policy concerning professional development activities include the following topics?
SC030	What resources does your school provide for professional development activities?
SC031	Which proportion of all professional development activities attended by science teachers from your school has been dedicated to each of the following three areas?
SC033	Please indicate the frequency that each of the following occurs in this school following a teacher appraisal.
SC038	Did the last internal evaluation at your school address any of the following?
SC039	Thinking about the last internal evaluation in your school. Do the following statements apply?
SC043	Which policies regarding grade retention are implemented in your school?
SC044	Which measures to support students at risk of academic failure are implemented in your school?
SC045	Among the students in your school’s final grade, what proportion of them left school without a diploma or alternative credential (e.g., a GED) in the last academic year?
SC046	Does your school have a policy on additional instruction for grades with 15-year-olds (e.g., tutoring, private lessons, additional instruction)?
SC047	How do your school, district or other governing authority or government handle additional instruction?
SC049	Schools differ in the way they address cultural diversity. For each of the following pairs of statements, please choose the statement that the majority of teachers in your school would agree with.
SC050	How many teachers in your school would agree with the following statements?
SC051	Do the following statements reflect practices for multicultural learning in your school?
SC054	Do the following statements regarding students’ lunch apply to your school?
SC055	Who pays for the following things at your school?
SC056	Which of the following statements about science education apply to your school?
SC057	Does your school offer science lessons in addition to the science classes offered during the usual school hours?
SC058	What is the purpose of these additional science lessons?
SC060	Do you agree with the following statements about science education at your school?
SC062	Think about the teachers in your school. How much do you agree with the following statements?
SC065	Which statement below best characterizes parental expectations towards your school?
SC066	Which of the following statements about science related collaboration is true for your school?

2. Student Questionnaire

1. Modified Items

2015 Main Study
Item #	Item
ST011	Note: Responses Q13NA, Q14NA, and Q15NA have been deleted. Which of the following are in your home? Q13NA) A DVD player Q14NA) Flat screen TV/Plasma TV/LCD TV Q15NA) Cable TV/Pay TV/Satellite TV
ST012	Note: Response Q04NA has been deleted. How many of these are there at your home? Q04NA) Cell phones without Internet access
ST034	Note: Responses Q07TA, Q08TA, and Q09TA have been deleted. Thinking about your school: to what extent do you agree with the following statements? Q07TA) I feel happy at school. Q08TA) Things are ideal in my school. Q09TA) I am satisfied with my school.
ST064	Note: Response Q04NA has been deleted. To what extent can you choose the following for your science courses? Q04NA) I can choose from different teachers teaching the same course(s).
ST082	Note: Responses Q04NA, Q05NA, Q06NA, Q07NA, Q10NA, Q11NA, Q15NA, Q16NA, Q17NA, and Q18NA have been deleted. To what extent do you disagree or agree with the following statements about yourself? Q04NA) I like to be in charge of groups or projects. Q05NA) I enjoy sharing ideas. Q06NA) I convince others to see things my way. Q07NA) I enjoy exchanging ideas. Q10NA) I like convincing peers. Q11NA) I enjoy bringing a team together. Q15NA) I enjoy cooperating with peers. Q16NA) I am open to all sorts of opinions. Q17NA) I enjoy providing feedback. Q18NA) I am flexible when working with a team.
ST092	Note: Responses Q03NA, and Q07TA have been deleted. How informed are you about the following environmental issues? Q03NA) Acid rain Q07NA) Energy shortages
ST093	Note: Responses Q02NA, and Q09TA have been deleted. Do you think problems associated with the environmental issues below will improve or get worse over the next 20 years? Q02NA) Energy shortages Q09NA) Acid rain
ST095	Note: Responses Q01NA, Q02NA, Q03NA, Q05NA, Q06NA, Q09NA, Q10NA, Q11NA, Q12NA, Q14NA, and Q16NA have been deleted. To what extent are you interested in the following science topics? Q01NA) The concept of an organism (e.g., unicellular and multicellular) Q02NA) Humans (e.g., health, nutrition) Q03NA) Populations (e.g., species, evolution, biodiversity) Q05NA) Structure of matter (e.g., particle model, bonds) Q06NA) Chemical changes of matter (e.g., chemical reactions, energy transfer) Q09NA) Interactions between energy and matter (e.g., light and radio waves) Q10NA) Change in Earth systems (e.g., plate tectonics, constructive and destructive forces) Q11NA) Earth’s history (e.g., fossils, origin and evolution) Q12NA) Earth in space (e.g., gravity, solar systems, galaxies) Q14NA) Explanations of how things work Q16NA) How science can help solve environmental problems
ST098	Note: Responses Q04NA, Q11NA, Q12NA, and Q13NA have been deleted. When learning science topics at school, how often do the following activities occur? Q04NA) The conclusions of experiments integrate students’ arguments. Q11NA) Students have the opportunity to repeat experiments and compare observations. Q12NA) Students are encouraged to question and critique scientific arguments made by other students. Q13NA) Students are required to engage in discussions among
ST103	Note: Responses Q02NA, Q04NA, Q05NA, Q06NA, Q07NA, Q09NA, Q10NA, Q12NA, Q13NA, Q14NA, Q15NA, Q16NA, Q17NA, Q18NA, Q19NA and Q20NA have been deleted. How often do these things happen in your classes for this science course? Q02NA) A small group discussion between students takes place. Q04NA) Current scientific issues are discussed. Q05NA) Students make calculations using scientific formulas. Q06NA) The teacher uses an interactive white board. Q07NA) Students do their own scientific study and related research. Q09NA) Students carry out practical work. Q10NA) Students write up laboratory reports. Q12NA) The teacher discusses questions of practical relevance. Q13NA) Students read materials from a textbook. Q14NA) Students take notes from the board. Q15NA) Students discuss materials from a textbook. Q16NA) Students watch videos. Q17NA) Students use the internet. Q18NA) The class corrects homework or a test. Q19NA ) Students fill out worksheets. Q20NA) Students present something to the rest of the class.
ST107	Note: Response Q04NA has been deleted. How often do these things happen in your classes for this science course? Q04NA) The teacher provides individual support for advanced students.
ST123	Note: Responses Q05NA, Q06NA, Q07, Q08NA, and Q09NA have been deleted. Thinking about this school year: to what extent do you agree or disagree with the following statements? Q05NA) My parents support school rules and school discipline. Q06NA) My parents try to protect me from family problems that may interfere with my schooling. Q07NA) My parents encourage me to learn new skills. Q08NA) My parents provide me with support to complete school tasks. Q09NA) My parents ask me questions to initiate a deeper understanding of what I’m studying.
ST131	Note: Responses Q02NA, Q05NA, Q07NA, Q09NA, Q10NA, and Q12NA have been deleted. How much do you disagree or agree with the statements below? Q02NA) Some ideas in science today are different than what scientists used to think. Q05NA) There are some questions that scientists cannot answer. Q07NA) New discoveries can change what scientists think is true. Q09NA) It is good to have an idea before I start an experiment. Q10NA) One important part of science is doing experiments to come up with ideas about how things work. Q12NA) In science, there can be more than one way for scientists to test their ideas.

2. Deleted items

Item #	Stem
ST009	What is your mother currently doing?
ST010	What is your father currently doing?
ST017	Please imagine a ladder with steps numbered from zero at the bottom to ten at the top. The top of the ladder represents the best possible life for you and the bottom of the ladder represents the worst possible life for you.
ST018	Thinking about your life: how much do you disagree or agree with each of the following statements?
ST020	How old were you when you arrived in the United States?
ST023	Which language do you usually speak with the following people?
ST024	Discrimination refers to negative behavior directed against people because of their membership in a particular group. Would you describe yourself as a member of a cultural or ethnic group that is discriminated against in the United States?
ST025	Thinking about your experiences in school: to how many teachers in your school do the following statements apply?
ST028	When you eat lunch at school, where do you usually get the food you eat?
ST029	During the past 7 days, how many days did you eat or drink the following?
ST030	Were you taught any of the following in your classes this school year or last school year?
ST033	How much do you disagree or agree with each of the following statements about yourself?
ST036	Thinking about your school: to what extent do you agree with the following statements?
ST037	How often do these things happen in your classes?
ST040	Thinking about the teachers at your school: to what extent do you agree with the following statements?
ST041	Who pays for the following things at your school?
ST042	How many weeks did you spend on the following activities during your school vacation?
ST045	Regarding the activities mentioned in the previous question, do you or your family have to save money in advance to participate in them?
ST046	How often do you do the following things with your parents or someone else in your home?
ST048	Is your mother working as an employee or is she self-employed?
ST049	At her job, does your mother have any formal responsibility for supervising the work of other employees?
ST050	How many people work for your mother’s employer at the place where she works?
ST051	Does your mother work on her own or does she have employees?
ST052	How many people does your mother employ?
ST053	Is your father working as an employee or is he self-employed?
ST054	At his job, does your father have any formal responsibility for supervising the work of other employees?
ST055	How many people work for your father’s employer at the place where he works?
ST056	Does your father work on his own or does he have employees?
ST057	How many people does your father employ?
ST058	How many class periods per week are you typically required to attend for the following subjects?
ST066	How often do these things happen in this science course?
ST067	In classes for this science course, approximately what proportion of the class time is usually lost due to class interruptions described in the previous question?
ST069	Think about your last two science class periods in the course you named above. About what proportion of time were you not concentrating on things related to this science course?
ST070	This school year or the last school year, have you taken additional instruction in the following subjects?
ST072	Think about your last two sessions in your additional science instruction. How often do these things happen in this additional, non-compulsory science instruction?
ST073	Think about your last two sessions in your additional science instruction. About what proportion of time were you not concentrating on things related to this additional science instruction?
ST074	During the last ten sessions of additional science instruction, how many did you miss?
ST075	The following questions ask about the most recent day you attended school. At what time did you do the following?
ST077	On the most recent day you attended school, how long did you study in the morning before going school?
ST079	On the most recent day you attended school, how long did you study after leaving school?
ST083	How often do you do the following at school?
ST084	How often do you do the following out-of-school?
ST085	How well does each of the following statements below describe you?
ST086	How well does each of the following statements describe you?
ST087	To what extent do you disagree or agree with the following statements about yourself?
ST088	To what extent do you disagree or agree with the following statements about yourself?
ST089	To what extent do you disagree or agree with the following statements about yourself?
ST090	Please read the descriptions about the following three students. Based on the information provided here, how much would you disagree or agree with the statement that this student is organized?
ST091	What kind of work do your father and mother do for their main jobs?
ST096	How interested are you in the following school subjects?
ST099	Within the last month at school, has a computer or a similar device (e. g., a tablet) ever been used for the following purposes in your science classes?
ST102	How often do these things happen in your science classes?
ST105	How often do these things happen in your classes for this science course?
ST106	How do you respond to the teacher’s feedback for this science course?
ST108	Thinking of your past two lessons in this science course: how much do you disagree or agree with the following statements?
ST109	Thinking of your past two lessons in this science course: how much do you disagree or agree with the following statements?
ST110	Thinking of your past two lessons in this science course: how much do you disagree or agree with the following statements?
ST112	How much do you agree with the statement below?
ST115	How informed are you about these topics?
ST116	Please read the descriptions about the following three students. Based on the information provided here, how much would you disagree or agree with the statement that this student is anxious about taking a test?
ST117	Please read the descriptions about the following three students. Based on the information provided here, how much would you disagree or agree with the statement that this student is anxious about taking a test?
ST118	To what extent do you disagree or agree with the following statements about yourself?
ST120	Thinking about your school: to what extent do you agree or disagree with the following statements?
ST122	Thinking about people who are important to you and how they view science: to what extent do you disagree or agree with the following statements?
ST124	Did you attend pre-school?
ST132	How easy do you think it would be for you to perform the following tasks on your own?
ST133	We are interested in what you think about the need for scientific skills in the job market today. How much do you disagree or agree with the following statements?
ST134	How much do you agree with the statements below?
ST135	How important should Lucy rate the statements below in her decision to buy the medicine?
ST136	How strongly should the statements below affect Joel’s decision to buy a helmet?
ST137	How strongly should the following statements affect Sandra’s decision to set up a training plan?
ST138	How strongly should the following statements affect Sandra’s decision to set up a training plan?
ST139	How strongly should the following statements affect Sandra’s decision to set up a training plan?
ST140	How strongly should the following statements affect Sandra’s decision to set up a training plan?
ST141	How much do you disagree or agree with the following statements about yourself?
ST142	Below you can see pairs of different occupations. In each line, please choose the one that you admire more.
ST143	How much do you disagree or agree with the statements about the use of new technologies (e.g., tablet computer, smartphone, Blu-ray player) below?
ST144	If you were in a position to decide on the application of a certain technology, which of the two arguments is more important in deciding to use each new technology?
ST145	How often do you use the following?

General Teacher Questionnaire

1. Modified Items

   2015 Main Study
   Item #	Item
   General Directions	Note: The time to complete the teacher questionnaire has reduced from 45 minutes to 30 minutes.
   TC007	Note: Responses Q03NA and Q04NA have been deleted. How many years of work experience do you have? Q03NA01) Year(s) working in other education roles (do not include years working as a teacher) ____ Q04NA01) Year(s) working in other jobs ____
   TC026	Note: Responses Q03NA and Q08NA have been deleted. We would like to know how you generally feel about your job. How strongly do you agree or disagree with the following statements? Q03NA01) I would like to change to another school if that were possible. Q08NA01) I think that the teaching profession is valued in society.
   TC055	Note: Responses Q09NA, Q10NA, Q12NA, Q15NA, and Q16NA have been deleted. To what extent do you use the following approaches to assign final semester grades to students in 10th grade? Q09NA) I recognize exceptional academic merit (e.g., taking part in a national contest). Q10NA) I use a set scale of percentages of correct responses to assess the students' performance. Q12NA) I consider the quality of homework. Q15NA) I consider students’ behavior (e.g., disruptive or supportive behavior). Q16NA) I base grades on my overall judgement of student's learning.
   TC060	Note: Responses Q01NA, Q03NA, Q05NA, Q08NA, Q10NA, Q11NA, Q12NA, Q13NA and Q14NA have been deleted. Q01NA) The principal gives us a sense of purpose Q03NA) The principal has high expectations for us as professionals Q05NA) The principal is aware of my expertise Q08NA) The principal delegates activities that are critical for attaining objectives Q10NA) The principal takes the community’s aspirations and requests into account Q11NA) The principal ensures that staffing is equitable Q12NA) The principal regularly observes classroom activities Q13NA) It is easy for staff to approach the principal Q14NA) The principal has brought a lot of autonomy to the school

2. Deleted Items

Item #	Stem
TC003	Have you taught 10th grade in the last 12 months?
TC008	Did you submit a written application for the position as a teacher directly to your school?
TC009	You did not submit an application to your school directly. Did the district or other governing agency assign you to your position?
TC010	Was your current position at your school your first choice?
TC011	How important were the following reasons for applying to work at this particular school?
TC016	You attended a standard teacher education or training program. What level of formal education did you reach there?
TC017	For how many months did you attend the teacher education or training program?
TC019	During the last 12 months, did you participate in any of the following professional development activities, and if yes, for how many days did they last?
TC022	How many days within a school year are you required to take part in professional development activities?
TC023	How does your school support your professional development activities?
TC024	Teachers play an important role in communication with parents. From your perspective, to what extent are the following topics of parent-teacher communication important?
TC025	To what extent are the following ways of communicating with your students’ parents important to you?
TC047	In your teaching, to what extent can you do the following?
TC049	For these activities, do teams use online collaboration tools such as email, chat, shared documents, etc.?
TC050	What do you do to improve your students’ collaboration skills?
TC027	To what extent do you disagree or agree with the following statements about your teaching practices?
TC056	Please estimate the percentage of students in 10th grade at this school who have the following characteristics.
TC057	Schools differ in the way they address cultural diversity. For each of the following pairs of statements, please choose the statement that the majority of teachers in your school would agree with.
TC058	How many teachers in your school would agree with the following statements?
TC059	Do the following statements reflect practices for multicultural learning in your school?
TC061	In relation to the most successful countries in international comparison studies, how much is the education system in the United States losing ground?
TC062	To what extent do you disagree or agree with the following statements regarding your school?
TC063	During the last academic year has there been at least one internal evaluation at your school?
TC064	Did the last internal evaluation at your school address any of the following?
TC065	Thinking about the last internal evaluation in your school. Do the following statements apply?
TC066	Based on your last internal school evaluation results, did your school implement any measures in the following areas?
TC068	Concerning the feedback you have received at this school, to what extent has it directly led to a positive change in any of the following?

Science Teacher Questionnaire

1. Modified Items

Note that TC007 and TC026 are general and science teacher questionnaire common items that have been modified.

2015 Main Study
Item #	Item
General Directions	Note: The time to complete the teacher questionnaire has reduced from 45 minutes to 30 minutes.
TC007	Note: Responses Q03NA and Q04NA have been deleted. How many years of work experience do you have? Q03NA01) Year(s) working in other education roles (do not include years working as a teacher) ____ Q04NA01) Year(s) working in other jobs ____
TC026	Note: Responses Q03NA and Q08NA have been deleted. We would like to know how you generally feel about your job. How strongly do you agree or disagree with the following statements? Q03NA01) I would like to change to another school if that were possible. Q08NA01) I think that the teaching profession is valued in society.
TC031	Note: Responses Q01NA, Q02NA, Q03NA, Q05NA, Q06NA, Q08NA, Q09NA, Q10NA, Q12NA, Q16NA, Q17NA, Q19NA, Q21NA and Q22NA have been deleted. To what extent do you disagree or agree with the following statements about regular cooperation among your fellow science teachers and yourself? Q01NA) We pursue the same educational objectives. Q02NA) We decide together what teaching materials should be used in lessons. Q03NA) Apart from end-of-term conferences, we rarely discuss what targets students are expected to achieve in science. Q05NA) We communicate information about subject-related issues. Q06NA) We cooperate constructively on ways of teaching science. Q08NA) We use our planning periods to work together. Q09NA) We exchange tasks that we used to create written tests. Q10NA) Our criteria for grading tests differ very little. Q12NA) We discuss ways to stimulate students' science skills. Q16NA) I discuss lessons with other science teachers even outside official meetings and conferences. Q17NA) We exchange teaching material. Q19NA) Our discussions mostly focus on the achievements of our students. Q21NA) We discuss strategies for coping with professional problems. Q22NA) We talk in an open manner about our experiences with classroom teaching.
TC033	Note: Responses Q01NA, Q02NA, Q03NA, and Q07NA have been deleted. To what extent can (or could) you do the following? Q01NA) Explain basic scientific ideas (such as energy or photosynthesis) to my science class Q02NA) Design good test questions for my students Q03NA) Provide an alternative explanation, for example when students are confused Q07NA) Identify science related misconceptions through scrutiny of student work
TC034	Note: Responses Q03NA, Q05NA, and Q07NA have been deleted. To what extent can (or could) you do the following? Q03NA) Pass an entrance exam for a science bachelor’s course at college Q05NA) Recommend high-quality science presentations in the media (TV, magazines) to my students Q07NA) Use formal models to explain scientific phenomena

2. Deleted Items

Note that TC003, 008, 009, 010, 011, 016, 017, 019, 022, 023, 024, and 025 are general and science teacher questionnaire common items that have been deleted.

TC003	Have you taught 10th grade in the last 12 months?
TC008	Did you submit a written application for the position as a teacher directly to your school?
TC009	You did not submit an application to your school directly. Did the district or other governing agency assign you to your position?
TC010	Was your current position at your school your first choice?
TC011	How important were the following reasons for applying to work at this particular school?
TC016	You attended a standard teacher education or training program. What level of formal education did you reach there?
TC017	For how many months did you attend the teacher education or training program?
TC019	During the last 12 months, did you participate in any of the following professional development activities, and if yes, for how many days did they last?
TC022	How many days within a school year are you required to take part in professional development activities?
TC023	How does your school support your professional development activities?
TC024	Teachers play an important role in communication with parents. From your perspective, to what extent are the following topics of parent-teacher communication important?
TC025	To what extent are the following ways of communicating with your students’ parents important to you?
TC027	To what extent do you disagree or agree with the following statements about your teaching practices?
TC032	For the following pairs of statements, please choose the response that is more prevalent in your school.
TC035	Below you will find descriptions of four science teachers. Please read each of the descriptions of these teachers, then state to what extent you disagree or agree with the highlighted final statement about the respective teacher.
TC036	Keeping your main science subject in mind, how much do you disagree or agree with the following statements?
TC038	How often do these situations occur in your science classes?
TC040	Does the curriculum for science for 10th grade include the following topics?
TC042	In which format is the intended science curriculum for 10th grade made available to teaching staff?
TC044	Are students informed about the availability and content of the science curriculum (e.g., during a school assembly or in a newsletter)?

12